Distortion sensing method and device based on remote sensing image super-resolution quality

Abstract

The invention relates to the technical field of image processing, in particular to a distortion sensing method and device based on remote sensing image super-resolution quality, and the method comprises the steps: generating a global prior vector related to distortion based on a de-mean contrast normalization coefficient modulated by fusion gradient information of an original remote sensing image; using a pre-constructed blind super-resolution network to differentially train the global prior vector to generate a distortion enhancement representation; extracting a multi-channel feature map of the original remote sensing image according to the distortion enhancement representation and a pre-constructed blind super-resolution network, and calculating a query sequence of the multi-channel feature map by using a pre-designed distortion map so as to calculate attention maps of distortion features of the original remote sensing image in multiple feature dimensions and fuse the attention maps; and generating a multi-dimensional distortion feature map of the original remote sensing image, and finally reconstructing the super-resolution remote sensing image. According to the method, the defects of the neural network in global structure modeling can be overcome, the super-division performance is effectively improved, and the calculation burden of modeling is reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a distortion perception method and device based on super-resolution quality of remote sensing images. BACKGROUND

[0002] In related technologies, the super-resolution task of a single remote sensing image is a typical inverse problem, which is highly coupled with the degradation model and has an inevitable ill-conditioned nature, which needs to be studied in depth at a low-level vision level. Therefore, the super-resolution method of remote sensing images based on deep learning mostly relies on an assumed or fixed degradation process, jointly models the estimation of the degradation kernel and the non-blind super-resolution algorithm. This method needs to estimate the parameters of the degradation process, i.e., the Gaussian blur kernel, from the input LR (Low-Resolution) image, and then generate the final SR (Super-Resolution) image according to the estimated results, but these methods are very sensitive to degradation estimation. When the real-world degradation does not match the assumption, the mismatch of the blur kernel will produce SR reconstruction errors, and the deep neural network will further amplify this error layer by layer, resulting in performance degradation, such as the occurrence of unexpected artifacts, which in turn leads to a serious decline in the performance of these super-resolution reconstruction methods based on data mapping.

[0003] However, in related technologies, the blind super-resolution method still relies on explicit degradation estimation or iterative schemes to handle various unknown types of degradation, and does not take into account the characteristics of remote sensing images with many edges and complex structures. At the same time, the degradation modeling process itself will bring a large computational burden, and the accumulation of time consumption and estimation error will inevitably lead to limited SR performance, which needs to be solved urgently. SUMMARY

[0004] The present application provides a distortion perception method and device based on super-resolution quality of remote sensing images to solve the problems in related technologies that the blind super-resolution method still relies on explicit degradation estimation or iterative schemes to handle various unknown types of degradation, and does not take into account the characteristics of remote sensing images with many edges and complex structures. At the same time, the degradation modeling process itself will bring a large computational burden, and the accumulation of time consumption and estimation error will inevitably lead to limited SR performance.

[0005] The first aspect embodiment of the application provides a distortion perception method based on remote sensing image super-resolution quality, including the following steps: based on the image gradient of the original remote sensing image, calculating the fusion gradient information modulated de-mean contrast normalization coefficient of the original remote sensing image, to generate a global prior vector related to the distortion of the original remote sensing image based on the fusion gradient information modulated de-mean contrast normalization coefficient; using a pre-constructed blind super-resolution network to distinguish the global prior vector to generate a distortion enhanced representation for statistically expressing the degradation characteristics of the original remote sensing image and identifying the degradation mode of the original remote sensing image; according to the distortion enhanced representation and the pre-constructed blind super-resolution network, extracting the multi-channel feature map of the original remote sensing image, and using a pre-designed distortion map to calculate the query sequence of the multi-channel feature map, to calculate the attention map of the distortion characteristics of the original remote sensing image in multiple feature dimensions based on the query sequence, and fuse the attention map of the distortion characteristics of the multiple feature dimensions to generate a multi-dimensional distortion feature map of the original remote sensing image.

[0006] Optionally, in an embodiment of the application, the fusion gradient information modulated de-mean contrast normalization coefficient of the original remote sensing image is calculated based on the image gradient, including: based on the image gradient, determining a modulation factor of the standard deviation of the image gradient; based on the image gradient and the modulation factor, calculating the fusion gradient information modulated de-mean contrast normalization coefficient of the original remote sensing image.

[0007] Optionally, in an embodiment of the application, the global prior vector related to the distortion of the original remote sensing image is extracted based on the fusion gradient information modulated de-mean contrast normalization coefficient, including: respectively acquiring the product empirical distribution of the adjacent fusion gradient information modulated de-mean contrast normalization coefficients of the original remote sensing image in the vertical direction, the horizontal direction, the main diagonal direction and the sub-diagonal direction; based on the product empirical distribution, generating the interference statistical characteristics of the distortion of the original remote sensing image in multiple directions of the original remote sensing image; combining the fusion gradient information modulated de-mean contrast normalization coefficient and the interference statistical characteristics, extracting the global prior vector related to the distortion of the original remote sensing image.

[0008] Optionally, in an embodiment of the application, the expression of the pre-designed distortion map is:

[0009]

[0010]

[0011] wherein, The first feature map represents the first feature map. There are 1 channel, total One channel, and These represent the height and width of the feature map, respectively. These are the position coordinates of the original feature map. Indicates the current The 3x3 neighborhood window, centered on the location and extended one pixel up, down, left, and right, is used to predict the distortion map. These are the coordinates of each position within the currently opened 3x3 neighborhood window. In feature map First Feature matrix of each channel To be within the currently opened window Position at No. Feature map matrix of each channel In order to be in The first in the local window area The average value of the feature map matrix of each channel. Represents the entire distortion image In the window opening area Distortion channel component map of each channel.

[0012] Optionally, in one embodiment of this application, the method further includes: obtaining the correlation between the multidimensional distortion feature map and multiple feature channels based on the multidimensional distortion feature map; segmenting the multidimensional distortion feature map in the channel dimension according to the correlation to filter out at least one first-layer feature to be processed and at least one first-layer retained feature; segmenting the first-layer feature to be processed to obtain at least one second-layer feature to be processed and at least one second-layer retained feature, until a preset stopping condition is reached to obtain multiple retained features; and repairing the multiple retained features to reconstruct the super-resolution remote sensing image of the original remote sensing image.

[0013] The second aspect embodiment of the application provides a distortion perception device based on remote sensing image super-resolution quality, comprising: a calculation module configured to calculate a fusion gradient information modulated de-mean contrast normalization coefficient of an original remote sensing image based on an image gradient of the original remote sensing image, to generate a global prior vector related to distortion of the original remote sensing image based on the fusion gradient information modulated de-mean contrast normalization coefficient; a generation module configured to distinguish the global prior vector using a pre-constructed blind super-resolution network to generate a distortion enhanced representation for statistically expressing degradation characteristics of the original remote sensing image and identifying a degradation mode of the original remote sensing image; and a processing module configured to extract a multi-channel feature map of the original remote sensing image according to the distortion enhanced representation and the pre-constructed blind super-resolution network, to calculate a query sequence of the multi-channel feature map using a pre-designed distortion map, to calculate an attention map of distortion characteristics of the original remote sensing image in multiple feature dimensions based on the query sequence, and to generate a multi-dimensional distortion feature map of the original remote sensing image by fusing the attention maps of the distortion characteristics of the multiple feature dimensions.

[0014] Optionally, in an embodiment of the application, the calculation module comprises a determination unit configured to determine a modulation factor of a standard deviation of the image gradient based on the image gradient, and a calculation unit configured to calculate the fusion gradient information modulated de-mean contrast normalization coefficient of the original remote sensing image based on the image gradient and the modulation factor.

[0015] Optionally, in an embodiment of the application, the calculation module comprises an acquisition unit configured to acquire a product empirical distribution of adjacent fusion gradient information modulated de-mean contrast normalization coefficients of the original remote sensing image in a vertical direction, a horizontal direction, a main diagonal direction and a sub-diagonal direction, respectively, a generation unit configured to generate interference statistical characteristics of distortion of the original remote sensing image in multiple directions of the original remote sensing image based on the product empirical distribution, and an extraction unit configured to extract the global prior vector related to distortion of the original remote sensing image in combination with the fusion gradient information modulated de-mean contrast normalization coefficient and the interference statistical characteristics.

[0016] Optionally, in an embodiment of the application, an expression of the pre-designed distortion map is:

[0017]

[0018]

[0019] wherein, represents a first channel of a feature map, and there are a total of C channels. ​One channel, and These represent the height and width of the feature map, respectively. These are the position coordinates of the original feature map. Indicates the current The 3x3 neighborhood window, centered on the location and extended one pixel up, down, left, and right, is used to predict the distortion map. These are the coordinates of each position within the currently opened 3x3 neighborhood window. In feature map First Feature matrix of each channel To be within the currently opened window Position at No. Feature map matrix of each channel In order to be in The first in the local window area The average value of the feature map matrix of each channel. Represents the entire distortion image In the window opening area Distortion channel component map of each channel.

[0020] Optionally, in one embodiment of this application, it further includes: a filtering module, configured to obtain the correlation between the multidimensional distortion feature map and multiple feature channels based on the multidimensional distortion feature map, and segment the multidimensional distortion feature map in the channel dimension according to the correlation to filter out at least one first-layer feature to be processed and at least one first-layer retained feature; a segmentation module, configured to segment the first-layer feature to be processed to obtain at least one second-layer feature to be processed and at least one second-layer retained feature, until a preset stopping condition is reached to obtain multiple retained features; and a reconstruction module, configured to repair the multiple retained features to reconstruct the super-resolution remote sensing image of the original remote sensing image.

[0021] A third aspect of this application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the distortion sensing method based on the super-resolution quality of remote sensing images as described in the above embodiments.

[0022] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described distortion sensing method based on the super-resolution quality of remote sensing images.

[0023] A fifth aspect of this application provides a computer program product, including a computer program that, when executed, is used to implement the above-described distortion sensing method based on the super-resolution quality of remote sensing images.

[0024] The embodiment of the present application can generate a global prior vector related to distortion of the original remote sensing image based on the fusion gradient information modulation de-mean contrast normalization coefficient of the original remote sensing image, and generate a distortion enhancement representation by using a pre-constructed blind super-resolution network, so as to generate a multi-dimensional distortion feature map of the original remote sensing image according to the distortion enhancement representation and the pre-constructed blind super-resolution network, and reconstruct a super-resolution remote sensing image of the original remote sensing image. In this way, the local light difference is suppressed and the linear correlation of the neighborhood pixels is reduced by using the fusion gradient modulation de-mean contrast normalization coefficient MSCN G, the characteristics of the edge of the remote sensing image are fully considered, the distribution is closer to the high-strength, and the global prior information is used to supplement the shortage of the neural network in the global structure modeling, so that the super-resolution performance is effectively improved. The corresponding distortion enhancement representation De-R is estimated implicitly by contrast learning, and is embedded into the pre-constructed blind super-resolution SR network containing multiple distortion decoders, so as to ensure that the present application can match different forms of distortion. Further, the distortion perception self-attention mechanism guided channel segmentation strategy DCS in the present application can further reduce the parameter amount and realize light weight without affecting the feature extraction, and greatly reduce the calculation burden. In this way, the problems in the related art that the blind super-resolution method still relies on explicit degradation estimation or iterative scheme to process various unknown types of degradation, and the characteristics of the edge of the remote sensing image are not considered, and the degradation modeling process itself will bring a large calculation burden, and the accumulation of time consumption and estimation error will inevitably lead to the problem that the SR performance is limited are solved.

[0025] Additional aspects and advantages of the present application will be in part apparent and in part pointed out hereinafter. BRIEF DESCRIPTION OF DRAWINGS

[0026] The above and / or additional aspects and advantages of the present application will become apparent and be readily appreciated from the following description, including the accompanying drawings, in which: Figure 1 A flowchart of a distortion perception method based on remote sensing image super-resolution quality according to an embodiment of the present application; Figure 2 A schematic diagram of a global prior extractor based on MSCN G coefficient of an embodiment of the present application; Figure 3 A modeling process schematic diagram of a distortion enhancement representation based on contrast learning of an embodiment of the present application; Figure 4 A structure schematic diagram of a pre-constructed blind super-resolution network of an embodiment of the present application; Figure 5A structural diagram of a distortion-aware self-attention (DAA) module according to an embodiment of the present application; Figure 6 A structural diagram of a distortion-aware device based on remote sensing image super-resolution quality according to an embodiment of the present application; Figure 7 A structural diagram of an electronic device according to an embodiment of the present application.

[0027] Reference signs: 10 - distortion-aware device based on remote sensing image super-resolution quality; 100 - a computing module, 200 - a generating module, and 300 - a processing module; 701 - a memory, 702 - a processor, and 703 - a communication interface. DETAILED DESCRIPTION

[0028] Embodiments of the present application are described in detail below with reference to examples illustrated in the accompanying drawings, in which the same or similar reference signs represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by reference to the accompanying drawings are exemplary and are intended to explain the present application, and cannot be understood as limiting the present application.

[0029] A method and device for distortion perception based on super-resolution quality of remote sensing images are described below with reference to the accompanying drawings. In the related art mentioned in the background, blind super-resolution methods still rely on explicit degradation estimation or iterative schemes to handle various unknown types of degradation, and do not take into account the characteristics of remote sensing images with many edges and complex structures. At the same time, the degradation modeling process itself will bring a large computational burden, and the accumulation of time consumption and estimation error will inevitably lead to limited SR performance. The present application provides a method for distortion perception based on super-resolution quality of remote sensing images. In this method, a global prior vector related to the distortion of the original remote sensing image can be generated based on the de-mean contrast normalization coefficient modulated by the fusion gradient information of the original remote sensing image. The distortion-enhanced representation is generated using a pre-constructed blind super-resolution network. Then, a multi-dimensional distortion feature map of the original remote sensing image is generated according to the distortion-enhanced representation and the pre-constructed blind super-resolution network, and a super-resolution remote sensing image of the original remote sensing image is reconstructed. In this way, the MSCN_G coefficient modulated by the fusion gradient is used to suppress local illumination differences and reduce the linear correlation of neighboring pixels, fully considering the characteristics of remote sensing images with many edges and complex structures, making the distribution closer to the high-strength, and serving as global prior information to supplement the deficiency of neural networks in global structure modeling, effectively improving the super-resolution performance. Through contrast learning, the corresponding distortion-enhanced representation De-R is implicitly estimated and embedded into the SR network containing multiple distortion decoders to ensure that the present application can match different forms of distortion. Further, the distortion perception self-attention mechanism guided channel segmentation strategy DCS in the present application can further reduce the parameter amount and realize lightweight without affecting feature extraction, greatly reducing the computational burden. In this way, the problems in the related art, such as the blind super-resolution method still relying on explicit degradation estimation or iterative scheme to handle various unknown types of degradation, not taking into account the characteristics of remote sensing images with many edges and complex structures, and the degradation modeling process itself bringing a large computational burden, the accumulation of time consumption and estimation error inevitably leading to limited SR performance, etc. are solved.

[0030] Specifically, Figure 1 A flowchart of a method for distortion perception based on super-resolution quality of remote sensing images is provided in the present application.

[0031] As Figure 1 shown, the method for distortion perception based on super-resolution quality of remote sensing images includes the following steps: In step S101, based on the image gradient of the original remote sensing image, the de-mean contrast normalization coefficient modulated by the fusion gradient information of the original remote sensing image is calculated, and the global prior vector related to the distortion of the original remote sensing image is generated based on the de-mean contrast normalization coefficient modulated by the fusion gradient information.

[0032] Those skilled in the art can understand that in the statistical modeling of natural images, the structural dependency between local regions is considered as an important indicator reflecting the "naturalness" of the image. Natural images will show stable local structure rules under distortion-free conditions. However, various distortions (such as blur, noise, compression artifacts, etc.) will break these rules, resulting in statistical bias.

[0033] By extracting the mean subtracted contrast normalized (MSCN) coefficient of the original image, the "unnaturalness" in the degraded image can be effectively captured, that is, the brightness (mean) and contrast (standard deviation) information of the local image is filtered out, and the remaining coefficient mainly reflects the texture and structure information of the image, and then reflects the distortion type and degree.

[0034] The MSCN coefficient is mainly obtained by local mean subtraction to obtain a local normalized brightness with decorrelation effect, that is, a certain transformation is performed on the image brightness, wherein the calculation formula of the MSCN coefficient can be but is not limited to expressed as follows:

[0035] wherein, is the local mean, is the local variance, and the constant is generally set to 1, is the image brightness.

[0036] Although the traditional MSCN coefficient can effectively weaken the linear correlation between the pixel and its neighborhood by normalization, its response capability is limited when facing typical non-stationary images such as remote sensing images which have a large number of edges and texture details.

[0037] In some embodiments, in order to better adapt to the image characteristics of remote sensing images, the embodiments of the present application construct the MSCN_G coefficient containing gradient information modulation on the basis of considering the image gradient of the original remote sensing image, that is, the mean subtracted contrast normalized coefficient fusing gradient information modulation in the embodiments of the present application.

[0038] Wherein, the original remote sensing image can be understood as the initial image data which is not subjected to any artificial intervention or data processing after the electromagnetic radiation signal reflected / emitted by the ground object is directly received and recorded by the remote sensing sensor, and is the basis for all subsequent remote sensing image processing and application. The image gradient of the remote sensing image can be understood as the "difference degree" and "difference direction" of the gray values between adjacent pixels in the remote sensing image; the higher the gradient value, the more obvious the edge / structure at the position of the pixel; the lower the gradient value, the smoother the image, that is, there is no obvious structure.

[0039] Specifically, in the method of explicitly estimating the blur kernel, the statistical characteristics of the natural image are destroyed due to various distortions, and the locally normalized brightness coefficients after the nonlinear transformation of the brightness can implicitly quantify the potential loss in the image. Due to high light changes, complex texture structures and multi-scale features, the statistical distribution of the remote sensing image often presents non-Gaussian and nonlinear dependence.

[0040] Therefore, the embodiment of the present application can make the distribution of the fusion gradient information modulated de-mean contrast normalization coefficient closer to the Gaussian characteristic on the basis of suppressing the local illumination difference in the original remote sensing image and reducing the linear correlation of the neighborhood pixels.

[0041] Therefore, the embodiment of the present application can make the distribution of the fusion gradient information modulated de-mean contrast normalization coefficient closer to the Gaussian characteristic on the basis of suppressing the local illumination difference in the original remote sensing image and reducing the linear correlation of the neighborhood pixels.

[0042] By fusing the gradient information modulated de-mean contrast normalization coefficient to normalize the original remote sensing image, the perception ability of the super-resolution neural network to the non-stationary area can be enhanced, and the dependence between the neighborhoods is further reduced, and the distortion perception is more sensitive.

[0043] Among them, the calculation formula of the MSCN_G coefficient in the embodiment of the present application can be but not limited to defined as follows:

[0044] Among them, is a modulation factor of the gradient of the remote sensing image is one of the coefficients in is the gradient of each point in the remote sensing image.

[0045] Therefore, in the actual application process, the embodiment of the present application only needs to calculate the fusion gradient information modulated de-mean contrast normalization coefficient of the original remote sensing image based on the image gradient of the original remote sensing image, and can capture the global statistical characteristics of the original remote sensing image based on the fusion gradient information modulated de-mean contrast normalization coefficient, thereby generating a global prior related to the distortion of the original remote sensing image and a corresponding global prior vector, and then using the global prior to improve the super-resolution performance of the super-resolution neural network.

[0046] Optionally, in an embodiment of the present application, the fusion gradient information modulated de-mean contrast normalization coefficient of the original remote sensing image is calculated based on image gradients of the original remote sensing image, comprising: determining a modulation factor of a standard deviation of the image gradients based on the image gradients; and calculating the fusion gradient information modulated de-mean contrast normalization coefficient of the original remote sensing image based on the image gradients and the modulation factor.

[0047] As can be understood based on the related description in other embodiments, the calculation formula of the fusion gradient information modulated de-mean contrast normalization coefficient in the embodiment of the present application can be but is not limited to expressed as:

[0048] wherein, is the fusion gradient information modulated de-mean contrast normalization coefficient, is the image brightness, is the local mean, is the local variance, and the constant is generally set to 1, is a modulation factor of the remote sensing image gradient is one of the coefficients, is the gradient of each point in the remote sensing image.

[0049] In actual execution, the calculation formula of the fusion gradient information modulated de-mean contrast normalization coefficient in the embodiment of the present application selects the function with good performance to process the response of the gradient in the original remote sensing image to modulate the standard deviation, so as to ensure that the high gradient area (such as the edge) can perceive the strong structural change while not having too strong influence on the standard deviation calculation; the low gradient area (such as the flat area) is not affected and the original features are retained.

[0050] Therefore, in the process of actually calculating the fusion gradient information modulated de-mean contrast normalization coefficient of the original remote sensing image, the modulation factor of the standard deviation of the image gradient of each position in the original remote sensing image can be determined based on the image gradient of the position, so as to calculate the fusion gradient information modulated de-mean contrast normalization coefficient of the original remote sensing image in combination with the image gradient and the modulation factor.

[0051] For example, when processing the area with a large gradient value in the original remote sensing image, the result amplitude is reduced and the edge change is highlighted, but even if it approaches infinity, the modulation factor is only close to (1- ); and when processing the area with a small gradient value in the original remote sensing image, the modulation factor approaches 1, so that the standard deviation is basically not affected, thereby effectively retaining the original features of the original remote sensing image.

[0052] It should be noted that, The specific value can be adjusted and determined by technical personnel in the technical field through experiments and other technical means, and in the embodiment of the present application, it can be taken as 9.6 empirically, but the embodiment of the present application is only exemplary and not limited.

[0053] The fusion gradient information modulation de-mean contrast normalization coefficient in the embodiment of the present application effectively fuses the gradient modulation mechanism, so that the normalization amplitude can be appropriately suppressed in the image high gradient area, so that the model can more accurately distinguish "distortion" and "natural structure change", effectively realize the decoupling of natural structure and distortion information, thereby improving the perception and discrimination ability of image distortion, and reducing the false response of the edge area.

[0054] In addition, the global prior constructed based on the fusion gradient information modulation de-mean contrast normalization coefficient can guide the super-resolution neural network (model) to better identify and retain the true edge structure, and avoid misjudging the key details of the original remote sensing image as distortion and being incorrectly processed.

[0055] Optionally, in an embodiment of the present application, based on the fusion gradient information modulation de-mean contrast normalization coefficient, a global prior vector related to the distortion of the original remote sensing image is extracted, including: respectively acquiring the product empirical distribution of the adjacent fusion gradient information modulation de-mean contrast normalization coefficients of the original remote sensing image in the vertical direction, the horizontal direction, the main diagonal direction and the auxiliary diagonal direction; based on the product empirical distribution, generating the interference statistical characteristics of the distortion of the original remote sensing image in multiple directions of the original remote sensing image; combining the fusion gradient information modulation de-mean contrast normalization coefficient and the interference statistical characteristics, extracting the global prior vector related to the distortion of the original remote sensing image.

[0056] In other embodiments, considering that the regular structure of some adjacent coefficients will be destroyed when the MSCN_G coefficient is subjected to different degrees of distortion interference of environmental factors, the present application can respectively acquire the product empirical distribution of the adjacent fusion gradient information modulation de-mean contrast normalization coefficients of the original remote sensing image in the vertical direction, the horizontal direction, the main diagonal direction and the auxiliary diagonal direction.

[0057] Then, the embodiment of the present application can further refine the interference statistical characteristics of the distortion of the original remote sensing image in different directions according to the acquired product empirical distribution, and generate the interference statistical characteristics of the distortion of the original remote sensing image in multiple different directions of the original remote sensing image.

[0058] Specifically, the modeling formula of the interference statistical characteristics in the vertical direction, the horizontal direction, the main diagonal direction and the auxiliary diagonal direction in the embodiment of the present application can be but not limited to expressed as follows:

[0059]

[0060]

[0061]

[0062] in, Indicates the vertical direction. Indicates the horizontal direction. Indicates the main diagonal. Indicates the secondary diagonal. , , , These are all mean-free contrast normalization coefficients modulated by the fused gradient information at the corresponding positions.

[0063] Furthermore, by combining the mean-free contrast normalization coefficient of the fused gradient information modulation and the statistical characteristics of the interference in different directions, the embodiments of this application can extract the global prior related to the distortion of the original remote sensing image.

[0064] Furthermore, in order to achieve adaptive processing of various complex distortion types, this application also designs a distortion encoder—D-Encoder—that can extract natural image statistical characteristics as global priors. Figure 2 This is a schematic diagram of a global prior extractor based on the MSCN_G coefficients according to an embodiment of this application. Figure 2 As shown, the D-Encoder in this embodiment consists of two parts: a global prior extractor and an encoding network.

[0065] In practical applications, the embodiments of this application can modulate the mean-free contrast normalization coefficient (brightness) with fused gradient information containing distortion statistical characteristics. The statistical features of interference in various directions are concatenated together and input into the global prior extractor. The global prior extractor extracts multi-scale global prior vectors related to distortion through the spatial pyramid pooling layer. Then, it is input into a small 3-layer encoder to obtain the global prior g related to distortion.

[0066] Additionally, a small 3-layer encoder can be used with a distortion representation vector coding network in a D-Encoder. First, train it separately for 150 rounds, and then train it together with the entire blind SR network so that the extracted global prior g related to distortion can be better adapted to the blind SR task.

[0067] Step S102, the pre-constructed blind super-resolution network is used to distinguish the training global prior vector to generate a distortion-enhanced representation for statistically expressing the degradation characteristics of the original remote sensing image and identifying the degradation mode of the original remote sensing image.

[0068] As a possible implementation manner, in order to further improve the discrimination ability of the global prior on the characterization of distortion type, and make it more capable of distinguishing different degradation types in remote sensing images, the application can adopt the idea of "similar attraction, different repulsion" in contrast learning, and introduce a contrast learning framework based on InfoNCE loss to structure the distortion representation space.

[0069] Among them, the distortion representation space can be understood here as a mathematical space composed of high-dimensional feature vectors after various distortions (such as blur, noise, compression distortion, etc.) in the image are converted into these vectors. Each "point" in the distortion representation space corresponds to a distortion feature vector, and the distance or positional relationship between points in the space reflects the "similarity" of different distortion types.

[0070] Specifically, unlike the traditional setting of directly encoding degradation from image content, the embodiment of the application can directly distinguish and train the global prior vector corresponding to the global prior g in the distortion representation space of the original remote sensing image to generate a distortion-enhanced representation (De-R) for statistically expressing the degradation characteristics of the original remote sensing image and identifying the degradation mode of the original remote sensing image.

[0071] It should be noted that the distortion-enhanced representation vector in the embodiment of the application is a mathematical expression of the distortion-enhanced representation in neural network processing, which has the same essence as the distortion-enhanced representation.

[0072] By directly distinguishing and training the global prior vector corresponding to the global prior g in the distortion representation space of the original remote sensing image, the obtained distortion-enhanced representation not only has the statistical expression ability of image degradation, but also has the ability to distinguish different degradation modes under the driving of contrast learning.

[0073] Specifically, the embodiment of the application pre-constructs a blind super-resolution network for global prior guided distortion-enhanced representation learning. The key module D-Encoder distortion encoder in the blind super-resolution network can be used to generate a distortion-enhanced representation De-R. Figure 3 The modeling process of the distortion-enhanced representation based on contrast learning of an embodiment of the application is shown in the figure. Figure 3 As shown in the figure, Figure 3 Specifically, the generation process of the distortion-enhanced representation De-R based on contrast learning in the embodiment of the application is shown.

[0074] In a containing In the training batch of LR images (which can be understood as original remote sensing images) corresponding to different distortions, the embodiments of this application can first extract two non-overlapping patches from each image and calculate their corresponding global distortion feature vectors. and ,in =1, 2, 3…, .

[0075] In comparative learning, Treated as a query vector As positive samples, the distortion vectors of the remaining patches in the image As negative samples, after passing through five encoder layers and an MLP, De-R can be generated. The generated De-R can be, but is not limited to, denoted as: , , .

[0076] It should be noted that, in this embodiment, it is assumed that the distortion type in the patch extracted from the same image is the same, while the distortion type between patches from different images is different. Furthermore, this embodiment also constructs an InfoNCE loss function to constrain the distortion representation space, which can be defined, but is not limited to, as follows:

[0077] in, For temperature hyperparameters, the dot product operation is used to measure the similarity between vectors; This is a dynamically updated sample queue maintained to enhance training stability and degradation diversity, used to store negative sample vectors from different images and degradation conditions. In the embodiments of this application, based on experience with existing methods, and They can be set to 8192 and 0.07 respectively, but are not limited to.

[0078] Considering the limitation on the number of channels in the global prior output, this embodiment of the application can set the encoding network in the distortion encoder D-Encoder to five layers, directly feed the obtained distortion enhancement representation into the five-layer feature encoding network to obtain the distortion enhancement representation vector De-R, and then feed the distortion enhancement representation vector into a two-layer perceptron (MLP). The MLP network consists of two fully connected layers and an intermediate LeakyReLU activation function, which is used to complete the projection into the contrastive learning latent space.

[0079] Therefore, the embodiment of the present application can map the global prior g derived from the image statistical structure and sensitive to degradation to a feature space with stronger discrimination, so that the distortion enhanced representation De-R with discrimination and stability is obtained after contrast learning, which is embedded into the subsequent entire blind super-resolution network, thereby helping to guide the super-resolution reconstruction process and improving the super-resolution performance of the entire blind super-resolution network.

[0080] In step S103, a multi-channel feature map of the original remote sensing image is extracted according to the distortion enhanced representation and the pre-constructed blind super-resolution network, a query sequence of the multi-channel feature map is calculated using a pre-designed distortion map, an attention map of the distortion features of the original remote sensing image in multiple feature dimensions is calculated based on the query sequence, and a multi-dimensional distortion feature map of the original remote sensing image is generated by fusing the attention maps of the distortion features in multiple feature dimensions.

[0081] In some embodiments, after obtaining the distortion enhanced representation guided by the global prior based on the original remote sensing image, the present application can further extract a multi-channel feature map of the original remote sensing image through the pre-constructed blind super-resolution network according to the distortion enhanced representation, that is, a feature map of multiple channels of the original remote sensing image extracted by the pre-constructed blind super-resolution network.

[0082] Then, the present application embodiment can calculate a query sequence of the multi-channel feature map using a pre-designed distortion map, so as to calculate an attention map of the distortion features of the original remote sensing image in multiple feature dimensions with respect to the original remote sensing image according to the query sequence of the multi-channel feature map, and fuse the attention maps of the distortion features in multiple feature dimensions, thereby generating a multi-dimensional distortion feature map of the original remote sensing image.

[0083] Specifically, Figure 4 The structure diagram of the pre-constructed blind super-resolution network of an embodiment of the present application is shown. As shown in Figure 4 The present application embodiment can generate a distortion enhanced representation De-R based on a D-Encoder distortion encoder in the pre-constructed blind super-resolution network (GDANet), and the pre-constructed blind super-resolution network also contains a DCSA module of a distortion perception self-attention mechanism guided channel segmentation strategy.

[0084] The D-Encoder distortion encoder can send the global prior g extracted before into the five-layer feature encoding network The distortion enhanced representation vector De-R is generated, and the De-R first enters the distortion feature decoder D-Decoder in the DCSA module.

[0085] The embodiment of the present application introduces a distortion-aware self-attention module DAA in the D-Decoder. Figure 5 The structural diagram of the distortion-aware self-attention DAA module of an embodiment of the present application is shown in FIG. 1. Figure 5 As shown in FIG. 1, in the main branch of the D-Decoder, the tensor of De-R after 3x3 convolution is input into the distortion-aware self-attention module DAA.

[0086] It can be understood by those skilled in the art that the self-attention mechanism (SA) can efficiently expand the receptive field and more accurately capture the key features of the original image to achieve efficient super-resolution reconstruction. However, in the widely used standard attention mechanism, the network based on the self-attention mechanism (SA) pays more attention to the areas where the distortion features are slightly expressed, because these areas have higher correlation with the original clean image.

[0087] In order to break through this traditional limitation and make the network pay attention to all areas with distortion features, whether the distortion features are slightly expressed or severely expressed, the embodiment of the present application proposes a distortion-aware self-attention mechanism, i.e., a distortion-aware self-attention module DAA, which can make the feature map pay more attention to the local areas affected by different degrees of distortion before being input into the subsequent module.

[0088] By introducing the distortion-aware self-attention mechanism in the distortion-aware self-attention module, the embodiment of the present application can directly calculate the query sequence (Query) by using the distortion map estimated in advance through the DAA module, and can generate the set of Key and Value based on the input feature map The generation of Key and Value can be learned by a learnable 1x1 convolution. The convolution expression of Query, Key and Value can be but not limited to represented as follows:

[0089]

[0090]

[0091] Further, the embodiment of the present application can calculate the distortion feature-based attention map Atten (Distortion Attention Map) of the original remote sensing image in multiple feature dimensions based on the calculated query sequence, where the distortion feature-based Query can help the self-attention mechanism understand the related information of the distortion level to some extent, and provide more discriminative input features for the subsequent designed channel segmentation strategy DCS. The expression of the distortion perception self-attention mechanism in the embodiment of the present application can be but is not limited to as follows:

[0092] wherein, is an input tensor, and Atten is a distortion feature-based attention map.

[0093] Meanwhile, in order to enhance the network's ability to capture different distortion models, the embodiment of the present application also sets a multi-head attention in the distortion perception self-attention module DAA, which groups different feature dimensions into multiple heads to allow each head to learn different attention, and then projects and aggregates the information from different attention heads by Conv2D into unified representation to obtain the final multi-dimensional distortion feature map, so as to realize multi-dimensional attention to different distortion modes and ensure that the attention will not ignore the important features contained in the distortion region.

[0094] It should be noted that in the embodiment of the present application, the distortion is mainly regarded as a sharp change form different from the global image mode in the local area of the image, and based on this, the root mean square RMS of the local area mean deviation can be approximately calculated as a distortion map Distortion Map to represent the severity of distortion. In the estimation process, the embodiment of the present application can but is not limited to calculate the local area mean is set to 3x3, and the local area mean is calculated by using the average pooling Avgpooling , and then the RMS is calculated position, wherein, and the distortion map of the channel may but is not limited to be as follows:

[0095]

[0096]

[0097] wherein, represents the i-th channel of the feature map, and there are N channels in total, and ​​​These represent the height and width of the feature map, respectively. These are the position coordinates of the original feature map. Indicates the current The 3x3 neighborhood window, centered on the location and extended one pixel up, down, left, and right, is used to predict the distortion map. These are the coordinates of each position within the currently opened 3x3 neighborhood window. In feature map First Feature matrix of each channel To be within the currently opened window Position at No. Feature map matrix of each channel In order to be in The first in the local window area The average value of the feature map matrix of each channel. Represents the entire distortion image In the window opening area The distortion map consists of the distortion channel component maps of each channel, and the sum of all the distortion channel component maps at each location is the final distortion map. .

[0098] Optionally, in one embodiment of this application, the method further includes: obtaining the correlation between the multidimensional distortion feature map and multiple feature channels based on the multidimensional distortion feature map; segmenting the multidimensional distortion feature map in the channel dimension according to the correlation to filter out at least one first-layer feature to be processed and at least one first-layer retained feature; segmenting the first-layer feature to be processed to obtain at least one second-layer feature to be processed and at least one second-layer retained feature, until a preset stopping condition is reached to obtain multiple retained features; and repairing the multiple retained features to reconstruct the super-resolution remote sensing image of the original remote sensing image.

[0099] As one possible implementation, after obtaining the multidimensional distortion feature map, this application embodiment can also use a channel segmentation strategy (DCS) guided by a distortion-aware self-attention mechanism to segment the feature maps of multiple channels layer by layer. In each segmentation process, feature information that is strongly correlated with the distortion of the original remote sensing image is retained. Then, the distortion of the original remote sensing image is further processed through a multi-layer processing module, and finally, a super-resolution remote sensing image of the original remote sensing image is reconstructed.

[0100] Still Figure 4 As shown, it can be understood that the blind super-resolution network pre-built in this application embodiment also includes a DCSA module with a distortion-aware self-attention mechanism-guided channel segmentation strategy. As the main basic module for building the pre-built blind super-resolution network, the DCSA module not only includes a distortion feature decoder (D-Decoder) but also a distortion-guided channel segmentation module (CS module).

[0101] Specifically, after inputting the global prior g into the pre-distortion feature encoder D-Encoder, the distortion feature encoder D-Encoder generates a multi-dimensional distortion feature map while generating a sequence of decision channel weights according to De-R, that is, according to the correlation between the distortion enhancement representation and the plurality of feature channels, generates weights corresponding to the plurality of feature channels, and sorts the feature channels according to the weights, divides the feature channels according to a certain rule according to the sorting sequence, retains the feature channels (corresponding to the feature map) with strong distortion correlation with the original remote sensing image, and transmits the feature channels with weak distortion correlation with the original remote sensing image to the next DCSA module.

[0102] Briefly speaking, the feature map of the distortion feature decoder has a plurality of channels, each of which captures different image features (such as edges, colors, textures, etc.). D-Decoder can analyze De-R, and then determine which feature channels are the most important and most need to be carefully processed and which are secondary according to the specific distortion of the current image, sort them, and obtain the correlation (importance) order between the feature channels and the distortion enhancement representation.

[0103] The importance order of the feature channels can guide the DCSA module to divide the input feature map in the channel dimension, for example, important channels are sent to a more complex and parameter-rich subnetwork for deep processing to repair severe distortion; secondary channels are sent to a lighter subnetwork to save computing power and preserve original information.

[0104] In the embodiments of the present application, the DCSA module includes three CS modules. The three CS modules divide the feature map input by the D-Decoder in the channel dimension, and each division obtains a layer of distortion features (features with high correlation with the distortion (distortion enhancement representation) of the original remote sensing image, received and processed by feature channels sensitive to the distortion) and a layer of preserved features (features matched with a specific distortion after refinement, received and processed by feature channels not so sensitive to the distortion of the original remote sensing image), and finally the channel information (which can be understood as the local repaired remote sensing image after each layer of distortion features and each layer of preserved features are repaired) obtained by the three channel division modules is aggregated and spliced again, that is, a remote sensing image reconstruction result after distortion processing by a DCSA module can be obtained.

[0105] Further, the embodiments of the present application can combine a plurality of DCSA modules, so that the output result (the remote sensing image reconstruction result output by each DCSA module) of each DCSA module is sent to the next layer for further processing and distortion vector more relevant features, while adding local residual error between each DCSA module to alleviate the learning difficulty.

[0106] It should be noted that the specific number of DCSM modules and the number of CS modules can be adjusted and determined by a person skilled in the art according to actual conditions, and the embodiments of the present application are only exemplary and not specifically limited.

[0107] Finally, the shuffle pixel layer is applied through the upsampling strategy, that is, the spatial resolution is reconstructed through the channel information, that is, the super-resolution remote sensing image of the original remote sensing image can be generated.

[0108] For example, the process of obtaining the distortion enhancement representation De-R by the encoding network in the D-Encoder through the global prior g can be but not limited to defined as follows:

[0109] In addition, in order to reduce the feature loss after subsequent channel segmentation, inspired by the adaptive processing of multiple degradations through rescaling different channels before entering the channel segmentation, the channel weight coefficients generated by the D-Decoder through two FC layers and sigmoid activation functions can be used to scale the different channel components input first, realizing explicit channel enhancement based on the distortion enhancement representation De-R.

[0110] Then, the original feature channels can be sorted by the channel weight coefficients, and then the channel segmentation strategy (DCS) guided by the distortion is performed through three CS modules, and the channel segmentation and screening operation is performed layer by layer through the three CS modules in succession, and the network can gradually extract the channel subset more sensitive to the current distortion condition, realizing the adaptive matching of the model feature channels to different distortion forms. This feature screening mechanism improves the model's ability to distinguish complex degradation forms while effectively compressing the parameter quantity in the channel dimension. The distortion enhancement representation De-R extracted based on the MSCN_G parameter modulated by the fusion gradient information can capture the statistical characteristic changes of the image naturalness level due to the distortion, thereby better representing the globality of the remote sensing image, reducing the misjudgment rate of the distortion, and indirectly helping the network to better restore the edge features.

[0111] Each CS module in the DCSA module consists of a channel segmentation layer (CSL) and a 3×3 convolutional layer (CL). The 3×3 convolutional layer is mainly used to extract preliminary features (features with low correlation to image distortion after the first segmentation by the CSL layer (Figure), which are then fed to subsequent CS segmentation modules to find features with relatively high correlation to distortion features among these features with low correlation to image distortion) and adjust the number of channels to achieve layer-by-layer feature refinement. The CSL layer performs segmentation operations on the features of different channels, generating two parts of features. One part of the features is retained (the CL features), and the other part (distortion features) is sent to the next degradation information decoder. The retained part can be regarded as the refined feature that matches the specific distortion. Given input features After the first The process of a CS module can be described as follows:

[0112]

[0113]

[0114]

[0115]

[0116]

[0117] in, Indicates the first Each channel segmentation module Indicates the first A 3×3 convolutional layer, Indicates the first A D-Decoder, Representing the A coarse feature that needs further processing. This represents a refined feature of the output of a channel that has a high degree of matching with a specific distortion. This indicates a splicing operation along the channel dimension. The table shows the full-channel features after channel splicing. Table 1 is a hyperparameter table of the entire channel segmentation architecture in one embodiment of this application, which can be represented as follows: Table 1

[0118] The distortion perception method based on the super-resolution quality of remote sensing images proposed in the embodiments of this application can generate a global prior vector related to the distortion of the original remote sensing image based on the mean-free contrast normalization coefficient modulated by the fusion gradient information of the original remote sensing image, and generate a distortion enhancement representation using a pre-constructed blind super-resolution network. Then, a multi-dimensional distortion feature map of the original remote sensing image is generated based on the distortion enhancement representation and the pre-constructed blind super-resolution network, and the super-resolution remote sensing image of the original remote sensing image is reconstructed. This approach achieves the suppression of local illumination differences and reduction of linear correlation among neighboring pixels by fusing gradient-modulated MSCN_G coefficients. It fully considers the characteristics of remote sensing images, which are characterized by numerous edges and complex structures, making the distribution closer to Gaussian properties. Furthermore, it serves as global prior information to supplement the shortcomings of neural networks in global structure modeling, effectively improving super-resolution performance. The corresponding distortion enhancement representation (De-R) is implicitly estimated through contrastive learning and embedded into an SR network containing multiple distortion decoders, ensuring that this application can match different forms of distortion. Further, the distortion-aware self-attention mechanism-guided channel segmentation strategy (DCS) in this application can further reduce the number of parameters and achieve lightweighting without affecting feature extraction, significantly reducing the computational burden. This addresses the problems in related technologies where blind super-resolution methods still rely on explicit degradation estimation or iterative schemes to handle various unknown types of degradation, failing to consider the characteristics of remote sensing images with numerous edges and complex structures. Additionally, the degradation modeling process itself incurs a significant computational burden, and the accumulation of time and estimation errors inevitably limits SR performance.

[0119] Next, referring to the accompanying drawings, a distortion sensing device based on the super-resolution quality of remote sensing images is described according to an embodiment of this application.

[0120] Figure 6 This is a schematic diagram of the distortion sensing device based on the super-resolution quality of remote sensing images according to an embodiment of this application.

[0121] like Figure 6 As shown, the distortion sensing device 10 based on the super-resolution quality of remote sensing images includes: a calculation module 100, a generation module 200, and a processing module 300.

[0122] The calculation module 100 is used to calculate the mean-free contrast normalization coefficient of the fused gradient information modulation of the original remote sensing image based on the image gradient of the original remote sensing image, and to generate a global prior vector related to the distortion of the original remote sensing image based on the mean-free contrast normalization coefficient of the fused gradient information modulation.

[0123] The generation module 200 is used to distinguish and train global prior vectors using a pre-built blind super-resolution network to generate distortion-enhanced representations that statistically represent degradation features of the original remote sensing image and identify degradation patterns of the original remote sensing image.

[0124] The processing module 300 is configured to extract a multi-channel feature map of the original remote sensing image according to the distortion enhancement representation and the pre-constructed blind super-resolution network, calculate a query sequence of the multi-channel feature map by using a pre-designed distortion map, calculate an attention map of distortion features of the original remote sensing image in a plurality of feature dimensions based on the query sequence, and fuse the attention maps of the distortion features of the plurality of feature dimensions to generate a multi-dimensional distortion feature map of the original remote sensing image.

[0125] Optionally, in an embodiment of the present application, the calculation module 100 comprises a determination unit and a calculation unit.

[0126] The determination unit is configured to determine a modulation factor of a standard deviation of the image gradient based on the image gradient.

[0127] The calculation unit is configured to calculate a fusion gradient information modulation de-mean contrast normalization coefficient of the original remote sensing image based on the image gradient and the modulation factor.

[0128] Optionally, in an embodiment of the present application, the calculation module 100 comprises an acquisition unit, a generation unit and an extraction unit.

[0129] The acquisition unit is configured to respectively acquire a product empirical distribution of adjacent fusion gradient information modulation de-mean contrast normalization coefficients in a plurality of directions of the original remote sensing image.

[0130] The generation unit is configured to generate a distortion-to-interference statistical feature of the original remote sensing image in a plurality of directions of the original remote sensing image based on the product empirical distribution.

[0131] The extraction unit is configured to extract a global prior vector related to the distortion of the original remote sensing image in combination with the fusion gradient information modulation de-mean contrast normalization coefficient and the interference statistical feature.

[0132] Optionally, in an embodiment of the present application, the expression of the pre-designed distortion map can be but is not limited to:

[0133]

[0134]

[0135] wherein, represents the i-th channel of the feature map, and there are C channels in total, and and respectively represent the height and the width of the feature map, ​a position coordinate of the original feature map, representing a 3*3 neighborhood window formed by extending one pixel up, down, left and right of the current position, which is used to predict the distortion map subsequently, a coordinate value of each position in the 3*3 neighborhood window opened at the current, a feature matrix of the first channel at the feature map , a feature map matrix of the first channel at the position in the window opened at the current, an average value of the first channel feature map matrix in the local window region, representing a distortion channel component map of the first channel in the windowed region in the entire distortion map .

[0136] Optionally, in an embodiment of the present application, further comprising: a screening module, a segmentation module and a reconstruction module.

[0137] The screening module is configured to obtain a correlation degree between the multi-dimensional distortion feature map and a plurality of feature channels based on the multi-dimensional distortion feature map, and segment the multi-dimensional distortion feature map in a channel dimension according to the correlation degree, so as to screen out at least one first layer to-be-processed feature and at least one first layer reserved feature.

[0138] The segmentation module is configured to segment the first layer to-be-processed feature to obtain at least one second layer to-be-processed feature and at least one second layer reserved feature, until a preset stop condition is reached, and obtain a plurality of reserved features.

[0139] The reconstruction module is configured to repair the plurality of reserved features to reconstruct a super-resolution remote sensing image of the original remote sensing image.

[0140] It should be noted that the foregoing explanation and description of the embodiment of the distortion perception method based on the super-resolution quality of the remote sensing image also apply to the embodiment of the distortion perception device based on the super-resolution quality of the remote sensing image, which will not be described here again.

[0141] ​​The distortion perception device based on remote sensing image super-resolution quality provided by the embodiment of the application can generate a global prior vector related to distortion of an original remote sensing image based on a de-meaning contrast normalization coefficient modulated by fusion gradient information of the original remote sensing image, and generate a distortion enhancement representation by using a pre-constructed blind super-resolution network, so as to generate a multi-dimensional distortion feature map of the original remote sensing image according to the distortion enhancement representation and the pre-constructed blind super-resolution network, and reconstruct a super-resolution remote sensing image of the original remote sensing image. In this way, the MSCN_G coefficient modulated by the fusion gradient is used to suppress local illumination differences and reduce linear correlation of neighboring pixels, the characteristics of the remote sensing image, such as more edges and complex structure, are fully considered, the distribution is made closer to the high-strength, the global prior information is used to supplement the shortage of the neural network in global structure modeling, the super-resolution performance is effectively improved, the corresponding distortion enhancement representation De-R is implicitly estimated by contrast learning, and the distortion enhancement representation De-R is embedded into the SR network containing multiple distortion decoders, so that the application can match different forms of distortion. Further, the channel segmentation strategy DACS guided by the distortion perception self-attention mechanism can further reduce the parameter quantity and realize light weight without affecting feature extraction, and greatly reduce the calculation burden. In this way, the problems in the related art are solved, such as that the blind super-resolution method still depends on explicit degradation estimation or iterative scheme to process various unknown types of degradation, the characteristics of the remote sensing image, such as more edges and complex structure, are not considered, the degradation modeling process itself brings a large calculation burden, and the accumulation of time consumption and estimation error inevitably leads to limited SR performance.

[0142] Figure 7 The structural schematic diagram of the electronic device provided by the embodiment of the application is provided. The electronic device can include: The memory 701, the processor 702, and the computer program stored in the memory 701 and executable on the processor 702.

[0143] The processor 702 implements the distortion perception method based on remote sensing image super-resolution quality provided in the above embodiment when executing the program.

[0144] Further, the electronic device further includes: The communication interface 703 is used for communication between the memory 701 and the processor 702.

[0145] The memory 701 is used to store the computer program executable on the processor 702.

[0146] The memory 701 can include a high-speed RAM memory, and can also include a non-volatile memory, such as at least one disk memory.

[0147] If the memory 701, the processor 702 and the communication interface 703 are implemented independently, the communication interface 703, the memory 701 and the processor 702 can be connected with each other through a bus and complete communication between each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For convenience of representation, Figure 7 Only one thick line is used to represent the bus in the figure, but it does not mean that there is only one bus or only one type of bus.

[0148] Optionally, in a specific implementation, if the memory 701, the processor 702 and the communication interface 703 are integrated on a chip, the memory 701, the processor 702 and the communication interface 703 can complete communication between each other through an internal interface.

[0149] The processor 702 can be a Central Processing Unit (CPU), or an Application Specific Integrated Circuit (ASIC), or one or more integrated circuits configured to implement one or more embodiments of the present application.

[0150] The embodiment of the present application further provides a computer readable storage medium, which stores a computer program, and the program is executed by a processor to implement the distortion perception method based on super-resolution quality of a remote sensing image as described above.

[0151] The embodiment of the present application further provides a computer program product, which includes a computer program, and the computer program can run computer instructions, and the computer instructions are executed by a processor to implement the distortion perception method based on super-resolution quality of a remote sensing image provided by the embodiment of the present application.

[0152] In the description of the application, reference to "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" means that a particular feature, structure, material, or characteristic being described is included in at least one embodiment or example of the application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment or example. Furthermore, the described specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples. In addition, the usage of "N" means at least two, for example, two, three or the like, unless explicitly stated otherwise.

[0153] Furthermore, the terms "first", "second", or the like, are used merely as a designation of certain elements or features, and do not imply or connote relative importance or a specific order of categorization of the indicated features. Accordingly, features described as "first" or "second" can be explicitly or implicitly included in at least one of the features. In the description of the application, the term "N" means at least two, for example, two, three, etc., unless explicitly stated otherwise.

[0154] Any process or method descriptions or blocks in flow charts or otherwise described herein represent embodiments which can be managed as one or more modules, segments, or portions of code which include one or more executable instructions for implementing specific logic functions or steps, and alternate implementations are possible. In some embodiments, the processes and methods described can be executed by one or more apparatuses or devices, either directly or after conversion to another language. Alternate implementations are possible.

[0155] The logic and / or steps represented in the flowcharts and / or described herein, for example, can be considered as a sequence of executable instructions, and can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor- containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For purposes of this specification, a "computer-readable medium" can be any apparatus that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. The computer-readable medium can be a product of the manufacturing and / or processing. The computer-readable medium can include, but is not limited to, the following: an electronic connection (an electronic device with one or N wires), a portable computer diskette (a magnetic device), a RAM (random access memory), a ROM (read-only memory), an EPROM (erasable programmable ROM) or a Flash memory, an optical fiber, and a portable CD ROM. In addition, the computer-readable medium can even be paper or another suitable medium upon which the program can be printed, because the program can be electronically captured, via the optically scanning of the paper or other medium, then compiled, interpreted, or otherwise processed in a suitable manner, if necessary, and stored in the computer memory.

[0156] It should be understood that aspects of the application can be implemented in hardware, software, firmware or combinations thereof. In the above embodiments, the N steps or methods can be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. If implemented in hardware and in another embodiment, the hardware can be implemented using any or a combination of the following technologies, which are all well known in the art: a discrete logic circuit having logic gates for implementing logic functions on data signals, an application specific integrated circuit having appropriate combinational logic gates, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0157] Those of skill in the art would understand that the steps carried out in the above-mentioned embodiment methods can be implemented by programs instructing relevant hardware to complete all or part of the steps, and the programs can be stored in a computer-readable storage medium. When the programs are executed, they include one or a combination of the steps of the method embodiments.

[0158] In addition, each of the functional units in the various embodiments of the present application can be integrated in one processing module, or each of the units can be physically present separately, or two or more units can be integrated in one module. The integrated module can be implemented in the form of hardware or in the form of a software functional module. When the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer readable storage medium.

[0159] The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it should be understood that the above embodiments are exemplary and should not be construed as limiting the present application, and those skilled in the art can make changes, modifications, replacements and variations to the above embodiments within the scope of the present application.

Claims

1. A distortion perception method based on remote sensing image super-resolution quality, characterized in that, The method comprises the following steps: Based on the image gradient of the original remote sensing image, the fusion gradient information modulated de-mean contrast normalization coefficient of the original remote sensing image is calculated, and the global prior vector related to the distortion of the original remote sensing image is extracted based on the fusion gradient information modulated de-mean contrast normalization coefficient. The global prior vector is trained by using a pre-constructed blind super-resolution network to generate a distortion enhancement representation for statistically expressing the degradation characteristics of the original remote sensing image and identifying the degradation mode of the original remote sensing image. According to the distortion enhancement representation and the pre-constructed blind super-resolution network, the multi-channel feature map of the original remote sensing image is extracted, and the query sequence of the multi-channel feature map is calculated by using a pre-designed distortion map, so as to calculate the attention map of the distortion characteristics of the original remote sensing image in multiple feature dimensions based on the query sequence, and fuse the attention maps of the distortion characteristics of the multiple feature dimensions to generate a multi-dimensional distortion feature map of the original remote sensing image.

2. The method of claim 1, wherein, The method comprises the following steps: Based on the image gradient, a modulation factor of the standard deviation of the image gradient is determined. Based on the image gradient and the modulation factor, the fusion gradient information modulated de-mean contrast normalization coefficient of the original remote sensing image is calculated.

3. The method of claim 1, wherein, The method comprises the following steps: The product empirical distribution of the adjacent fusion gradient information modulated de-mean contrast normalization coefficients in the vertical direction, the horizontal direction, the main diagonal direction and the auxiliary diagonal direction of the original remote sensing image is obtained respectively. Based on the product empirical distribution, the interference statistical characteristics of the distortion of the original remote sensing image in multiple directions of the original remote sensing image are generated. The global prior vector related to the distortion of the original remote sensing image is extracted by combining the fusion gradient information modulated de-mean contrast normalization coefficient and the interference statistical characteristics.

4. The method of claim 1, wherein, The expression of the pre-designed distortion map is: in, The first feature map represents the first feature map. There are 1 channel, total One channel, and These represent the height and width of the feature map, respectively. These are the position coordinates of the original feature map. Indicates the current The 3x3 neighborhood window, centered on the location and extended one pixel up, down, left, and right, is used to predict the distortion map. These are the coordinates of each position within the currently opened 3x3 neighborhood window. In feature map First Feature matrices of each channel To be within the currently opened window Position at No. n Feature map matrix of each channel In order to be in The first in the local window area The average value of the feature map matrix of each channel. Represents the entire distortion map In the window opening area Distortion channel component map of each channel.

5. The method of claim 1, wherein, The method further comprises the following steps: Based on the multi-dimensional distortion feature map, the correlation between the multi-dimensional distortion feature map and multiple feature channels is obtained, and the multi-dimensional distortion feature map is segmented in the channel dimension according to the correlation, so as to screen out at least one first layer to-be-processed feature and at least one first layer reserved feature. The first layer to-be-processed feature is segmented to obtain at least one second layer to-be-processed feature and at least one second layer reserved feature, until a preset stop condition is reached, and multiple reserved features are obtained. The multiple reserved features are repaired to reconstruct a super-resolution remote sensing image of the original remote sensing image.

6. A distortion perception device based on the super-resolution quality of a remote sensing image, characterized by The computing module is configured to calculate a fusion gradient information modulated mean-removed contrast normalization coefficient of the original remote sensing image based on image gradients of the original remote sensing image, so as to generate a global prior vector related to distortion of the original remote sensing image based on the fusion gradient information modulated mean-removed contrast normalization coefficient. The generating module is configured to distinguish the global prior vector by using a pre-constructed blind super-resolution network, so as to generate a distortion enhanced representation for statistically expressing degradation characteristics of the original remote sensing image and identifying a degradation mode of the original remote sensing image. The processing module is configured to extract a multi-channel feature map of the original remote sensing image according to the distortion enhanced representation and the pre-constructed blind super-resolution network, calculate a query sequence of the multi-channel feature map by using a pre-designed distortion map, calculate an attention map of distortion characteristics of the original remote sensing image in a plurality of feature dimensions based on the query sequence, and generate a multi-dimensional distortion feature map of the original remote sensing image by fusing the attention maps of the distortion characteristics of the plurality of feature dimensions.

7. The apparatus of claim 6, wherein, The computing module comprises: A determining unit configured to determine a modulation factor of a standard deviation of the image gradients based on the image gradients; A calculating unit configured to calculate the fusion gradient information modulated mean-removed contrast normalization coefficient of the original remote sensing image based on the image gradients and the modulation factor.

8. An electronic device, comprising: It comprises: A memory, a processor and a computer program stored on the memory and executable on the processor, the processor executing the program to implement the distortion perception method based on the super-resolution quality of the remote sensing image according to any one of claims 1-5.

9. A computer readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the distortion perception method based on the super-resolution quality of the remote sensing image according to any one of claims 1-5.

10. A computer program product comprising a computer program, characterized in that, The computer program is executed to implement the distortion perception method based on the super-resolution quality of the remote sensing image according to any one of claims 1-5. The computer program is executed to implement the distortion perception method based on the super-resolution quality of the remote sensing image according to any one of claims 1-5.

Images