Remote sensing image processing method and device based on dynamic degradation modulation

Abstract

The invention discloses a remote sensing image processing method and device based on dynamic degradation modulation, and relates to the technical field of image processing, and the method comprises the steps: obtaining a to-be-processed remote sensing image and a degradation parameter, and carrying out the structural coding of a fuzzy kernel width through a full connection layer in a preset kernel prior embedding module and a preset activation function, generating a structured degradation vector in combination with the noise intensity, and extracting initial shallow layer features of the image by using a preset convolutional layer; inputting data including the structured degradation vector and the initial shallow features into each degradation modulation layer in a preset residual module to modulate the first to-be-processed data to obtain second to-be-processed data; processing the data by using the convolutional layer to obtain to-be-fused data, and fusing the to-be-fused data with the initial shallow layer features to obtain a fusion result; and processing the fusion result by using a preset up-sampling module to obtain a target remote sensing image with a resolution greater than the resolution corresponding to the to-be-processed remote sensing image so as to improve the resolution of the remote sensing image.

Description

A remote sensing image processing method and apparatus based on dynamic degradation modulation Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a remote sensing image processing method and apparatus based on dynamic degradation modulation. Background Technology

[0002] Currently, high-resolution remote sensing images are the core data support for civilian applications. Their spatial resolution directly determines the accuracy of ground feature detail identification, the completeness of target feature extraction, and the reliability of subsequent decision-making. However, due to factors such as imaging system hardware constraints (payload cost, size, power consumption), atmospheric interference, platform vibration, and data transmission bandwidth limitations, the remote sensing images actually acquired generally suffer from insufficient resolution and blurred details, making it difficult to meet the needs of high-precision applications.

[0003] Super-resolution image technology reconstructs high-resolution images from low-resolution images through information processing, while restoring detailed image information. It boasts advantages such as low cost, strong adaptability, and ease of engineering deployment, providing key technical support for overcoming the bottleneck of remote sensing image resolution. Existing super-resolution technologies can be divided into two stages: traditional methods and deep learning methods. Traditional methods include interpolation, reconstruction, and example learning, each with drawbacks such as blurred edges, computational complexity, and weak generalization ability. Deep learning methods have significantly driven technological progress. Early methods relied on the idealized degradation assumption of bicubic downsampling, leading to a sharp drop in performance in real-world scenarios. Subsequent introductions of attention mechanisms and Transformers improved feature modeling capabilities, but the problem of adapting to complex degradation remains unsolved. To overcome this limitation, researchers have begun to focus on more realistic complex degradation problems and, based on whether degradation information is known, have further divided super-resolution technologies into non-blind super-resolution and blind super-resolution. Non-blind super-resolution requires a known degradation model and often uses a direct concatenation of degradation information and image features, which makes it difficult to capture the spatial variability of remote sensing degradation and causes semantic conflicts due to domain differences. Blind super-resolution requires simultaneous estimation of the degradation model and the reconstructed image, but due to degradation estimation errors, the accuracy of key detail recovery is insufficient.

[0004] In remote sensing systems equipped with collaborative cameras, precise degradation parameters such as blur kernel width and noise level can be directly obtained through hardware collaboration, providing important support for overcoming existing technical bottlenecks. However, existing methods still do not fully utilize this type of degradation information and are difficult to adapt to "blur-noise" composite degradation scenarios. The generalization ability and detail recovery performance need to be improved.

[0005] As can be seen from the above, how to improve the resolution of remote sensing images in the process of remote sensing image processing based on dynamic degradation modulation is an urgent problem to be solved. Summary of the Invention

[0006] In view of this, the purpose of this invention is to provide a remote sensing image processing method and apparatus based on dynamic degradation modulation, which can improve the resolution of remote sensing images during the processing of remote sensing images based on dynamic degradation modulation. The specific solution is as follows: Firstly, this application provides a remote sensing image processing method based on dynamic degradation modulation, comprising: acquiring a remote sensing image to be processed and corresponding degradation parameters including blur kernel width and noise intensity; performing structured encoding on the blur kernel width using a fully connected layer and a preset activation function in a preset kernel prior embedding module to obtain an encoding result; generating a structured degradation vector based on the encoding result and the noise intensity; and performing shallow feature extraction on the remote sensing image to be processed using a first preset convolutional layer to obtain initial shallow features; and converting the structured degradation vector and the initial... The first data to be processed, representing initial shallow features, is input into each degenerate modulation layer in a preset residual module. The first data to be processed is modulated using a normalization layer and an attention mechanism in each degenerate modulation layer to obtain second data to be processed. The second data to be processed is then processed using a second preset convolutional layer to obtain data to be fused. This data to be fused is then fused with the initial shallow features to obtain a fusion result. The fusion result is then processed using a preset upsampling module to obtain a target remote sensing image. The resolution of the target remote sensing image is greater than the resolution of the remote sensing image to be processed.

[0007] Optionally, the step of acquiring the remote sensing image to be processed and the corresponding degradation parameters including the blur kernel width and noise intensity includes: acquiring the remote sensing image to be processed, and acquiring the degradation parameters corresponding to the remote sensing image to be processed based on an anisotropic Gaussian model, a first random eigenvalue, a second random eigenvalue, a random rotation angle and a preset Gaussian kernel size; wherein, the degradation parameters include the blur kernel width and noise intensity corresponding to the remote sensing image to be processed.

[0008] Optionally, the step of using the fully connected layers and preset activation functions in the preset kernel prior embedding module to perform structured encoding on the fuzzy kernel width to obtain the encoding result includes: using several fully connected layers, preset slopes, and preset activation functions in the preset kernel prior embedding module to perform deep correlation feature extraction on the fuzzy kernel width in a preset multilayer perceptron to obtain correlation feature extraction results; performing feature dimensionality reduction on the correlation feature extraction results to obtain the result to be encoded; and then performing structured encoding on the result to be encoded to obtain the encoding result; wherein each of the fully connected layers is connected sequentially.

[0009] Optionally, the step of generating a structured degradation vector based on the encoding result and the noise intensity, and performing shallow feature extraction on the remote sensing image to be processed using a first preset convolutional layer to obtain initial shallow features includes: performing channel concatenation processing on the encoding result and the noise intensity to obtain a structured degradation vector corresponding to the remote sensing image to be processed; the structured degradation vector includes structured prior information corresponding to the degradation parameters; determining the image size corresponding to the remote sensing image to be processed, and performing shallow feature extraction on the remote sensing image to be processed based on the image size and using the first preset convolutional layer to obtain initial shallow features with the same size as the image size.

[0010] Optionally, the step of inputting the first data to be processed, including the structured degradation vector and the initial shallow features, into each degradation modulation layer in a preset residual module, and modulating the first data to be processed using the normalization layer and attention mechanism in each degradation modulation layer to obtain the second data to be processed, includes: constructing the first data to be processed based on the structured degradation vector and the initial shallow features, and inputting the first data to be processed into a preset residual module including several cascaded degradation modulation layers; normalizing the first data to be processed using the normalization layer in each degradation modulation layer in the preset residual module to obtain normalized features, and combining the normalized features with the structured degradation vector to obtain a combination result, and then using each preset projection matrix. Tensor generation is performed on the combined result to obtain corresponding query tensors and key and value tensors fused with the structured degradation vector. A learnable relative position bias term and attention mechanism are used to generate matrices from the query tensor, key tensor, and value tensor to obtain an attention matrix. The attention matrix is ​​then residually superimposed with the first data to be processed to obtain intermediate features. The intermediate features are normalized to obtain features to be transformed. A multilayer perceptron including a nonlinear activation function is then used to transform the features to be transformed. The transformation result is residually superimposed with the intermediate features to obtain a superimposed result. Features are extracted and fused from the superimposed result using the normalization layer and attention mechanism in each degradation modulation layer to obtain the second data to be processed.

[0011] Optionally, the step of processing the second data to be processed using a second preset convolutional layer to obtain data to be fused, and fusing the data to be fused with the initial shallow features to obtain a fusion result, includes: adjusting the channel dimension of the second data to be processed using a second preset convolutional layer to obtain data to be fused; the second preset convolutional layer and the first preset convolutional layer have the same number of convolutional kernels; adding the data to be fused and the initial shallow features element-wise at their corresponding spatial positions and channels to obtain a fusion result including deep semantic information and low-level detail information; the fusion result includes global structural features and local detail features corresponding to the remote sensing image to be processed.

[0012] Optionally, after processing the fusion result using a preset upsampling module to obtain the target remote sensing image, the method further includes: processing the remote sensing image to be processed using several super-resolution algorithms to obtain corresponding processing results; the super-resolution algorithms include DPSR algorithm, USRNet algorithm, DASR algorithm, and KDSR algorithm; determining the peak signal-to-noise ratio corresponding to each processing result in several complex scenarios, and determining the visual fidelity corresponding to each processing result under different noise intensities; then performing generalization verification on each processing result in a real scene to obtain corresponding verification results; determining the performance corresponding to each processing result to obtain corresponding performance results; and processing the target remote sensing image based on the peak signal-to-noise ratio, the visual fidelity, the verification results, and the performance results to obtain a new target remote sensing image.

[0013] Secondly, this application provides a remote sensing image processing device based on dynamic degradation modulation, comprising: an encoding result generation module, used to acquire a remote sensing image to be processed and corresponding degradation parameters including blur kernel width and noise intensity, and to perform structured encoding on the blur kernel width using a fully connected layer and a preset activation function in a preset kernel prior embedding module to obtain an encoding result; a feature extraction module, used to generate a structured degradation vector based on the encoding result and the noise intensity, and to perform shallow feature extraction on the remote sensing image to be processed using a first preset convolutional layer to obtain initial shallow features; and a data determination module, used to determine the data to be processed, including the structured degradation vector and the... The first data to be processed, consisting of initial shallow features, is input into each degenerate modulation layer in a preset residual module. The first data to be processed is modulated using a normalization layer and an attention mechanism in each degenerate modulation layer to obtain second data to be processed. A fusion result generation module processes the second data to be processed using a second preset convolutional layer to obtain data to be fused, and fuses the data to be fused with the initial shallow features to obtain a fusion result. A remote sensing image determination module processes the fusion result using a preset upsampling module to obtain a target remote sensing image. The resolution of the target remote sensing image is greater than the resolution of the remote sensing image to be processed.

[0014] Optionally, the data to be processed determination module includes: a data input unit, configured to construct first data to be processed based on the structured degradation vector and the initial shallow features, and input the first data to be processed into a preset residual module comprising a plurality of cascaded degradation modulation layers; and a normalized feature determination unit, configured to normalize the first data to be processed using the normalization layers in the degradation modulation layers of the preset residual module to obtain normalized features, combine the normalized features with the structured degradation vector to obtain a combination result, and then use preset projection matrices to generate tensors from the combination result to obtain corresponding query tensors and key tensors fused with the structured degradation vector. The system comprises: a value tensor; an attention matrix generation unit, used to generate an attention matrix from the query tensor, the key tensor, and the value tensor using a learnable relative position bias term and an attention mechanism, and to perform residual superposition of the attention matrix with the first data to be processed to obtain intermediate features; a feature transformation unit, used to normalize the intermediate features to obtain features to be transformed, and then to transform the features to be transformed using a multilayer perceptron including a nonlinear activation function, and to perform residual superposition of the transformation result with the intermediate features to obtain a superposition result, and to perform feature extraction and fusion on the superposition result using the normalization layer and attention mechanism in each of the degradation modulation layers to obtain the second data to be processed.

[0015] Optionally, the fusion result generation module includes: a data to be fused determination unit, used to adjust the channel dimension of the second data to be processed using a second preset convolutional layer to obtain data to be fused; the second preset convolutional layer has the same number of convolutional kernels as the first preset convolutional layer; and a fusion result generation subunit, used to add the data to be fused and the initial shallow features element-wise at their corresponding spatial positions and channels to obtain a fusion result including deep semantic information and low-level detail information; the fusion result includes global structural features and local detail features corresponding to the remote sensing image to be processed.

[0016] Thirdly, this application provides an electronic device, comprising: a memory for storing a computer program; and a processor for executing the computer program to implement the aforementioned remote sensing image processing method based on dynamic degradation modulation.

[0017] Fourthly, this application provides a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the aforementioned remote sensing image processing method based on dynamic degradation modulation.

[0018] As can be seen from the above, before performing remote sensing image processing based on dynamic degradation modulation, this application needs to obtain the remote sensing image to be processed and the corresponding degradation parameters including the blur kernel width and noise intensity. The blur kernel width is then structured and encoded using a fully connected layer and a preset activation function in a preset kernel prior embedding module to obtain the encoding result. A structured degradation vector is generated based on the encoding result and noise intensity, and a first preset convolutional layer is used to extract shallow features from the remote sensing image to be processed, obtaining initial shallow features. The first data to be processed, including the structured degradation vector and the initial shallow features, is input into each degradation modulation layer in a preset residual module. The first data to be processed is modulated using a normalization layer and an attention mechanism in each degradation modulation layer to obtain second data to be processed. The second data to be processed is processed using a second preset convolutional layer to obtain data to be fused, and the data to be fused is fused with the initial shallow features to obtain a fusion result. Finally, the fusion result is processed using a preset upsampling module to obtain the target remote sensing image.

[0019] Therefore, this application first needs to acquire the remote sensing image to be processed and the corresponding degradation parameters, including the blur kernel width and noise intensity. Then, it uses a fully connected layer and a preset activation function in the preset kernel prior embedding module to perform structured encoding of the blur kernel width, obtaining the encoding result. Next, a structured degradation vector is generated based on the encoding result and noise intensity, and a first preset convolutional layer is used to extract shallow features from the remote sensing image to be processed, obtaining initial shallow features. Then, the first data to be processed, including the structured degradation vector and the initial shallow features, is input into each degradation modulation layer in the preset residual module, and the normalization layer and attention mechanism in each degradation modulation layer are used to modulate the first data to be processed, obtaining the second data to be processed. Finally, the second data to be processed is processed using a second preset convolutional layer to obtain the data to be fused, and the data to be fused is fused with the initial shallow features to obtain the fusion result. The fusion result is then processed using a preset upsampling module to obtain the target remote sensing image. In this way, the efficiency of remote sensing image resolution is improved in the process of remote sensing image processing based on dynamic degradation modulation, thereby enhancing the user experience. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0021] Figure 1 is a flowchart of a remote sensing image processing method based on dynamic degradation modulation disclosed in this application; Figure 2 is a schematic diagram of a specific remote sensing image processing method based on dynamic degradation modulation disclosed in this application, wherein Figure 2(a) is a schematic diagram of the overall framework of remote sensing image processing based on dynamic degradation modulation, Figure 2(b) is a schematic diagram of the kernel prior embedding module, Figure 2(c) is a schematic diagram of the SwinTransformer residual module, and Figure 2(d) is a schematic diagram of the SwinTransformer degradation modulation layer; Figure 3 is a schematic diagram of the quantization results of four specific comparison algorithms disclosed in this application and the present invention under super-resolution tasks, different noise intensities, and five anisotropic blur kernels; Figure 4 is a schematic diagram of the PSNR comparison of various methods disclosed in this application in low, medium, and high complexity scenarios constructed on the AID dataset. Figure 5 is a schematic diagram of the visualization results of different super-resolution methods under different degradation conditions disclosed in this application. Figure 5(a) is a schematic diagram of the visualization results of an image containing an anisotropic blur kernel and noise degradation under different super-resolution methods. The image in the red box area of ​​Figure 5(a) is the anisotropic blur kernel corresponding to Figure 5(a), and the noise intensity is 0. Figure 5(b) is another degraded image containing anisotropic blur kernel and noise degradation under different super-resolution methods. The image in the red box area of ​​Figure 5(b) is the anisotropic blur kernel corresponding to Figure 5(b), and the noise intensity is 10. Figure 6 is a schematic diagram of the performance comparison of different module combinations disclosed in this application. Figure 7 is a schematic diagram of the structure of a remote sensing image processing device based on dynamic degradation modulation disclosed in this application. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] Currently, high-resolution remote sensing images are core data support in the civilian field, and their spatial resolution directly determines the accuracy of ground feature detail identification, the completeness of target feature extraction, and the reliability of subsequent decision-making. However, due to factors such as imaging system hardware constraints, atmospheric interference, platform vibration, and data transmission bandwidth limitations, the acquired remote sensing images generally suffer from insufficient resolution and blurred details, making it difficult to meet the requirements of high-precision applications. To address this, this application provides a remote sensing image processing method based on dynamic degradation modulation, which can improve the resolution of remote sensing images during the dynamic degradation modulation-based remote sensing image processing process.

[0024] Referring to Figure 1, this embodiment of the invention discloses a remote sensing image processing method based on dynamic degradation modulation, including: step S11, acquiring the remote sensing image to be processed and the corresponding degradation parameters including the blur kernel width and noise intensity, and using the fully connected layer and the preset activation function in the preset kernel prior embedding module to perform structured encoding on the blur kernel width to obtain the encoding result.

[0025] In this embodiment, the overall network framework of this application is shown in Figure 2. Figure 2(a) is a schematic diagram of the overall framework for remote sensing image processing based on dynamic degradation modulation; Figure 2(b) is a schematic diagram of the kernel prior embedding module; Figure 2(c) is a schematic diagram of the SwinTransformer residual module; and Figure 2(d) is a schematic diagram of the SwinTransformer degradation modulation layer. The SwinTransformer residual module includes a SwinTransformer degradation modulation layer: First, the input low-resolution image is processed through a 3×3 convolutional layer to perform preliminary shallow feature extraction. Then, the features are fed into six SwinTransformer residual modules for deep feature processing. Simultaneously, the kernel prior embedding module is used to explicitly model image degradation information, generating a structured image degradation vector, which is then input into the first SwinTransformer degradation modulation layer of each SwinTransformer residual module. Through normalization processing and a multi-head self-attention mechanism, the image degradation vector is effectively encoded into the super-resolution network. After deep feature extraction, the output features are processed through a 3×3 convolutional layer and then fused with the initially extracted shallow features to achieve a balanced optimization of detailed information and global structure. Finally, the fused features are upsampled to obtain a high-resolution reconstructed image.

[0026] In this embodiment, the structured degradation modeling with kernel prior embedding and the dynamic feature modulation mechanism of SwinTransformer are combined, and the super-resolution performance of remote sensing images in complex scenes is improved through full-process degradation guidance, taking into account both detail recovery accuracy and scene robustness.

[0027] Furthermore, to verify the effectiveness and engineering applicability of this application, the embodiments of this application use simulated remote sensing datasets and real remote sensing datasets to conduct dual-dimensional experimental verification. In the experimental training phase, simulated satellite remote sensing degradation scenarios are constructed based on the DIV2K and Flickr2K datasets. In the verification phase, the BSD100 dataset is used to quantify the performance. The real-scene test uses the AID (Aerial Image Dataset) large-scale aerial image dataset. The AID dataset covers 30 typical land cover scenarios such as airports, farmland, and residential areas. It contains complex degradation parameters such as platform motion blur, atmospheric scattering, and sensor noise in the real imaging process, without artificially synthesized degradation annotations, thereby effectively verifying the generalization and transfer capabilities of the algorithm.

[0028] In this embodiment, for the task of super-resolution of remote sensing images, the main process of the remote sensing image super-resolution method based on dynamic degradation modulation in this application embodiment includes the following steps: First, acquire the low-resolution remote sensing image and the corresponding degradation parameters, wherein the degradation parameters include the blur kernel width and the noise level, wherein the anisotropic Gaussian blur kernel is a Gaussian probability density function with zero mean and variable covariance matrix. Describe, The random eigenvalues ​​are uniformly distributed in the interval [0.2, 4.0]. , and in the interval Random rotation angles uniformly distributed within It is confirmed that the Gaussian kernel size is fixed at 21×21; the noise level range is set to [0,25].

[0029] Specifically, acquiring the remote sensing image to be processed and the corresponding degradation parameters, including the blur kernel width and noise intensity, may include: acquiring the remote sensing image to be processed, and acquiring the degradation parameters corresponding to the remote sensing image to be processed based on an anisotropic Gaussian model, a first random eigenvalue, a second random eigenvalue, a random rotation angle, and a preset Gaussian kernel size; wherein the degradation parameters include the blur kernel width and noise intensity corresponding to the remote sensing image to be processed.

[0030] Subsequently, in this embodiment of the application, the degradation parameters need to be input into the kernel prior embedding module for structured encoding to generate a high-dimensional degradation vector. The specific process is as follows: First, the reconstructed 21×21 Gaussian blur kernel is stretched into a one-dimensional tensor and input into a multilayer perceptron (MLP) for feature learning and dimensionality reduction. The MLP consists of five fully connected layers and four Leaky ReLU activation functions, with the output dimensions of each fully connected layer being 256, 128, 64, 32, and 16, respectively. Then, deep correlation information of the degraded features is extracted through nonlinear transformation, and the dimensionality-reduced blur kernel vector is output. Subsequently, the 16-dimensional blur kernel vector is concatenated with a one-dimensional noise level parameter (value range [0,25]) to generate a 17-dimensional structured degradation vector. The aforementioned vectors are compatible with the feature dimensions of subsequent super-resolution networks, providing a structured prior for degraded modulation.

[0031] Specifically, the fuzzy kernel width is structured and encoded using fully connected layers and a preset activation function in the preset kernel prior embedding module to obtain the encoding result. This can include: using several fully connected layers, a preset slope, and a preset activation function in the preset kernel prior embedding module to perform deep correlation feature extraction on the fuzzy kernel width in a preset multilayer perceptron to obtain the correlation feature extraction result; performing feature dimensionality reduction on the correlation feature extraction result to obtain the result to be encoded; and then performing structured encoding on the result to obtain the encoding result; wherein each fully connected layer is connected sequentially.

[0032] Step S12: Generate a structured degradation vector based on the encoding result and the noise intensity, and use the first preset convolutional layer to extract shallow features from the remote sensing image to be processed to obtain initial shallow features.

[0033] In this embodiment, shallow feature extraction is performed on the low-resolution remote sensing image, and then the input image is mapped through a 3×3 convolutional layer to output initial shallow features with the same size as the input, thereby preserving the basic structural information of the image.

[0034] Specifically, a structured degradation vector is generated based on the encoding result and noise intensity, and a first preset convolutional layer is used to perform shallow feature extraction on the remote sensing image to be processed to obtain initial shallow features. This may include: concatenating the encoding result and noise intensity channels to obtain a structured degradation vector corresponding to the remote sensing image to be processed; the structured degradation vector includes structured prior information corresponding to degradation parameters; determining the image size corresponding to the remote sensing image to be processed, and performing shallow feature extraction on the remote sensing image to be processed based on the image size and using the first preset convolutional layer to obtain initial shallow features with the same size as the image size.

[0035] Step S13: Input the first data to be processed, including the structured degradation vector and the initial shallow features, into each degradation modulation layer in the preset residual module, and use the normalization layer and attention mechanism in each degradation modulation layer to modulate the first data to be processed to obtain the second data to be processed.

[0036] In this embodiment, the initial shallow features and structured degradation vector are input into a full-process degradation modulation framework for dynamic optimization of deep features. The full-process degradation modulation framework consists of six cascaded SwinTransformer residual modules. Each SwinTransformer residual module contains six SwinTransformer degradation modulation layers and one convolutional residual block. Subsequently, the initial shallow X features are subjected to layer normalization, and X, ... , respectively through projection matrix , , Generate a query vector Q, a key vector K, and a value vector V, wherein the key vector K and the value vector V are respectively superimposed with a space mapping term of the degenerate vector. , And the corresponding calculation formula is as follows: ;in, , , These are the projection matrices, , It is the image degradation vector The mapping to the key-value space enables attention computation to dynamically respond to the degradation characteristics of satellite imagery. This indicates a broadcast operation. Among them, It is an H×W dimensional matrix of all ones, which can be used to construct a one-dimensional degenerate embedding vector. and Extending to the H×W×C dimension, thus achieving spatial size matching.

[0037] In this embodiment, a learnable relative position code B is introduced to construct a multi-head self-attention (MSA) module, and the calculation formula for the attention matrix is ​​determined as follows: ;in, It is a learnable relative position encoding.

[0038] In one specific implementation, the attention function is computed in parallel 6 times (h=6 heads) and then stitched together to enhance the ability to model the spatial relationships of ground features in remote sensing images.

[0039] In this embodiment, the present application uses a multilayer perceptron (MLP) for feature nonlinear transformation. The MLP consists of two fully connected layers with a GELU activation function embedded between them. The output dimensions of the fully connected layers are respectively... , ( (This refers to the number of feature channels). It is worth noting that in this embodiment, a normalization layer is embedded before both the MSA module and the MLP module, and a residual connection structure is used to improve feature transfer efficiency. The calculation process is as follows: Subsequently, the output features of the first Swing Transformer residual module are sequentially fed into the next five Swing Transformer residual modules. The first Swing Transformer degradation modulation layer of each module receives the same structured degradation vector, thereby achieving full-process degradation guidance.

[0040] Specifically, the first data to be processed, including a structured degradation vector and initial shallow features, is input into each degradation modulation layer in a preset residual module. The first data to be processed is then modulated using a normalization layer and attention mechanism in each degradation modulation layer to obtain the second data to be processed. This process can include: constructing the first data to be processed based on the structured degradation vector and initial shallow features, and inputting the first data to be processed into a preset residual module comprising several cascaded degradation modulation layers; normalizing the first data to be processed using the normalization layers in each degradation modulation layer of the preset residual module to obtain normalized features; combining the normalized features with the structured degradation vector to obtain a combined result; and then using each preset projection matrix... Tensor generation is performed on the combined results to obtain corresponding query tensors and key and value tensors fused with structured degenerate vectors. A learnable relative position bias term and attention mechanism are used to generate matrices from the query, key, and value tensors to obtain attention matrices. These attention matrices are then residually superimposed with the first data to be processed to obtain intermediate features. The intermediate features are normalized to obtain the features to be transformed. A multilayer perceptron including a nonlinear activation function is then used to transform these features. The transformed features are residually superimposed with the intermediate features to obtain a superimposed result. The superimposed result is then used for feature extraction and fusion using the normalization layers and attention mechanisms in each degenerate modulation layer to obtain the second data to be processed.

[0041] Step S14: Process the second data to be processed using the second preset convolutional layer to obtain the data to be fused, and fuse the data to be fused with the initial shallow features to obtain the fusion result.

[0042] In this embodiment, the output features of the sixth Swing Transformer residual module are channel-adjusted through a 3×3 convolutional layer, and the channel-adjusted results are fused with the extracted initial shallow features across layers. Then, element-wise addition is used to achieve a balance optimization between detailed information and global structure to obtain fused features.

[0043] Specifically, the second preset convolutional layer is used to process the second data to be processed to obtain the data to be fused, and the data to be fused is fused with the initial shallow features to obtain the fusion result. This may include: using the second preset convolutional layer to adjust the channel dimension of the second data to be processed to obtain the data to be fused; the second preset convolutional layer and the first preset convolutional layer have the same number of convolutional kernels; the data to be fused and the initial shallow features are added element-wise at the corresponding spatial positions and channels to obtain the fusion result including deep semantic information and low-level detail information; the fusion result includes global structural features and local detail features corresponding to the remote sensing image to be processed.

[0044] Step S15: Process the fusion result using a preset upsampling module to obtain a target remote sensing image; the resolution of the target remote sensing image is greater than the resolution of the remote sensing image to be processed.

[0045] In this embodiment, to verify the effectiveness of the proposed invention, this example is compared with four mainstream super-resolution algorithms: the non-blind super-resolution methods DPSR (Deep Plug-and-Play Super-Resolution) and USRNet (Unfolding Super-Resolution Network), and the blind super-resolution methods DASR (Degradation-Aware Super-Resolution) and KDSR (Kernel-Decoupled Super-Resolution). Peak Signal-to-Noise Ratio (PSNR) is then used as the core metric. Figure 3 shows the quantization results of the four comparison algorithms and the present invention under super-resolution tasks, different noise intensities (0 and 10), and five anisotropic blur kernels. As can be seen from Figure 3, the present invention achieves the optimal PSNR value in all degradation scenarios, verifying its adaptability to both "pure blur" and "blur-noise composite" degradation scenarios.

[0046] Furthermore, Figure 4 shows a comparison of PSNR of various methods in low, medium and high complexity scenarios constructed on the AID dataset. As can be seen from Figure 4, the PSNR of the present invention reaches 25.28dB in high complexity scenarios (AGM>30), which is significantly better than other methods, demonstrating the ability to recover high-frequency details in complex degradation and complex texture coupling scenarios.

[0047] In this embodiment, the visualization results of different super-resolution methods under different degradation are shown in Figure 5. Figure 5(a) is a picture containing an anisotropic blur kernel and noise degradation and its visualization results under different super-resolution methods. The red box area in Figure 5(a) is the anisotropic blur kernel corresponding to Figure 5(a), and the noise intensity is 0. Figure 5(b) is another degraded picture containing anisotropic blur kernel and noise degradation and its visualization results under different super-resolution methods. The red box area in Figure 5(b) is the anisotropic blur kernel corresponding to Figure 5(b), and the noise intensity is 10.

[0048] It is worth mentioning that in degradation scenarios with noise levels of 0 and 10, DASR and KDSR methods suffer from low texture recognition and blurred edge contours, while DPSR and USRNet lack accurate detail recovery. In contrast, this invention can clearly restore high-frequency textures, enhance key features such as building edges, and achieve superior visual fidelity in the reconstructed image. In airport scenes on the AID dataset, this invention effectively adapts to real-world remote sensing degradation, clearly restoring parking lot markings and aircraft outlines, and exhibits better background suppression than the contrasting methods.

[0049] To further verify the effectiveness of the core module, an ablation experiment was conducted in the embodiments of this application. Figure 6 is a schematic diagram comparing the performance of different module combinations. Among them, the PSNR of the K1 model without the core prior embedding module is 25.51dB, the PSNR of the K2 model without the degradation modulation module is 25.24dB, the PSNR of the E1 and E2 models with degradation vectors injected only in a single module are 26.62dB and 26.53dB, respectively, while the PSNR of the present invention (baseline model) including the complete module and full-process modulation reaches 26.81dB, proving that the co-design of core prior embedding and full-process degradation modulation is the key to performance improvement.

[0050] Specifically, after processing the fusion result using a preset upsampling module to obtain the target remote sensing image, the process may further include: processing the remote sensing image to be processed using several super-resolution algorithms to obtain corresponding processing results; the super-resolution algorithms include DPSR, USRNet, DASR, and KDSR algorithms; determining the peak signal-to-noise ratio (PSNR) corresponding to each processing result under several complex scenarios, and determining the visual fidelity corresponding to each processing result under different noise intensities; then performing generalization verification on each processing result in a real scene to obtain corresponding verification results; determining the performance corresponding to each processing result to obtain corresponding performance results; and processing the target remote sensing image based on the PSNR, visual fidelity, verification results, and performance results to obtain a new target remote sensing image.

[0051] As can be seen from the above, the embodiments of this application first need to obtain the remote sensing image to be processed and the corresponding degradation parameters including the blur kernel width and noise intensity. The blur kernel width is then structured and encoded using a fully connected layer and a preset activation function in a preset kernel prior embedding module to obtain the encoding result. Next, a structured degradation vector is generated based on the encoding result and noise intensity, and a first preset convolutional layer is used to extract shallow features from the remote sensing image to be processed, obtaining initial shallow features. Then, the first data to be processed, including the structured degradation vector and the initial shallow features, is input into each degradation modulation layer in a preset residual module. The first data to be processed is modulated using a normalization layer and an attention mechanism in each degradation modulation layer to obtain second data to be processed. Finally, the second data to be processed is processed using a second preset convolutional layer to obtain data to be fused, and the data to be fused is fused with the initial shallow features to obtain a fusion result. The fusion result is then processed using a preset upsampling module to obtain the target remote sensing image. In this way, the efficiency of remote sensing image resolution is improved in the process of remote sensing image processing based on dynamic degradation modulation, thereby enhancing the user experience.

[0052] Accordingly, referring to Figure 7, this application also provides a remote sensing image processing device based on dynamic degradation modulation, comprising: an encoding result generation module 11, used to acquire the remote sensing image to be processed and the corresponding degradation parameters including the blur kernel width and noise intensity, and to perform structured encoding on the blur kernel width using a fully connected layer and a preset activation function in a preset kernel prior embedding module to obtain an encoding result; a feature extraction module 12, used to generate a structured degradation vector based on the encoding result and the noise intensity, and to perform shallow feature extraction on the remote sensing image to be processed using a first preset convolutional layer to obtain initial shallow features; and a data determination module 13, used to determine the data including the structured degradation vector. The vector and the first data to be processed, along with the initial shallow features, are input into each degenerate modulation layer in the preset residual module. The first data to be processed is modulated using the normalization layer and attention mechanism in each degenerate modulation layer to obtain the second data to be processed. The fusion result generation module 14 is used to process the second data to be processed using a second preset convolutional layer to obtain data to be fused, and to fuse the data to be fused with the initial shallow features to obtain the fusion result. The remote sensing image determination module 15 is used to process the fusion result using a preset upsampling module to obtain the target remote sensing image. The resolution of the target remote sensing image is greater than the resolution of the remote sensing image to be processed.

[0053] In some specific embodiments, the encoding result generation module 11 may specifically include: a remote sensing image acquisition unit, used to acquire a remote sensing image to be processed, and to acquire degradation parameters corresponding to the remote sensing image to be processed based on an anisotropic Gaussian model, a first random feature value, a second random feature value, a random rotation angle and a preset Gaussian kernel size; wherein, the degradation parameters include the blur kernel width and noise intensity corresponding to the remote sensing image to be processed.

[0054] In some specific embodiments, the encoding result generation module 11 may specifically include: a feature extraction unit, used to perform deep correlation feature extraction on the fuzzy kernel width in a preset multilayer perceptron using several fully connected layers, a preset slope, and a preset activation function in a preset kernel prior embedding module, to obtain a correlation feature extraction result; and an encoding result generation unit, used to perform feature dimensionality reduction on the correlation feature extraction result to obtain a result to be encoded, and then perform structured encoding on the result to be encoded to obtain an encoding result; wherein, each of the fully connected layers is connected sequentially.

[0055] In some specific embodiments, the feature extraction module 12 may specifically include: a degradation vector generation unit, used to perform channel concatenation processing on the encoding result and the noise intensity to obtain a structured degradation vector corresponding to the remote sensing image to be processed; the structured degradation vector includes structured prior information corresponding to the degradation parameters; and an image size determination unit, used to determine the image size corresponding to the remote sensing image to be processed, so as to perform shallow feature extraction on the remote sensing image to be processed based on the image size and using a first preset convolutional layer to obtain initial shallow features with the same size as the image size.

[0056] In some specific embodiments, the data to be processed determination module 13 may specifically include: a data input unit, used to construct first data to be processed based on the structured degradation vector and the initial shallow features, and input the first data to be processed into a preset residual module including a plurality of cascaded degradation modulation layers; a normalized feature determination unit, used to normalize the first data to be processed using the normalization layer in each degradation modulation layer in the preset residual module to obtain normalized features, and combine the normalized features with the structured degradation vector to obtain a combination result, and then use each preset projection matrix to generate a tensor from the combination result to obtain corresponding query tensors and tensors fused with the structured degradation vector. The system comprises a key tensor and a value tensor of a vector; an attention matrix generation unit, used to generate an attention matrix from the query tensor, the key tensor, and the value tensor using a learnable relative position bias term and an attention mechanism, and then superimposing the attention matrix with the first data to be processed using residuals to obtain intermediate features; a feature transformation unit, used to normalize the intermediate features to obtain features to be transformed, then using a multilayer perceptron including a nonlinear activation function to transform the features to be transformed, and then superimposing the transformation result with the intermediate features using residuals to obtain a superimposed result, and then using the normalization layer and attention mechanism in each of the degradation modulation layers to extract and fuse features from the superimposed result to obtain the second data to be processed.

[0057] In some specific embodiments, the fusion result generation module 14 may specifically include: a data to be fused determination unit, used to adjust the channel dimension of the second data to be processed using a second preset convolutional layer to obtain data to be fused; the second preset convolutional layer has the same number of convolutional kernels as the first preset convolutional layer; and a fusion result generation subunit, used to add the data to be fused and the initial shallow features element-wise at their corresponding spatial positions and channels to obtain a fusion result including deep semantic information and low-level detail information; the fusion result includes global structural features and local detail features corresponding to the remote sensing image to be processed.

[0058] In some specific embodiments, the remote sensing image processing device based on dynamic degradation modulation may further include: a processing result generation unit, used to process the remote sensing image to be processed using several super-resolution algorithms to obtain corresponding processing results; the super-resolution algorithms include DPSR algorithm, USRNet algorithm, DASR algorithm, and KDSR algorithm; a peak signal-to-noise ratio (PSNR) determination unit, used to determine the PNR corresponding to each of the processing results under several complex scenarios, and determine the visual fidelity corresponding to each of the processing results under different noise intensities, then perform generalization verification on each of the processing results in a real scenario to obtain corresponding verification results, determine the performance corresponding to each of the processing results, and obtain corresponding performance results; and an image processing unit, used to process the target remote sensing image based on the PNR, the visual fidelity, the verification results, and the performance results to obtain a new target remote sensing image.

[0059] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0060] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0061] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0062] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0063] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A remote sensing image processing method based on dynamic degradation modulation, characterized in that, include: The remote sensing image to be processed and the corresponding degradation parameters, including the blur kernel width and noise intensity, are obtained. The blur kernel width is then structured and encoded using the fully connected layer and the preset activation function in the preset kernel prior embedding module to obtain the encoding result. Based on the encoding result and the noise intensity, a structured degradation vector is generated, and a first preset convolutional layer is used to extract shallow features from the remote sensing image to be processed to obtain initial shallow features. The first data to be processed, including the structured degradation vector and the initial shallow features, is input into each degradation modulation layer in the preset residual module, and the first data to be processed is modulated using the normalization layer and attention mechanism in each degradation modulation layer to obtain the second data to be processed. The second data to be processed is processed using a second preset convolutional layer to obtain data to be fused, and the data to be fused is fused with the initial shallow features to obtain a fusion result; The fusion result is processed using a preset upsampling module to obtain a target remote sensing image; the resolution of the target remote sensing image is greater than the resolution of the remote sensing image to be processed.

2. The remote sensing image processing method based on dynamic degradation modulation according to claim 1, characterized in that, The step of acquiring the remote sensing image to be processed and the corresponding degradation parameters, including the blur kernel width and noise intensity, includes: acquiring the remote sensing image to be processed, and acquiring the degradation parameters corresponding to the remote sensing image to be processed based on an anisotropic Gaussian model, a first random eigenvalue, a second random eigenvalue, a random rotation angle and a preset Gaussian kernel size; wherein, the degradation parameters include the blur kernel width and noise intensity corresponding to the remote sensing image to be processed.

3. The remote sensing image processing method based on dynamic degradation modulation according to claim 1, characterized in that, The step of using a pre-defined kernel prior embedding module to structurally encode the fuzzy kernel width using fully connected layers and a pre-defined activation function to obtain an encoding result includes: using several fully connected layers, a pre-defined slope, and a pre-defined activation function in a pre-defined multilayer perceptron to perform deep correlation feature extraction on the fuzzy kernel width to obtain a correlation feature extraction result; performing feature dimensionality reduction on the correlation feature extraction result to obtain a result to be encoded; and then performing structured encoding on the result to be encoded to obtain an encoding result; wherein each of the fully connected layers is connected sequentially.

4. The remote sensing image processing method based on dynamic degradation modulation according to claim 1, characterized in that, The process of generating a structured degradation vector based on the encoding result and the noise intensity, and extracting shallow features from the remote sensing image to be processed using a first preset convolutional layer to obtain initial shallow features, includes: concatenating the encoding result and the noise intensity into channels to obtain a structured degradation vector corresponding to the remote sensing image to be processed; the structured degradation vector includes structured prior information corresponding to the degradation parameters; determining the image size corresponding to the remote sensing image to be processed, and extracting shallow features from the remote sensing image to be processed based on the image size and using the first preset convolutional layer to obtain initial shallow features with the same size as the image size.

5. The remote sensing image processing method based on dynamic degradation modulation according to claim 1, characterized in that, The process of inputting first data to be processed, including the structured degradation vector and the initial shallow features, into each degradation modulation layer in a preset residual module, and modulating the first data to be processed using the normalization layer and attention mechanism in each degradation modulation layer to obtain second data to be processed, includes: constructing first data to be processed based on the structured degradation vector and the initial shallow features, and inputting the first data to be processed into a preset residual module comprising several cascaded degradation modulation layers; normalizing the first data to be processed using the normalization layer in each degradation modulation layer of the preset residual module to obtain normalized features, combining the normalized features with the structured degradation vector to obtain a combination result, and then using each preset projection matrix to modify the first data to be processed. The combined results are used to generate tensors, resulting in corresponding query tensors and key and value tensors fused with the structured degradation vector. A learnable relative position bias term and an attention mechanism are used to generate matrices from the query tensor, key tensor, and value tensor to obtain an attention matrix. The attention matrix is ​​then residually superimposed with the first data to be processed to obtain intermediate features. These intermediate features are normalized to obtain features to be transformed. A multilayer perceptron including a nonlinear activation function is then used to transform these features, and the transformation result is residually superimposed with the intermediate features to obtain a superimposed result. The superimposed result is then used to extract and fuse features using the normalization layer and attention mechanism in each degradation modulation layer to obtain the second data to be processed.

6. The remote sensing image processing method based on dynamic degradation modulation according to claim 1, characterized in that, The process of processing the second data to be processed using a second preset convolutional layer to obtain data to be fused, and fusing the data to be fused with the initial shallow features to obtain a fusion result, includes: adjusting the channel dimension of the second data to be processed using a second preset convolutional layer to obtain data to be fused; the second preset convolutional layer and the first preset convolutional layer have the same number of convolutional kernels; adding the data to be fused and the initial shallow features element-wise at their corresponding spatial positions and channels to obtain a fusion result including deep semantic information and low-level detail information; the fusion result includes global structural features and local detail features corresponding to the remote sensing image to be processed.

7. The remote sensing image processing method based on dynamic degradation modulation according to any one of claims 1 to 6, characterized in that, After processing the fusion result using a preset upsampling module to obtain the target remote sensing image, the method further includes: processing the remote sensing image to be processed using several super-resolution algorithms to obtain corresponding processing results; the super-resolution algorithms include DPSR algorithm, USRNet algorithm, DASR algorithm, and KDSR algorithm; determining the peak signal-to-noise ratio corresponding to each processing result in several complex scenarios, and determining the visual fidelity corresponding to each processing result under different noise intensities; then performing generalization verification on each processing result in a real scene to obtain corresponding verification results; determining the performance corresponding to each processing result to obtain corresponding performance results; and processing the target remote sensing image based on the peak signal-to-noise ratio, the visual fidelity, the verification results, and the performance results to obtain a new target remote sensing image.

8. A remote sensing image processing device based on dynamic degradation modulation, characterized in that, include: The encoding result generation module is used to acquire the remote sensing image to be processed and the corresponding degradation parameters including the blur kernel width and noise intensity, and to perform structured encoding on the blur kernel width using the fully connected layer and the preset activation function in the preset kernel prior embedding module to obtain the encoding result; The feature extraction module is used to generate a structured degradation vector based on the encoding result and the noise intensity, and to perform shallow feature extraction on the remote sensing image to be processed using a first preset convolutional layer to obtain initial shallow features. The data to be processed determination module is used to input the first data to be processed, including the structured degradation vector and the initial shallow features, into each degradation modulation layer in the preset residual module, and to modulate the first data to be processed using the normalization layer and attention mechanism in each degradation modulation layer to obtain the second data to be processed. The fusion result generation module is used to process the second data to be processed using a second preset convolutional layer to obtain data to be fused, and to fuse the data to be fused with the initial shallow features to obtain a fusion result; The remote sensing image determination module is used to process the fusion result using a preset upsampling module to obtain a target remote sensing image; the resolution of the target remote sensing image is greater than the resolution of the remote sensing image to be processed.

9. The remote sensing image processing apparatus based on dynamic degradation modulation according to claim 8, characterized in that, The data to be processed determination module includes: a data input unit, configured to construct first data to be processed based on the structured degradation vector and the initial shallow features, and input the first data to be processed into a preset residual module comprising several cascaded degradation modulation layers; and a normalized feature determination unit, configured to normalize the first data to be processed using the normalization layers in the degradation modulation layers of the preset residual module to obtain normalized features, combine the normalized features with the structured degradation vector to obtain a combination result, and then generate tensors from the combination result using preset projection matrices to obtain corresponding query tensors and key tensors and value tensors fused with the structured degradation vector. The system comprises: an attention matrix generation unit, used to generate an attention matrix from the query tensor, the key tensor, and the value tensor using a learnable relative position bias term and an attention mechanism; and residual superposition of the attention matrix with the first data to be processed to obtain intermediate features; and a feature transformation unit, used to normalize the intermediate features to obtain features to be transformed, and then transform the features to be transformed using a multilayer perceptron including a nonlinear activation function, and residual superposition of the transformation result with the intermediate features to obtain a superposition result, which is then used to extract and fuse features from the superposition result using the normalization layer and attention mechanism in each of the degradation modulation layers to obtain the second data to be processed.

10. The remote sensing image processing apparatus based on dynamic degradation modulation according to claim 8, characterized in that, The fusion result generation module includes: a data to be fused determination unit, used to adjust the channel dimension of the second data to be processed using a second preset convolutional layer to obtain data to be fused; the second preset convolutional layer has the same number of convolutional kernels as the first preset convolutional layer; and a fusion result generation subunit, used to add the data to be fused and the initial shallow features element-wise at their corresponding spatial positions and channels to obtain a fusion result including deep semantic information and low-level detail information; the fusion result includes global structural features and local detail features corresponding to the remote sensing image to be processed.

Images