Remote Sensing Multispectral Image Fusion Method and System Based on Multi-Stage Feature Correction

Abstract

The present invention relates to the field of computer image processing, and provides a remote sensing multispectral image fusion method and system based on multi-stage feature correction, including obtaining a spectral mask through a low-resolution multispectral image and a panchromatic image, covering the panchromatic image with the spectral mask to obtain a corrected panchromatic image; enhancing the scale features of the low-resolution multispectral image, and downsampling and convolving the corrected panchromatic image to obtain image scale features; obtaining an input of an attention block, mapping the input of the attention block to obtain a mapping vector group; using the mapping vector group for image correction to obtain a feature correction vector, iterating the feature correction vector to obtain a target feature correction vector, and obtaining a fusion feature matrix; iterating the fusion feature matrix to obtain a target fusion feature, and performing layer-by-layer feature refinement and convolution on the target fusion feature to obtain a high-resolution multispectral image. The present invention can obtain a high-fidelity high-resolution multispectral image.

Description

Technical Field

[0001] The present invention relates to the technical field of computer image processing, and particularly to a remote sensing multispectral image fusion method and system based on multi-stage feature correction. Background Art

[0002] Multispectral sensors cover common visible light and some near-infrared bands and can resolve specific spectral features, so they are widely used in remote sensing observation satellites. However, the spatial resolution of the images captured by multispectral sensors is restricted by physical properties and current technological conditions, and the performance in spatial resolution is not good. To make up for this defect, satellites usually carry panchromatic sensors to capture and record single-band panchromatic images with higher spatial resolution. Although panchromatic images can significantly enhance spatial details by integrating optical signals in a wide band, their single-band characteristics lead to a serious lack of spectral information and it is difficult to meet the application requirements that rely on multispectral feature analysis such as land cover classification and vegetation health diagnosis. Therefore, it is necessary to meet these requirements through multispectral panchromatic sharpening technology, and its goal is to fuse the high-frequency texture information of panchromatic images with the spectral features of low-spatial-resolution multispectral images through methods to generate high-resolution multispectral images with both high spatial resolution and multispectral fidelity. This technology has received considerable attention in the field of remote sensing and is of great significance in various remote sensing tasks such as remote sensing mapping, resource investigation, and classification.

[0003] Multispectral panchromatic sharpening methods are mainly divided into two categories: traditional methods and deep learning-based methods. Traditional methods can be divided into methods based on component replacement, methods based on multiresolution analysis, and methods based on variational optimization. However, due to limited feature representation and a strong dependence on linear mathematical modeling, these methods often suffer from significant spectral and spatial distortions and impose a high computational burden, which limits their practical applications.

[0004] In recent years, due to deep learning technology, especially the powerful feature extraction ability of convolutional neural networks, more and more deep learning-based multispectral panchromatic sharpening methods have emerged and achieved better results than traditional methods. However, existing methods will cause spectral and spatial distortions in the fused images due to ignoring the influence of the modal differences between the two input images on feature fusion. Summary of the Invention

[0005] The present invention aims to solve at least one of the technical problems existing in the related art. For this purpose, the present invention provides a remote sensing multispectral image fusion method and system based on multi-stage feature correction, which realizes the generation of high-resolution multispectral images with both high spatial resolution and multispectral fidelity.

[0006] The present invention provides a remote sensing multispectral image fusion method based on multi-stage feature correction, including:

[0007] S1: Obtain a low - resolution multispectral image and a panchromatic image, obtain a spectral mask from the low - resolution multispectral image and the panchromatic image, and use the spectral mask to cover the panchromatic image to obtain a corrected panchromatic image;

[0008] S2: Perform scale feature enhancement on the low - resolution multispectral image through a multi - scale channel attention block, and perform downsampling and convolution on the corrected panchromatic image to obtain image scale features;

[0009] S3: Obtain the input of the attention block from the image scale features, map the input of the attention block to obtain a mapped vector group including a panchromatic vector group and a low - resolution vector group;

[0010] S4: Use the mapped vector group to perform image correction to obtain a feature correction vector, iterate the feature correction vector to obtain a target feature correction vector, and input the target feature correction vector into a feed - forward neural network to obtain a fused feature matrix;

[0011] S5: Repeat S3 to S4 to iterate the fused feature matrix to obtain a target fused feature, construct a feature refinement block, and perform layer - by - layer feature fusion and convolution on the target fused feature through the feature refinement block to obtain a high - resolution multispectral image.

[0012] According to the remote - sensing multispectral image fusion method based on multi - stage feature correction provided by the present invention, in step S1, the low - resolution multispectral image is upsampled to obtain an upsampled multispectral image, local entropy operation is performed on the upsampled multispectral image, and guided filtering operation and normalization are performed on the upsampled multispectral image after local entropy operation and the panchromatic image to obtain a spectral mask.

[0013] According to the remote - sensing multispectral image fusion method based on multi - stage feature correction provided by the present invention, in step S1, the spectral mask is multiplied point - by - point with the panchromatic image to complete the covering of the panchromatic image.

[0014] According to the remote - sensing multispectral image fusion method based on multi - stage feature correction provided by the present invention, in step S2, the input of the multi - scale channel attention block is obtained from the low - resolution multispectral image. In the multi - scale channel attention block, multi - scale convolution and channel attention operation are performed on the input of the multi - scale channel attention block to obtain low - resolution scale features.

[0015] According to the remote - sensing multispectral image fusion method based on multi - stage feature correction provided by the present invention, in step S2, convolution and downsampling are performed on the corrected panchromatic image to obtain panchromatic scale features, and the image scale features are obtained through the panchromatic scale features and the low - resolution scale features.

[0016] According to the remote sensing multispectral image fusion method based on multi-stage feature correction provided by the present invention, in step S3, the attention block input is mapped into the Transformer structure to obtain a panchromatic vector group and a low-resolution vector group respectively.

[0017] According to the remote sensing multispectral image fusion method based on multi-stage feature correction provided by the present invention, in step S4, deformation operation and normalization are performed on the mapping vector group to obtain a vertical matrix and a horizontal matrix. A feature correction vector including a panchromatic feature correction vector and a low-spectral feature correction vector is obtained through the vertical matrix and the horizontal matrix. The low-spectral feature correction vector and the panchromatic feature correction vector are used as the attention block input, and the feature correction vector is iterated to obtain a target feature correction vector.

[0018] According to the remote sensing multispectral image fusion method based on multi-stage feature correction provided by the present invention, in step S4, the target feature correction vector is input into the feed-forward neural network, and convolution, normalization, and residual operations are performed on the target feature correction vector in the feed-forward neural network to obtain the fusion feature matrix.

[0019] According to the remote sensing multispectral image fusion method based on multi-stage feature correction provided by the present invention, in step S5, refined features are obtained through the target fusion feature. The refined features are iterated through the feature refinement block and the target fusion feature, and layer-by-layer feature fusion and convolution are performed on the target fusion feature to obtain target refined features. Feature refinement and convolution are performed on the target refined features to obtain the high-resolution multispectral image.

[0020] The present invention also provides a remote sensing multispectral image fusion system based on multi-stage feature correction, including:

[0021] Panchromatic image correction module: used to obtain a low-resolution multispectral image and a panchromatic image, obtain a spectral mask through the low-resolution multispectral image and the panchromatic image, and use the spectral mask to cover the panchromatic image to obtain a corrected panchromatic image;

[0022] Image scale feature module: used to enhance the scale features of the low-resolution multispectral image through a multi-scale channel attention block, and perform downsampling and convolution on the corrected panchromatic image to obtain image scale features;

[0023] Mapping vector group module: used to obtain the attention block input through the image scale features, map the attention block input, and obtain a mapping vector group including a panchromatic vector group and a low-resolution vector group;

[0024] Fusion Feature Matrix Module: It is used to perform image correction using a mapping vector group to obtain a feature correction vector, iterate the feature correction vector to obtain a target feature correction vector, and input the target feature correction vector into a feedforward neural network to obtain a fusion feature matrix;

[0025] High-Resolution Multispectral Image Module: It is used to repeat the mapping vector group module to the fusion feature matrix module to iterate the fusion feature matrix to obtain a target fusion feature, construct a feature refinement block, and perform layer-by-layer feature fusion and convolution on the target fusion feature through the feature refinement block to obtain a high-resolution multispectral image.

[0026] One or more of the above technical solutions in the embodiments of the present invention have at least one of the following technical effects:

[0027] The remote sensing multispectral image fusion method and system based on multi-stage feature correction provided by the present invention effectively extract and enhance the features of the low-resolution multispectral image through a multi-scale channel attention block, and in the process of obtaining the target fusion feature, use continuous sparse attention to replace single-layer dense attention, realizing lightweight calculation, reducing memory usage, establishing the relationship between the panchromatic image and the low-resolution multispectral image, thereby correcting the differences between the panchromatic image and the low-resolution multispectral image, reducing distortion and aberration caused by information interference during the fusion process, and thus generating a high-resolution multispectral image with less distortion.

[0028] The additional aspects and advantages of the present invention will be partially given in the following description, partially become obvious from the following description, or be understood through the practice of the present invention. Description of the Drawings

[0029] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0030] Figure 1 It is a schematic flowchart of the remote sensing multispectral image fusion method based on multi-stage feature correction provided by the present invention.

[0031] Figure 2 It is a comparison diagram of the mean absolute error map of the remote sensing multispectral image fusion method based on multi-stage feature correction provided by the present invention.

[0032] Figure 3 It is a schematic structural diagram of the remote sensing multispectral image fusion based on multi-stage feature correction provided by the present invention.

[0033] Reference Signs:

[0034] 100, Panchromatic Image Correction Module; 200, Image Scale Feature Module; 300, Mapping Vector Group Module; 400, Fusion Feature Matrix Module; 500, High-Resolution Multispectral Image Module. Detailed Implementation Manner

[0035] To make the objectives, technical solutions, and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below. Apparently, the described embodiments are some, but not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without making creative efforts fall within the protection scope of the present invention. The following embodiments are used to illustrate the present invention, but cannot be used to limit the scope of the present invention.

[0036] In the description of the embodiments of the present invention, it should be noted that the terms "first", "second", and "third" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance.

[0037] In the description of this specification, the description with reference to terms such as "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the embodiments of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, without conflict, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0038] The following combines Figures 1 to 3 to describe the implementation scheme of the present invention.

[0039] Figure 1 is a schematic flowchart of a remote sensing multispectral image fusion method based on multi-stage feature correction provided by the present invention. As shown in the figure, first, a spectral mask is obtained, and the panchromatic image is covered with the spectral mask to obtain a corrected panchromatic image; then, the scale features of the low-resolution multispectral image are enhanced, and the corrected panchromatic image is downsampled and convolved to obtain image scale features; subsequently, an input to the attention block is obtained, and the input to the attention block is mapped to obtain a mapping vector group including a panchromatic vector group and a low-resolution vector group; then, image correction is performed to obtain a feature correction vector, and the target feature correction vector is input into a feedforward neural network to obtain a fusion feature matrix; finally, a target fusion feature is obtained, and the target fusion feature is subjected to layer-by-layer feature refinement and convolution to obtain a high-resolution multispectral image.

[0040] The present invention provides a remote sensing multispectral image fusion method based on multi-stage feature correction, which specifically includes:

[0041] S1: Obtain a low-resolution multispectral image and a panchromatic image, obtain a spectral mask from the low-resolution multispectral image and the panchromatic image, and use the spectral mask to cover the panchromatic image to obtain a corrected panchromatic image;

[0042] Furthermore, the purpose of this stage is to obtain a spectral mask, so as to use the spectral mask to cover the panchromatic image to obtain a corrected panchromatic image. Specifically, in step S1, the low-resolution multispectral image is upsampled to obtain an upsampled multispectral image, local entropy operation is performed on the upsampled multispectral image, and guided filtering operation and normalization are performed on the upsampled multispectral image after local entropy operation and the panchromatic image to obtain a spectral mask.

[0043] In step S1, the spectral mask is multiplied by the panchromatic image point by point to complete the covering of the panchromatic image.

[0044] For the above steps, the specific implementation in this embodiment is as follows:

[0045] First, obtain the low-resolution multispectral image taken by a remote sensing satellite and the panchromatic image . Then, the low-resolution multispectral image is upsampled to obtain an upsampled multispectral image . Subsequently, local entropy operation is performed on the upsampled multispectral image, and guided filtering operation and normalization are performed on the upsampled multispectral image after local entropy operation and the panchromatic image to obtain a spectral mask SM:

[0046]

[0047] Among them, denotes performing local entropy operation on the content in the parentheses, denotes performing guided filtering operation on the content in the parentheses, and Norm() denotes performing min-max normalization on the content in the parentheses.

[0048] Finally, the spectral mask is multiplied by the panchromatic image point by point to complete the covering of the panchromatic image, and a corrected panchromatic image with the same number of bands as the low-resolution multispectral image is obtained :

[0049]

[0050] Among them, denotes point-by-point multiplication.

[0051] S2: Enhance the scale features of the low-resolution multispectral image through a multi-scale channel attention block, downsample and perform convolution on the corrected panchromatic image to obtain the image scale features;

[0052] Further, in this stage, the corrected panchromatic image is obtained through multi-scale channel attention, downsampling, and convolution. Specifically, in step S2, the input of the multi-scale channel attention block is obtained from the low-resolution multispectral image. In the multi-scale channel attention block, multi-scale convolution and channel attention operations are performed on the input of the multi-scale channel attention block to obtain the low-resolution scale features.

[0053] In step S2, convolution and downsampling are performed on the corrected panchromatic image to obtain the panchromatic scale features, and the image scale features are obtained through the panchromatic scale features and the low-resolution scale features.

[0054] For the above steps, the specific implementation in this embodiment is as follows:

[0055] First, determine the number of multi-scale channel attention blocks. In this embodiment, the number of multi-scale channel attention blocks is 3 and they are connected in sequence. Subsequently, for the first multi-scale channel attention block, 3×3 convolution is performed on the low-resolution multispectral image to obtain the input of the multi-scale channel attention block of the first multi-scale channel attention block , and for the other multi-scale channel attention blocks, 3×3 convolution is performed on the low-resolution scale features of the previous multi-scale channel attention block after upsampling, and it is used as the input of the multi-scale channel attention block.

[0056] In the multi-scale channel attention block, first perform multi-scale convolution on the input of the multi-scale channel attention block to obtain the multi-scale convolution output of the k-th scale in the s-th multi-scale channel attention block :

[0057]

[0058] Among them, means performing 1×1 convolution on the content in the parentheses, means performing k×k convolution on the content in the parentheses, where the value of k is 3, 5, and 7. Then perform channel attention operation to obtain the s-th low-resolution scale feature :

[0059]

[0060] Among them, Cat() means concatenating the content in the parentheses, and CA() means performing channel attention operation on the content in the parentheses.

[0061] Thus, the s-th low-resolution scale feature obtained from the s-th multi-scale channel attention block can be expressed as:

[0062]

[0063] where s is an integer, Up() represents the upsampling operation on the content within the parentheses, MCAB() represents inputting the content within the parentheses into the multi-scale channel attention block, and represents performing a 3×3 convolution on the content within the parentheses.

[0064] Next, perform convolution and downsampling on the corrected panchromatic image, and determine the number of times of convolution and downsampling according to the number of multi-scale channel attention blocks to obtain the s-th panchromatic scale feature :

[0065]

[0066] where Down() represents downsampling the content within the parentheses, and s starts from 3 and decreases by 1 each time convolution and downsampling are performed. Take all the panchromatic scale features and low-resolution scale features as image scale features.

[0067] S3: Obtain the input of the attention block from the image scale features, and map the input of the attention block to obtain a mapped vector group including a panchromatic vector group and a low-resolution vector group;

[0068] Furthermore, the purpose of this stage is to obtain the input of the attention block and perform mapping to obtain a mapped vector group. Specifically, in step S3, map the input of the attention block into the Transformer structure to obtain a panchromatic vector group and a low-resolution vector group respectively.

[0069] For the above steps, the specific implementation in this embodiment is as follows:

[0070] First, establish multiple feature cross-attention blocks according to the number of panchromatic scale features and low-resolution scale features. Since the number of both panchromatic scale features and low-resolution scale features is 3, the number of feature cross-attention blocks is also 3. Take the 1st panchromatic scale feature and the 1st low-resolution scale feature as the input of the 1st feature cross-attention block, that is, the input of the attention block. Then, let the input of the attention block including the 1st panchromatic scale feature and the 1st low-resolution scale feature enter the 1st multi-scale channel attention block, and map it into the Transformer structure in the multi-scale channel attention block, then a panchromatic vector group and a low-resolution vector group can be obtained respectively, thus forming a mapped vector group:

[0071]

[0072] Among them, LN() represents performing layer normalization operation of the Transformer structure on the content within the brackets. is the input of the s-th panchromatic attention block, which is the first panchromatic scale feature here. is the input of the s-th low-resolution attention block, which is the first low-resolution scale feature here. is the first panchromatic vector. is the second panchromatic vector. is the third panchromatic vector. is the first low-resolution vector. is the second low-resolution vector. is the third low-resolution vector.

[0073] S4: Use the mapping vector group to perform image correction to obtain a feature correction vector, iterate the feature correction vector to obtain a target feature correction vector, and input the target feature correction vector into a feed-forward neural network to obtain a fused feature matrix.

[0074] Furthermore, the purpose of this stage is to use the mapping vector group to perform image correction and iterate the feature correction vector to obtain a target feature correction vector, and finally output a fused feature matrix. Specifically, in step S4, perform a deformation operation and normalization on the mapping vector group to obtain a vertical matrix and a horizontal matrix, obtain a feature correction vector including a panchromatic feature correction vector and a hyperspectral feature correction vector through the vertical matrix and the horizontal matrix, use the hyperspectral feature correction vector and the panchromatic feature correction vector as attention block inputs and iterate the feature correction vector to obtain a target feature correction vector.

[0075] In step S4, input the target feature correction vector into a feed-forward neural network, and perform convolution, normalization, and residual operations on the target feature correction vector in the feed-forward neural network to obtain the fused feature matrix.

[0076] For the above steps, the specific implementation in this embodiment is as follows:

[0077] In the feature cross-attention block, during the process of performing a deformation operation and normalization on the mapping vector group, first perform a deformation operation and normalization on the first low-resolution vector and the second panchromatic vector in the horizontal dimension and the vertical dimension respectively to obtain the first vertical matrix of the s-th feature cross-attention block and the first horizontal matrix of the s-th feature cross-attention block :

[0078]

[0079] Among them, Indicates a deformation operation in the vertical dimension, Indicates a deformation operation in the vertical dimension. softmax() indicates performing a softmax transformation on the content within the parentheses, i.e., normalizing using the exponential function. T represents transpose, and × represents multiplication operation. Then, deformation operations and normalization are respectively performed on the first panchromatic vector and the second low-resolution vector in the horizontal and vertical dimensions to obtain the second vertical matrix of the s-th feature cross-attention block and the second horizontal matrix of the s-th feature cross-attention block :

[0080]

[0081] Taking the first vertical matrix and the second vertical matrix as the vertical matrix, and the first horizontal matrix and the second horizontal matrix as the horizontal matrix, thus completing the deformation operation and normalization on the mapping vector group. Subsequently, the panchromatic feature correction vector of the s-th feature cross-attention block and the low-spectral feature correction vector of the s-th feature cross-attention block are obtained, and they are used as the feature correction vector of the s-th feature cross-attention block:

[0082]

[0083]

[0084] Subsequently, the feature correction vector is iterated, i.e., the low-spectral feature correction vector and the panchromatic feature correction vector are respectively used as the input of the attention block and mapped into the Transformer structure to obtain a new panchromatic vector group and a new low-resolution vector group. Finally, a new low-spectral feature correction vector and a new panchromatic feature correction vector are obtained. The number of iterations R is determined according to experience, and this process is executed R times to obtain the target panchromatic feature correction vector of the s-th feature cross-attention block and the target low-resolution feature vector of the s-th feature cross-attention block .

[0085]

[0086] Subsequently, the target panchromatic feature correction vector, the target low-resolution feature vector, and the fusion correction vector are used as the target feature correction vector, and the target feature correction vector is input into the feed-forward neural network. In the feed-forward neural network, convolution, normalization, and residual operations can be performed on the target feature correction vector to obtain the fusion feature matrix of the s-th feature cross-attention block and output it from the feature cross-attention block:

[0087]

[0088] Among them, FFN() represents performing convolution, normalization, and residual operations on the content within the parentheses through a feed-forward neural network on the target feature correction vector. is the multi-spectral fusion feature of the s-th feature cross-attention block. is the panchromatic correction feature of the s-th feature cross-attention block. is the fusion feature of the s-th feature cross-attention block.

[0089] S5: Repeat S3 to S4 to iterate on the fusion feature matrix to obtain the target fusion feature, construct a feature refinement block, and perform layer-by-layer feature fusion and convolution on the target fusion feature through the feature refinement block to obtain a high-resolution multi-spectral image.

[0090] Furthermore, the purpose of this stage is to obtain the target fusion feature, construct a feature refinement block, and use the feature refinement block to obtain a high-resolution multi-spectral image. Specifically, in step S5, the refined feature is obtained through the target fusion feature, and the refined feature is iterated through the feature refinement block and the target fusion feature, and layer-by-layer feature fusion and convolution are performed on the target fusion feature to obtain the target refined feature, and feature refinement and convolution are performed on the target refined feature to obtain the high-resolution multi-spectral image.

[0091] For the above steps, the specific implementation in this embodiment is as follows:

[0092] First, repeat S3 to S4 to iterate on the fusion feature matrix. In this embodiment, it is to obtain the panchromatic attention block input and the low-resolution attention block input of the next feature cross-attention block through the fusion feature matrix and the image scale feature:

[0093]

[0094] And repeat S3 to S4, that is, obtain a new fusion feature matrix in the second and third feature cross-attention blocks, and use the fusion features output by each feature cross-attention block as the target fusion feature.

[0095] Then construct a feature refinement block, and perform layer-by-layer feature fusion and convolution on the target fusion feature through the feature refinement block. Since in this embodiment, the target fusion feature contains a total of three fusion features, the process of layer-by-layer feature fusion is also divided into three layers, and each layer can output a refined feature of the s-th layer :

[0096]

[0097] Among them, The fused features representing the first feature cross-attention block, MFRB() represents inputting the content within the parentheses into the feature refinement block, Ps() represents performing the PixShuffle operation on the content within the parentheses. The PixShuffle operation can refine the image, and the output of the refined feature refinement block is concatenated with to complete the feature fusion of. Among them, in the feature refinement block, for each input, there is:

[0098]

[0099] Among them, is the first feature refinement branch, is the second feature refinement branch. Linear() represents performing a fully connected operation on the content within the parentheses through a fully connected layer, DWConv() represents performing a depth convolution on the content within the parentheses, SiLU() represents activating the content within the parentheses through an s-shaped weighted linear unit activation function, ESS2D() represents performing an ESS2D mechanism operation on the content within the parentheses, and α is a learnable scaling factor. is the output of the feature refinement block.

[0100] Iteratively refine the features layer by layer to complete the layer-by-layer fusion and convolution of the target fused features, and use the refined features of the last layer, which is the 3rd layer in this embodiment, as the target refined features . Then, perform feature refinement and convolution on the target refined features to obtain a high-resolution multispectral image :

[0101]

[0102] To verify the effectiveness of the present invention, on the public dataset WV3, the high-resolution multispectral images output by the method provided by the present invention and other methods were compared, and a comparison graph of the mean absolute error graphs of the high-resolution multispectral images output by the method of the present invention and other methods was obtained, as Figure 2 shown. Among them, LAGNet, BiMPAN, PSCF-Net, and SSMNet are all existing methods. It can be seen that there are more dark blue areas in the mean absolute error graph of the method provided by the present invention, indicating that the fusion error is smaller. It can be seen that the method provided by the present invention has achieved the best resolution and the least distortion. The specific experimental results are compared in Table 1 as follows:

[0103] Table 1 Performance comparison table of high-resolution multispectral images obtained by the method provided by the present invention and other methods

[0104]

[0105] Among them, SSIM is the structural similarity, SCC is the spectral cross-correlation, SAM is the spectral angle mapping, RMSE is the root mean square error, ERGAS is the dimensionless global relative error, PSNR is the peak signal-to-noise ratio. An upward arrow indicates that the higher the value, the better, and a downward arrow indicates that the lower the value, the better. An underline indicates that the value of this method achieves a sub-optimal performance in this item, and bold indicates that the value of this method achieves the best performance in this item. It can be seen that the method provided by the present invention achieves the best performance in most values.

[0106] The remote sensing multispectral image fusion device based on multi-stage feature correction provided by the present invention will be described below. The remote sensing multispectral image fusion device based on multi-stage feature correction described below can be correspondingly referred to the remote sensing multispectral image fusion method based on multi-stage feature correction described above.

[0107] Figure 3 The structural schematic diagram of the remote sensing multispectral image fusion system based on multi-stage feature correction is exemplified, as Figure 3 shown, for performing the remote sensing multispectral image fusion method based on multi-stage feature correction as described above, including:

[0108] Panchromatic image correction module 100: for obtaining a low-resolution multispectral image and a panchromatic image, obtaining a spectral mask through the low-resolution multispectral image and the panchromatic image, and covering the panchromatic image with the spectral mask to obtain a corrected panchromatic image;

[0109] Image scale feature module 200: for enhancing the scale features of the low-resolution multispectral image through a multi-scale channel attention block, and performing downsampling and convolution on the corrected panchromatic image to obtain image scale features;

[0110] Mapping vector group module 300: for obtaining the input of the attention block through the image scale features, and mapping the input of the attention block to obtain a mapping vector group including a panchromatic vector group and a low-resolution vector group;

[0111] Fusion feature matrix module 400: for performing image correction using the mapping vector group to obtain a feature correction vector, iterating the feature correction vector to obtain a target feature correction vector, and inputting the target feature correction vector into a feedforward neural network to obtain a fusion feature matrix;

[0112] High-resolution multispectral image module 500: for iterating the fusion feature matrix by repeating the mapping vector group module to the fusion feature matrix module to obtain a target fusion feature, constructing a feature refinement block, and performing layer-by-layer feature fusion and convolution on the target fusion feature through the feature refinement block to obtain a high-resolution multispectral image.

[0113] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0114] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the essence of the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0115] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of each embodiment of the present invention.

Claims

1. A remote sensing multispectral image fusion method based on multi-stage feature correction, characterized in that Including: S1: Obtain a low-resolution multispectral image and a panchromatic image, obtain a spectral mask from the low-resolution multispectral image and the panchromatic image, and use the spectral mask to cover the panchromatic image to obtain a corrected panchromatic image; S2: Enhance the scale features of the low-resolution multispectral image through a multi-scale channel attention block, and perform downsampling and convolution on the corrected panchromatic image to obtain image scale features; S3: Obtain the input of the attention block from the image scale features, map the input of the attention block, and obtain a mapped vector group including a panchromatic vector group and a low-resolution vector group; S4: Use the mapped vector group for image correction to obtain a feature correction vector, iterate the feature correction vector to obtain a target feature correction vector, and input the target feature correction vector into a feed-forward neural network to obtain a fused feature matrix; S5: Repeat S3 to S4 to iterate the fused feature matrix to obtain a target fused feature, construct a feature refinement block, and perform layer-by-layer feature fusion and convolution on the target fused feature through the feature refinement block to obtain a high-resolution multispectral image.

2. The remote sensing multispectral image fusion method based on multi-stage feature correction according to claim 1, wherein In step S1, upsample the low-resolution multispectral image to obtain an upsampled multispectral image, perform a local entropy operation on the upsampled multispectral image, and perform a guided filtering operation and normalization on the upsampled multispectral image after the local entropy operation and the panchromatic image to obtain a spectral mask.

3. The remote sensing multispectral image fusion method based on multi-stage feature correction according to claim 1, characterized in that In step S1, multiply the spectral mask and the panchromatic image point by point to complete the covering of the panchromatic image.

4. The remote sensing multispectral image fusion method based on multi-stage feature correction according to claim 1, wherein In step S2, obtain the input of the multi-scale channel attention block from the low-resolution multispectral image. In the multi-scale channel attention block, perform multi-scale convolution and channel attention operation on the input of the multi-scale channel attention block to obtain low-resolution scale features.

5. The remote sensing multispectral image fusion method based on multi-stage feature correction according to claim 4, wherein In step S2, perform convolution and downsampling on the corrected panchromatic image to obtain panchromatic scale features, and obtain the image scale features from the panchromatic scale features and the low-resolution scale features.

6. The remote sensing multispectral image fusion method based on multi-stage feature correction according to claim 1, wherein In step S3, map the input of the attention block into a Transformer structure to obtain a panchromatic vector group and a low-resolution vector group respectively.

7. The remote sensing multispectral image fusion method based on multi-stage feature correction according to claim 1, characterized in that In step S4, perform a deformation operation and normalization on the mapped vector group to obtain a vertical matrix and a horizontal matrix, obtain a feature correction vector including a panchromatic feature correction vector and a low-spectral feature correction vector from the vertical matrix and the horizontal matrix, use the low-spectral feature correction vector and the panchromatic feature correction vector as the input of the attention block and iterate the feature correction vector to obtain a target feature correction vector.

8. The remote sensing multispectral image fusion method based on multi-stage feature correction according to claim 1, characterized in that In step S4, input the target feature correction vector into a feed-forward neural network, and perform convolution, normalization, and residual operations on the target feature correction vector in the feed-forward neural network to obtain the fused feature matrix.

9. The remote sensing multispectral image fusion method based on multi-stage feature correction according to claim 1, wherein, In step S5, obtain refined features from the target fused feature, iterate the refined features through the feature refinement block and the target fused feature, perform layer-by-layer feature fusion and convolution on the target fused feature to obtain target refined features, and perform feature refinement and convolution on the target refined features to obtain the high-resolution multispectral image.

10. A remote sensing multispectral image fusion system based on multi-stage feature correction, for performing the remote sensing multispectral image fusion method based on multi-stage feature correction according to any one of claims 1 to 9, characterized in that, Including: Corrected Panchromatic Image Module: It is used to obtain a low-resolution multispectral image and a panchromatic image, obtain a spectral mask from the low-resolution multispectral image and the panchromatic image, and use the spectral mask to cover the panchromatic image to obtain a corrected panchromatic image; Image Scale Feature Module: It is used to enhance the scale features of the low-resolution multispectral image through a multi-scale channel attention block, and perform downsampling and convolution on the corrected panchromatic image to obtain image scale features; Mapped Vector Group Module: It is used to obtain the input of the attention block from the image scale features, and map the input of the attention block to obtain a mapped vector group including a panchromatic vector group and a low-resolution vector group; Fused Feature Matrix Module: It is used to perform image correction using the mapped vector group to obtain a feature correction vector, iterate the feature correction vector to obtain a target feature correction vector, and input the target feature correction vector into a feedforward neural network to obtain a fused feature matrix; High-Resolution Multispectral Image Module: It is used to repeat the Mapped Vector Group Module to the Fused Feature Matrix Module to iterate the fused feature matrix to obtain a target fused feature, construct a feature refinement block, and perform layer-by-layer feature fusion and convolution on the target fused feature through the feature refinement block to obtain a high-resolution multispectral image.

Images