A remote sensing image fusion method and system based on a two-dimensional RWKV mechanism

Abstract

The application discloses a kind of based on double-dimension RWKV mechanism's remote sensing image fusion method and system, the present application method includes the high-resolution panchromatic remote sensing image and low-resolution multispectral remote sensing image to be fused input depth fusion network RWPNet, to obtain high-resolution multispectral remote sensing image, the depth fusion network RWPNet includes up-sampling operation, shallow feature extraction stage, deep feature extraction stage, double-branch fusion stage and reconstruction stage, deep feature extraction stage utilizes double-dimension RWKV module to extract global and local features to panchromatic remote sensing image and multispectral remote sensing image, double-dimension RWKV module includes spatial mixing unit, local detail perception unit and channel mixing unit.The present application aims to solve the deficiency of existing method in spectral consistency maintenance, spatial detail recovery, panchromatic-multispectral long-range dependence relationship construction and computational efficiency, improve the spectral fidelity of fused image, structure sharpness.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of remote sensing image processing, and particularly relates to a remote sensing image fusion method and system based on a double-dimension RWKV mechanism. BACKGROUND

[0002] Remote sensing imaging technology plays a key role in the fields of environmental monitoring, urban planning, agricultural management, resource investigation, and military reconnaissance. Current mainstream optical remote sensing satellites usually acquire high-resolution panchromatic images and low-resolution multispectral images simultaneously. The panchromatic remote sensing images have rich spatial details, while the multispectral remote sensing images contain complete spectral information. In order to obtain images with both high spatial resolution and high spectral fidelity, researchers generally use panchromatic remote sensing image sharpening technology (Pansharpening) to combine the two types of images to generate images with both high resolution and rich spectral information. However, how to maintain the spectral consistency of multispectral remote sensing images and fully utilize the spatial structure information provided by panchromatic remote sensing images during the fusion process is still an important problem in the field of remote sensing image processing. Traditional fusion methods, such as IHS, Brovey, PCA, and wavelet transform, can obtain good results in some scenarios, but they rely on manually designed fusion rules and are difficult to model complex spatial relationships and cross-spectral characteristics, which can easily cause obvious spectral distortion and detail blur. With the development of deep learning, convolutional neural networks (CNN) and Transformer architectures have been widely applied to remote sensing fusion tasks. These methods automatically extract spatial and spectral features through end-to-end learning, significantly outperforming traditional methods in performance. However, the receptive field of convolutional computation is limited, making it difficult to capture long-range dependencies. While the Transformer has the ability to model global relationships, it is accompanied by high computational overhead and the need for a large amount of training data, especially when the dimension of multispectral remote sensing images is large. Its complexity and memory consumption will increase dramatically, making it difficult to apply the model in resource-constrained remote sensing scenarios. In recent years, the RWKV (Receiving Weighted Key-Value) model has attracted attention as a new architecture that combines the linear time complexity of inference efficiency with the long-range dependency modeling capability of the Transformer. The RWKV mechanism achieves global modeling of sequences through the weighted key-value (WKV) mechanism and avoids the quadratic complexity of self-attention, making it have a significant advantage in long sequence modeling. However, the original RWKV mechanism is designed for one-dimensional sequence tasks and cannot be directly used to process two-dimensional structure data of remote sensing images, and it has limitations in extracting local detail information. Therefore, how to effectively extend the RWKV mechanism to the image field and construct a two-dimensional spatial feature representation and processing mechanism suitable for remote sensing image fusion has become a problem that needs to be solved. SUMMARY

[0003] The technical problem this invention aims to solve is as follows: Addressing the shortcomings of existing remote sensing image fusion techniques, such as the inability of convolution operations to effectively capture long-range dependencies, the high computational cost of Transformer structures, and insufficient multimodal feature interaction leading to spectral bias and detail loss, this invention provides a remote sensing image fusion method and system based on a two-dimensional RWKV mechanism. This invention aims to overcome the deficiencies of existing methods in maintaining spectral consistency, restoring spatial details, and constructing panchromatic-multispectral long-range dependencies, thereby improving the spectral fidelity and structural sharpness of the fused image.

[0004] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows: A remote sensing image fusion method based on a two-dimensional RWKV mechanism includes inputting a high-resolution panchromatic remote sensing image and a low-resolution multispectral remote sensing image to be fused into a deep fusion network RWPNet to obtain a high-resolution fused multispectral remote sensing image. The processing of the deep fusion network RWPNet includes: The upsampling operation is performed on the low-resolution multispectral remote sensing image by the upsampling module to make its spatial size consistent with that of the high-resolution panchromatic remote sensing image to be fused. In the shallow feature extraction stage, shallow feature extraction is performed on the high-resolution panchromatic remote sensing image to be fused and the upsampled multispectral remote sensing image, and the original image is mapped to shallow features. In the deep feature extraction stage, the shallow features of panchromatic remote sensing images and multispectral remote sensing images are extracted using a dual-dimensional RWKV module to extract deep global and local features. The dual-dimensional RWKV module includes a spatial blending unit, a local detail perception unit, and a channel blending unit. The spatial blending unit and the local detail perception unit are connected in sequence. The output features and input features of the spatial blending unit are used as the input features of the local detail perception unit after residual connection. The output features and input features of the channel blending unit are used as the output of the dual-dimensional RWKV module after residual connection. In the dual-branch fusion stage, deep features of the two branches, panchromatic remote sensing image and multispectral remote sensing image, are extracted and enhanced element-wise through channel shuffling and parallel dual-dimensional RWKV module to achieve deep interaction and obtain fused features. In the reconstruction stage, the fused features are reconstructed to obtain high-resolution multispectral remote sensing images.

[0005] Optionally, the shallow feature extraction stage includes: A 3×3 convolutional layer is used to extract shallow features from both the high-resolution panchromatic remote sensing image to be fused and the upsampled multispectral remote sensing image, thus obtaining shallow features from both branches of the panchromatic and multispectral remote sensing images.

[0006] Optionally, the spatial blending unit's processing of input features includes: performing layer normalization on the input features of the spatial blending unit; mapping the layer-normalized input features into an acceptability matrix, a key matrix, and a value matrix through three 1×1 convolutional layers, respectively, while preserving the two-dimensional spatial structure of the features; activating the acceptability matrix using a Sigmoid activation function; performing bidirectional WKV calculation on the key matrix and value matrix using a bidirectional weighted key-value attention mechanism to obtain global spatial attention features; performing a dot product operation on the global spatial attention features and the acceptability matrix activated by the Sigmoid activation function to obtain the dot product result; performing channel-dimensional WKV attention calculation on the global spatial attention features, while simultaneously obtaining channel WKV results through a Sigmoid activation function; and adding the output features obtained by multiplying the dot product result and the channel WKV result together with the dot product result to obtain the output features of the spatial blending unit.

[0007] Optionally, the processing of input features by the local detail-aware unit includes: extracting local detail information from the input features of the local detail-aware unit through a depthwise separable convolutional layer; adding the extracted local detail information to the input features of the local detail-aware unit through a residual connection; and finally performing a convolution operation on the added features through a 3×3 convolutional layer to obtain the output features of the local detail-aware unit.

[0008] Optionally, the processing of input features by the channel mixing unit includes: performing layer normalization on the input features of the channel mixing unit; mapping the layer-normalized input features into an acceptance matrix and a key matrix respectively through a linear mapping layer; activating the acceptance matrix using the Sigmoid activation function; activating the key matrix using the SquaredReLU activation function; and performing a dot product calculation on the two output features obtained from the activation process to obtain the output features of the channel mixing unit.

[0009] Optionally, the bidirectional WKV calculation includes: employing a horizontal zigzag scanning path strategy to spatially traverse the input two-dimensional key matrix and value matrix, rearranging them into a one-dimensional key sequence and value sequence; performing a first WKV calculation on the one-dimensional key sequence and value sequence according to the horizontal order of the sequence to capture the long-distance dependencies of image features on the horizontal path; restoring the one-dimensional key sequence and using a vertical zigzag scanning path strategy as the vertically traversed key sequence, and restoring the output features of the first WKV calculation to a two-dimensional form and using a vertical zigzag scanning path strategy as the vertically traversed value sequence; performing a second WKV calculation on the vertically traversed key sequence and value sequence according to the vertical order of the sequence to capture the long-distance dependencies of image features on the vertical path to obtain global spatial attention features.

[0010] Optionally, the channel-dimensional WKV attention calculation of the global spatial attention features includes: compressing the input features of the channel WKV calculation module in the spatial dimension through average pooling to obtain a one-dimensional sequence of channel features; mapping the one-dimensional sequence of channel features to channel key sequences and value sequences through linear layers respectively, and then performing WKV calculation on the key sequences and value sequences to capture long-distance dependencies between different spectral bands, while generating enhanced channel feature weights through a Sigmoid activation function; multiplying the channel feature weights and the input features of the channel WKV calculation module to obtain the output features of the channel WKV calculation module.

[0011] Optionally, the dual-branch fusion stage includes multiple cascaded dual-branch cross-fusion modules. The processing of input features by these modules includes: performing a channel-dimensional shuffling operation on the features of the panchromatic remote sensing image and the multispectral remote sensing image, so that the features of the two branches are alternately arranged in the channel dimension and used as input features for the two fusion branches respectively; each of the two fusion branches includes a dual-dimensional RWKV module and a dot product module, and the two fusion branches share an addition module and a convolution module. The dual-dimensional RWKV module extracts features from the mixed feature maps of the corresponding branches and outputs them to the dot product module and addition module of the corresponding fusion branch. The addition module adds the features of the two fusion branches and then performs convolution processing through the convolution module. The dot product module performs a dot product operation on the features of the fusion branch and the convolutionally processed features to obtain the output features of the fusion branch; finally, the output features of the two fusion branches are added together and then processed by a convolution module to extract the fused features output by the dual-branch cross-fusion module.

[0012] Optionally, the reconstruction stage includes: passing the fused features sequentially through a two-level dual-dimensional RWKV module, two 3×3 convolutional layers, and a residual module, and then adding the output features of the last 3×3 convolutional layer and the upsampled multispectral remote sensing image through the residual module to obtain a fused multispectral remote sensing image.

[0013] The present invention also provides a remote sensing image fusion system based on a two-dimensional RWKV mechanism, including a microprocessor and a memory interconnected thereto, wherein the microprocessor is programmed or configured to execute the remote sensing image fusion method based on the two-dimensional RWKV mechanism.

[0014] The present invention also provides a computer-readable storage medium storing a computer program or instructions that are programmed or configured to execute the remote sensing image fusion method based on the two-dimensional RWKV mechanism by a processor.

[0015] The present invention also provides a computer program product, including a computer program or instructions, which are programmed or configured to execute the remote sensing image fusion method based on the two-dimensional RWKV mechanism via a processor.

[0016] Compared with existing technologies, this invention mainly achieves the following beneficial effects: 1. The dual-dimensional RWKV module proposed in this invention can capture and represent long-range dependencies in both the spatial and spectral dimensions, solving the problems of limited receptive field of traditional convolution, huge computational overhead of Transformer, and lack of local detail capture capability, thus realizing efficient collaborative extraction of spatial structure information and spectral correlation information. 2. The dual-branch fusion stage designed in this invention realizes deep interaction between panchromatic and multispectral branch features. Through shuffling, parallel dual-dimensional RWKV feature extraction, and element-wise enhancement mechanisms, it achieves efficient fusion of panchromatic and multispectral information, effectively improving the spectral consistency and spatial detail quality of the fused image. 3. The reconstruction module proposed in this invention combines the global dependency capture capability of RWKV with the fine expression capability of convolutional operations for local details, achieving deep restoration of spatial details and accurate preservation of spectral structure, making the generated high-resolution multispectral image significantly superior to traditional methods in terms of visual quality and performance metrics. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the network structure of the deep fusion network RWPNet in an embodiment of the present invention.

[0018] Figure 2 This is a schematic diagram of the network structure of the two-dimensional RWKV module in an embodiment of the present invention.

[0019] Figure 3 This is a schematic diagram of the network structure of the spatial mixing unit in an embodiment of the present invention.

[0020] Figure 4 This is a schematic diagram of the network structure of the local detail perception unit in an embodiment of the present invention.

[0021] Figure 5 This is a schematic diagram of the network structure of the channel mixing unit in an embodiment of the present invention.

[0022] Figure 6 This is a schematic diagram of the network structure of the bidirectional WKV calculation module in an embodiment of the present invention.

[0023] Figure 7 This is a schematic diagram of the network structure of the channel WKV calculation module in an embodiment of the present invention.

[0024] Figure 8This is a schematic diagram of the network structure of the dual-branch cross-fusion module in an embodiment of the present invention. Detailed Implementation

[0025] To enable those skilled in the art to better understand the technical solutions of the present invention, the technical solutions of the present invention will be further described in detail below with reference to the accompanying drawings in the embodiments of the present invention.

[0026] This embodiment of the remote sensing image fusion method based on the two-dimensional RWKV mechanism includes inputting a high-resolution panchromatic remote sensing image (PAN image) and a low-resolution multispectral remote sensing image (LRMS image) to be fused into a deep fusion network RWPNet to obtain a high-resolution fused multispectral remote sensing image (HRMS image). In this embodiment, the size of the high-resolution panchromatic remote sensing image to be fused is... ,in and These represent the height and width of the image, respectively; the size of a low-resolution multispectral remote sensing image is... ,in Indicates the number of spectral bands. This indicates the spatial resolution ratio.

[0027] like Figure 1 As shown, the processing of the deep fusion network RWPNet in this embodiment includes: The upsampling operation is performed on the low-resolution multispectral remote sensing image by the upsampling module to make its spatial size consistent with that of the high-resolution panchromatic remote sensing image to be fused. In the shallow feature extraction stage, shallow feature extraction is performed on the high-resolution panchromatic remote sensing image to be fused and the upsampled multispectral remote sensing image, and the original image is mapped to shallow features. In the deep feature extraction stage, the shallow features of panchromatic remote sensing images and multispectral remote sensing images are extracted using a dual-dimensional RWKV module to extract deep global and local features. The dual-dimensional RWKV module includes a spatial blending unit, a local detail perception unit, and a channel blending unit. The spatial blending unit and the local detail perception unit are connected in sequence. The output features and input features of the spatial blending unit are used as the input features of the local detail perception unit after residual connection. The output features and input features of the channel blending unit are used as the output of the dual-dimensional RWKV module after residual connection. In the dual-branch fusion stage, deep features of the two branches, panchromatic remote sensing image and multispectral remote sensing image, are extracted and enhanced element-wise through channel shuffling and parallel dual-dimensional RWKV module to achieve deep interaction and obtain fused features. In the reconstruction stage, the fused features are reconstructed to obtain high-resolution multispectral remote sensing images.

[0028] To unify the spatial scale, this embodiment uses an upsampling module to upsample the low-resolution multispectral remote sensing image so that its spatial size is consistent with that of the panchromatic remote sensing image, thus obtaining a spatially upsampled multispectral remote sensing image (UPMS image). Subsequently, the UPMS image and the high-resolution panchromatic remote sensing image (PAN image) are respectively fed into the shallow feature extraction stage of the network.

[0029] like Figure 1 As shown, the shallow feature extraction stage in this embodiment includes: A 3×3 convolutional layer is used to extract shallow features from both the high-resolution panchromatic remote sensing image to be fused and the upsampled multispectral remote sensing image, thus obtaining shallow features from both branches of the panchromatic and multispectral remote sensing images.

[0030] The shallow feature extraction stage primarily utilizes separate 3×3 convolutional layers to map the PAN and UPMS images into the feature space. Specifically, let the convolution operation be... If the upsampling operation is Upsample, then the extracted PAN features and multispectral features It can be represented as as well as This step lays the foundation for subsequent deep feature interactions, ensuring that the initial features retain their respective characteristics while having the same channel dimension and spatial size.

[0031] The deep fusion network RWPNet in this embodiment consists of three core stages: deep feature extraction stage, dual-branch fusion stage, and reconstruction stage.

[0032] In the deep feature extraction stage, a two-dimensional RWKV module is used to extract deep features. Subsequently, the extracted deep features from both the panchromatic and multispectral remote sensing images are fed into the two-branch fusion stage. To achieve sufficient mixing of the two types of image features, this embodiment employs multiple cascaded two-branch cross-fusion modules in the two-branch fusion stage. The input to the second dual-branch cross-fusion module is the first... The fusion results of the two-branch cross-fusion modules and the features of the panchromatic remote sensing image branches are used to output the fused features through deep exchange and fusion, as shown below. Finally, the reconstruction module maps the fused features back to the image space and uses residual connections to add the reconstructed image to the upsampled LRMS image, obtaining a high-resolution fused multispectral remote sensing image (HRMS image), which can be represented as: .

[0033] in, For HRMS images, For upsampling operation, This is the upsampled multispectral remote sensing image. To rebuild the module, This is a feature of fusion.

[0034] like Figure 2 As shown, the Two-Dimensional RWKV module (DD-RWKV) includes a spatial mixing unit, a local detail-aware unit, and a channel mixing unit. The spatial mixing unit and the local detail-aware unit are connected sequentially. The output and input features of the spatial mixing unit are residually concatenated to serve as the input features of the local detail-aware unit. Similarly, the output and input features of the channel mixing unit are residually concatenated to serve as the RWKV features output by the two-dimensional RWKV module. The two-dimensional RWKV module (DD-RWKV) aims to address the high computational complexity of traditional Transformers and the limited receptive field of traditional CNNs. This module adopts the design concept of Vision-RWKV, consisting of two main parts: spatial mixing and channel mixing. However, its internal structure has been improved for the pan-color sharpening task. Specifically, the DD-RWKV module introduces channel WKV (C-WKV) computation and a local detail-aware unit into the spatial mixing unit, establishing a closer connection between spatial mixing and channel mixing.

[0035] The primary goal of spatial blending units is to establish long-distance dependencies between image tokens in a spatial dimension. For example... Figure 3 As shown, the spatial blending unit in this embodiment processes the input features as follows: It performs layer normalization on the input features; through three 1×1 convolutional layers, it maps the normalized input features into an acceptability matrix, a key matrix, and a value matrix, respectively, while preserving the two-dimensional spatial structure of the features; it activates the acceptability matrix using the Sigmoid activation function; it performs bidirectional WKV calculation on the key matrix and value matrix using a bidirectional weighted key-value attention mechanism to obtain global spatial attention features; it performs a dot product operation on the global spatial attention features and the acceptability matrix activated by the Sigmoid activation function to obtain the dot product result; it performs channel-dimensional WKV attention calculation and Sigmoid activation on the global spatial attention features to obtain channel WKV results; and it adds the output features obtained by multiplying the dot product result and the channel WKV result to the dot product result to obtain the output features of the spatial blending unit. The working principle of the spatial blending unit is as follows: Given input features... First, layer normalization is performed using a layer normalization module to maintain training stability, resulting in... Next, unlike the linear layers used in standard RWKV, this embodiment uses three parallel layers. Convolutional layers will Mapped to accept ( ),key( ) and value ( Three matrices. Using The advantage of using convolutional layers instead of fully connected linear layers is that it can perform linear transformations on the channel dimension of each pixel while sharing weights and maintaining the two-dimensional spatial grid structure, thus preserving the translational equivariance of the image. (Generated) , , The data is then flattened in the spatial dimension and fed into the bidirectional WKV computation module (Bi-WKV module). A bidirectional WKV computation mechanism is used here, performing WKV operations in two directions to capture more comprehensive global contextual information. The calculated global spatial attention features are denoted as... Subsequently, the cells processed using the Sigmoid activation function... As a gating mechanism, it is related to global spatial attention features Perform element-wise multiplication to obtain the dot product. Next, The inter-channel attention relationship is calculated in the WKV calculation module of the input channel. Output characteristics of spatial mixing unit Finally, it is obtained through a residual connection structure (residual module), and the specific formula can be expressed as: ; The role of the Local Detail Aware Unit (CRM) is to inject local details into RWKV features. For example... Figure 4 As shown, the local detail-aware unit in this embodiment processes the input features as follows: it extracts local detail information from the input features of the local detail-aware unit through a depthwise separable convolutional layer; it adds the extracted local detail information to the input features of the local detail-aware unit through a residual connection; and finally, it performs a convolution operation on the added features through a 3×3 convolutional layer to obtain the output features of the local detail-aware unit. Since RWKV feature extraction is inherently good at capturing global dependencies, it ignores local detail features. The local detail-aware unit extracts local information through a depthwise separable convolutional layer and integrates it back into the main feature flow through residual connections. This effectively achieves complementarity between global dependencies and local details without significantly increasing computational overhead.

[0036] Channel blending units are primarily used for feature fusion and nonlinear transformation along the channel dimension. For example... Figure 5As shown, the channel mixing unit in this embodiment processes the input features as follows: it performs layer normalization on the input features; through a linear mapping layer, it maps the normalized input features into an acceptance matrix and a key matrix, respectively; it activates the acceptance matrix using the Sigmoid activation function; it activates the key matrix using the Squared ReLU activation function; and it performs a dot product of the two output features obtained from the activation process to obtain the output features of the channel mixing unit. The input features of the channel mixing unit are denoted as... First, perform layer normalization: ; in, The features are those after layer normalization. Layer normalization is performed. Then, the receptivity matrix of the channel mixing unit is generated through a linear mapping layer. Bond matrix and the key matrix Use the squared ReLU function to generate the value matrix. : ; ; ; in, It is the Sigmoid activation function. and The weights of the linear mapping layer, For squared ReLU function, The weights are denoted as . Finally, the final output of the channel mixing unit can be expressed as . .in, This is the dot product module. This structure controls the flow of information through a gating mechanism, further enhancing the non-linear expressive power of features.

[0037] In this embodiment, the key matrix and value matrix are processed using a bidirectional weighted key-value attention mechanism to perform bidirectional WKV calculations to obtain global spatial attention features. This is implemented using a bidirectional WKV calculation module. Figure 6As shown, the bidirectional WKV calculation module in this embodiment includes: employing a horizontal zigzag scanning path strategy to spatially traverse the input two-dimensional key matrix and value matrix, rearranging them into a one-dimensional key sequence and value sequence; performing a first WKV calculation on the one-dimensional key sequence and value sequence according to the horizontal order of the sequence to capture the long-distance dependencies of image features on the horizontal path; restoring the one-dimensional key sequence and using a vertical zigzag scanning path strategy as the vertically traversed key sequence, and using the output features of the first WKV calculation as the vertically traversed value sequence according to the vertical zigzag scanning path strategy; performing a second WKV calculation on the vertically traversed key sequence and value sequence according to the vertical order of the sequence to capture the long-distance dependencies of image features on the vertical path to obtain global spatial attention features.

[0038] When performing WKV computation on one-dimensional key and value sequences in the horizontal order of the sequences to capture long-distance dependencies of image features along the horizontal path, for lengths of... key / value sequence , ,in Key sequence The first in One key, Value sequence The first in The nth value, then the nth indivual Output for: ; in, express The result of WKV calculation at time t, where For learnable vectors, First, expand the input two-dimensional key matrix and value matrix row-wise into a one-dimensional form. and The output sequence of the first WKV calculation is: ; ; The sequence is then restored to a two-dimensional layout: ;

[0039] in, Defined as the operation of restoring a two-dimensional structure from a horizontal orientation.

[0040] During the second WKV calculation, the output of the first stage is... The value sequence is expanded column-wise and used as the second-level WKV calculation. At the same time, the key matrix is ​​expanded vertically in the same way, serving as the key matrix for the second-level WKV calculation. : The output sequence of the second WKV calculation is: ; ; Similarly, the summation form of WKV is completely equivalent to that described above, and finally it is restored to two dimensions. For longitudinal restoration operation: .

[0041] To further supplement the feature information captured only in the spatial dimension, this embodiment adds a channel WKV calculation module within the spatial mixing unit. For example... Figure 7 As shown, in this embodiment, channel-dimensional WKV attention calculation for global spatial attention features includes: compressing the input features of the channel WKV calculation module in the spatial dimension using average pooling to obtain a one-dimensional sequence of channel features; mapping the one-dimensional sequence of channel features to channel key sequences and value sequences through linear layers, then performing WKV calculation on the key sequences and value sequences to capture long-range dependencies between different spectral bands, and processing them with a Sigmoid activation function to generate enhanced channel feature weights; multiplying the channel feature weights with the input features of the channel WKV calculation module to obtain the output features of the channel WKV calculation module. The input features of the channel WKV calculation module are compressed in the spatial dimension using average pooling to obtain a one-dimensional sequence of channel features. , can be represented as: ; in, For global average pooling, The input features of the channel WKV calculation module are mapped to the channel key sequences through linear layers. Sum sequence It can be represented as: ; ; in, and For the weights of the linear layer. Similarly, global attention in the channel dimension is obtained through WKV calculation, and finally, the spatial dimension is expanded through a broadcast mechanism and multiplied with the spatial WKV calculation result to enhance the relationship between channels, which can be expressed as: ; in, This is the output of the channel WKV calculation module. Key sequence Sum sequence The WKV calculation results, The input features are those of the channel WKV computation module. After obtaining the global spatial attention features, the channel WKV computation module first performs average pooling on them in the spatial dimension to obtain a vector that can represent global channel information. Then, this vector is treated as a sequence input, and the WKV computation module uses the recurrent structure of RWKV to explicitly capture long-range dependencies in the channel dimension. This differs from traditional attention modules; the channel WKV computation module (C-WKV) achieves more dynamic and context-aware channel modulation through a recurrent mechanism.

[0042] The Dual-Branch Cross-Fusion Module (DCFM) aims to achieve deep fusion of panchromatic image (PAN) features and multispectral image (LRMS) features. For example... Figure 8 As shown, the dual-branch cross-fusion module (DCFM) in this embodiment processes the input features as follows: For the features of the panchromatic remote sensing image and the multispectral remote sensing image, a channel-dimensional shuffling operation is performed, so that the features of the two branches are alternately arranged in the channel dimension and used as input features for two independent fusion branches respectively; each of the two independent fusion branches includes a dual-dimensional RWKV module and a dot product module, and the two fusion branches share an addition module and a convolution module. The dual-dimensional RWKV module mixes the corresponding branches... Feature extraction is performed on the feature map. The dual-dimensional RWKV module extracts features from the feature map after the corresponding branches are mixed, and then outputs them to the dot product module of the corresponding fusion branch and the addition module shared by the two fusion branches. The addition module adds the features of the two fusion branches and then performs convolution processing through the convolution module. The dot product module performs a dot product operation on the features of the fusion branch and the convolution-processed features to obtain the output features of the fusion branch. Finally, the output features of the two fusion branches are added together and then processed by a convolution module to extract the fused features output by the dual-branch cross-fusion module.

[0043] To avoid information redundancy and feature conflicts caused by direct splicing, the dual-branch cross-fusion module adopts a structured fusion strategy. First, it receives high-dimensional features from the PAN and LRMS branches. To facilitate sufficient information exchange between the two images, a joint channel shuffle operation is performed on the input PAN and LRMS features. Channel shuffle interleaves and reorganizes the channels of the two types of image features, enabling subsequent processing units to simultaneously capture feature information from both sources. The shuffled features are then fed into two independent fusion branches. These two branches process the fused features separately, utilizing the aforementioned global-local feature capture capability to further extract long-distance dependencies and complementary features from the shuffled image. After the parallel branch output, a dot product operation is used to fuse the two sets of features. This interactive approach effectively enhances the common salient features of both branches while suppressing non-informative background noise and redundant information. Finally, the fused features are compressed and reconstructed through a convolutional layer, outputting a unified fused feature representation, which serves as input to the next layer or enters the reconstruction stage.

[0044] The reconstruction phase, the final stage of the network, is responsible for mapping the deep fusion features back to the target image space, i.e., the high-resolution multispectral image space. For example... Figure 1 As shown, the reconstruction stage includes: sequentially passing the fused features through a two-level dual-dimensional RWKV module, two 3×3 convolutional layers, and a residual module. The residual module then adds the output features of the final 3×3 convolutional layer to the spatially upsampled multispectral remote sensing image to obtain the fused multispectral remote sensing image. The fused features are first further refined through a two-level dual-dimensional RWKV module to ensure sufficient global contextual information and local details before restoring the image size. Subsequently, the number of channels is adjusted through two levels of 3×3 convolutional layers to match the number of bands in the target multispectral image. As mentioned earlier, the reconstruction process uses residual connections; the network learns a high-frequency detail residual map, which is added element-wise to the upsampled low-resolution multispectral image to ultimately generate an HRMS image containing rich spatial details and high spectral fidelity.

[0045] In this embodiment, the network parameters of the deep fusion network RWPNet are determined by optimizing the loss function to minimize it during training using the training set. The training set includes the input low-resolution multispectral image, high-resolution panchromatic image, and a reference high-resolution multispectral image (Ground Truth, GT) as a supervision signal. The loss function refers to the error between the fused high-resolution image obtained by using the RWPNet network model on the input PAN and LRMS images and the corresponding reference ground truth image. As an optional implementation, in this embodiment, the remote sensing image panchromatic sharpening network model uses the L1 loss function during training to minimize the pixel-level absolute difference between the generated image and the real image. Specifically, the function expression of the loss function used by the deep fusion network RWPNet during training is: ; in, For loss function, To deeply fuse the network parameters of the RWPNet network, This represents the number of samples in the training batch. For the deep fusion network RWPNet, RWPNet, a deep fusion network based on network parameters right The generated fused multispectral remote sensing image; For the first Multispectral remote sensing images of a sample; For the first Panchromatic remote sensing image corresponding to each sample; For the first Reference multispectral remote sensing image of each sample, It has the L1 norm. This loss function... This is used to define the optimization objective throughout the end-to-end training phase, guiding the iterative update of network weight parameters to minimize the pixel differences between the reconstructed high-resolution multispectral image and the real reference image. Compared to mean squared error (L2 loss), the L1 loss function has better robustness, reducing the model's sensitivity to abnormal pixel values ​​and effectively suppressing the over-smoothing effect of the reconstructed image. This better preserves the spatial structural clarity and texture restoration integrity of the image, achieving a balance between spatial detail enhancement and spectral fidelity.

[0046] To verify the effectiveness of the method in this embodiment, two mainstream remote sensing satellite datasets, Gaofen-2 (GF2) and WorldView-III (WV3), covering different resolutions and a wide range of scenes, were used for evaluation. The experiment followed the Wald protocol to generate a simulated dataset containing a one-to-one correspondence between the original image and the label for evaluation with reference metrics, while full-resolution images were used for evaluation without reference metrics. The number of feature channels in this embodiment was set to 32, and the initial learning rate was set to 2×10⁻⁶. -4 The learning rate decay strategy was set to halve every 120 epochs, with a total of 400 training epochs. The optimizer used was the Adam algorithm. This embodiment's method was mainly compared with three representative traditional methods (AWLP, BDSD-PC, and MTF-GLP), three advanced deep learning methods based on convolutional neural networks (FusionNet, LACNet, and LGPNet), two Transformer-based methods (PanFormer and LGTEUN), and two methods based on the latest Mamba architecture (FusionMamba and MambaDFuse). Objective evaluation metrics included the general image quality index (Q↑), spectral angle mapping (SAM↓), relative global synthesis error (ERGAS↓), spatial correlation coefficient (SCC↑), n-band spectral quality index (Qn↑), and no-reference index (QNR↑). The results are shown in Tables 1 and 2.

[0047] Table 1: Objective evaluation metrics of fusion results of various comparison methods on the WV3 dataset

[0048] As shown in Table 1, on the WorldView-III (WV3) dataset, the method of this embodiment achieved the following results: a general image quality index Q of 0.9149, a spectral angle mapping SAM of 3.0249, a relative global synthesis error ERGAS of 2.2518, a spatial correlation coefficient SCC of 0.9701, an N-band spectral quality index Qn of 0.9132, and a referenceless quality index QNR of 0.9505. It is worth noting that the method of this embodiment achieved optimal values ​​in all six evaluation metrics, demonstrating comprehensive superiority. In particular, compared with the second-best MambaDFuse method, it shows significant advantages in the SAM index (reflecting spectral fidelity) and the ERGAS index (reflecting overall synthesis error) (SAM decreased by 0.0785, and ERGAS decreased by 0.0746), indicating that the method of this embodiment has significant advantages in maintaining spectral consistency and reducing reconstruction errors.

[0049] Table 2: Objective evaluation metrics of fusion results of various comparison methods on the GF2 dataset

[0050] As shown in Table 2, on the Gaofen2 (GF2) dataset, the evaluation metrics of the method in this embodiment are as follows: Q-index 0.9819, SAM 0.7442, ERGAS 0.6604, SCC 0.9823, Qn 0.9813, and QNR 0.9495. The method in this embodiment achieves the best results in all degradation resolution evaluation metrics (Q, SAM, ERGAS, SCC, Qn). For example, it achieves a Q-index of 0.9819, which is better than the second-best method, FusionMamba (0.9801); and a SAM metric of 0.7442, which is significantly better than FusionMamba (0.8082), demonstrating the excellent spectral and spatial fidelity of this model at low resolution scales. Although it is slightly lower than FusionMamba (0.9501) in the full-resolution metric QNR, the difference is extremely small (only 0.0006), and the images generated by the method in this embodiment perform better in terms of detail sharpness and noise suppression.

[0051] To further verify the effectiveness of each key component in the proposed RWPNet network and its contribution to the overall model performance, detailed ablation experiments were conducted on the WorldView-3 (WV3) dataset. The experiments were analyzed from three dimensions: the internal component design of the dual-dimensional RWKV module (DD-RWKV), the necessity of the overall network architecture modules, and the selection of the core algorithm mechanism. The experimental results are shown in Table 3, where "baseline" represents the complete deep fusion network RWPNet proposed in this embodiment.

[0052] Table 3: Objective evaluation indicators of ablation results for each module of the WV3 dataset

[0053] The two-dimensional RWKV module (DD-RWKV) is the core feature extraction component of the deep fusion network RWPNet in this embodiment. It introduces a Local Detail Aware Unit (CRM) and a Channel WKV computation module (C-WKV) on top of the traditional RWKV. To verify the role of these two components, variant models were designed with the CRM removed (without CRM), CWKV removed (without C-WKV), and both removed simultaneously (without CRM & C-WKV). As shown in Table 3, when the CRM is removed, the general image quality index (Q) decreases from 0.9149 to 0.9135, while the Spectral Angle Mapping (SAM) error increases from 3.0249 to 3.0776. This indicates that the local convolution operation introduced by the CRM effectively enhances the model's ability to capture local details. When the channel WKV computation module (C-WKV) is removed, the Q-index decreases to 0.9126, and the no-reference quality metric (QNR) decreases from 0.9505 to 0.9489. This demonstrates that the channel WKV computation module (C-WKV) effectively computes long-range dependencies between spectra by introducing WKV operations in the channel dimension, improving spectral fidelity. When both the Local Detail Aware Unit (CRM) and the channel WKV computation module (C-WKV) are removed simultaneously, i.e., when the model degenerates to the basic structure, the model performance declines significantly, with the Q-index dropping to 0.9099 and the ERGAS error increasing to 2.4184.

[0054] The two-dimensional WKV module, the two-branch cross-fusion module, and the reconstruction module are the key modules of the deep fusion network RWPNet in this embodiment. To verify the rationality of the design of the key modules in this embodiment, experiments were conducted to replace or remove the two-dimensional RWKV feature extraction module, the two-branch cross-fusion module, and the reconstruction module. In Table 3, "DD-RWKV->Conv" indicates that in the feature extraction stage, the two-dimensional RWKV module (DD-RWKV) in the two-dimensional feature extraction module was replaced with a standard 3×3 convolutional layer. The results show that the Q value decreased by 0.0025, and the ERGAS error increased by 0.0624. This indicates that simple convolution operation is limited by the local receptive field and cannot capture long-range spatial-spectral dependencies simultaneously like DD-RWKV. "DCFM->None" indicates that the two-branch cross-fusion module (DCFM) was removed in the feature fusion stage, and only simple element-wise addition was used for fusion. Experimental results show that all indicators decreased to varying degrees (SAM error increased to 3.1159), verifying the important role of the DCFM module in promoting deep interaction between panchromatic and multispectral features and suppressing redundant information. "RECM->Conv" indicates replacing the reconstruction module in the image reconstruction stage with ordinary convolution. The results show that the reconstruction module designed in this embodiment can better preserve detail information when mapping fused features back to the image space.

[0055] Comparative Analysis of Core Methods: To demonstrate the superiority of choosing the improved RWKV as the core mechanism, the core algorithm of this embodiment was compared with other mechanisms. First, the core two-dimensional RWKV module (DD-RWKV) was replaced with a convolutional module, resulting in significantly lower performance than the baseline in spectral angle mapping and relative global synthesis error. Compared with the linear attention mechanism: the model using the linear attention mechanism performed the worst, with a Q-index of only 0.8600 and a SAM error as high as 4.9511. This indicates that ordinary linear attention is difficult to effectively handle the complex spatial-spectral features in the panchromatic sharpening task. Compared with the original Vision-RWKV (VRWKV): the original Vision-RWKV module was used directly without the two-dimensional improvement. The results show that although its performance is better than pure convolution and linear attention, it is still lower than the method in this embodiment (Q-index 0.9127 vs 0.9149). This further confirms that the dual-dimensional (spatial + channel) dual-dimensional RWKV module designed for full-color sharpening tasks in this embodiment is more advantageous than the general VRWKV.

[0056] As can be seen, the ablation experiment results in Table 3 strongly support the technical solution of this embodiment. The complete deep fusion network RWPNet proposed in this embodiment achieved optimal values ​​on all evaluation indicators (Q=0.9149, SAM=3.0249, ERGAS=2.2518, SCC=0.9701, Qn=0.9132, QNR=0.9505), proving that each improved module (DD-RWKV, CRM, C-WKV, DCFM) proposed in the deep fusion network RWPNet plays an indispensable role in improving the full-color sharpening performance, and there is a good synergistic effect among the modules.

[0057] In summary, this embodiment employs the RWPNet deep fusion network specifically designed for panchromatic sharpening tasks. In the feature extraction stage, it innovatively introduces an improved RWKV mechanism, namely the two-dimensional RWKV module (DD-RWKV). This module enhances local context aggregation by introducing a Local Detail Aware Unit (CRM) after spatial mixing, and captures global spectral dependencies by embedding a newly added channel WKV (CWKV) computation into the spatial mixing unit, thus overcoming the limitations of traditional RWKV, which is only applicable to one-dimensional sequences and ignores inter-channel correlations. In the fusion stage, a dual-branch cross-fusion module (DCFM) is designed, utilizing channel shuffle and cross-multiplication operations to promote deep interaction and information complementarity between panchromatic and multispectral features. Extensive experiments demonstrate that this embodiment outperforms existing traditional methods, CNN methods, and state-of-the-art models based on Transformer or Mamba on both the WV3 and GF2 benchmark datasets. Compared to existing technologies, the method in this embodiment has the following significant advantages: 1. This embodiment applies the improved RWKV mechanism to the panchromatic sharpening task, effectively overcoming the technical bottleneck of the traditional Transformer architecture where the computational load increases quadratically with image size by utilizing its linear computational complexity. Simultaneously, compared to traditional convolutional neural networks, the method in this embodiment can more effectively establish long-distance spatial-spectral dependencies, achieving a balance between global feature capture capability and resource consumption while maintaining high computational efficiency. 2. Addressing the two-dimensional spatial characteristics of remote sensing images, the dual-dimensional RWKV feature extraction module designed in this embodiment retains the global dependency capture capability of RWKV while significantly improving the extraction capability of local details through the embedding of detail enhancement units. Furthermore, by introducing channel-dimensional weighted key value calculation (C-WKV), dynamic calculation and representation of the correlation between spectral channels are achieved. This dual-dimensional collaborative design enables the model to effectively restore clear edge structures during the sharpening process while minimizing spectral distortion. 3. To address the issue of feature differences between panchromatic and multispectral images, this embodiment enhances the feature interaction flow between the two branches through a channel shuffling strategy in the dual-branch cross-fusion module. Furthermore, it utilizes a two-dimensional RWKV feature extraction module and convolution operations to achieve adaptive feature selection and complementarity. This design effectively suppresses irrelevant redundant information during the fusion process, ensuring the effective fusion of spatial information from the panchromatic image and spectral information from the multispectral sub-image, thereby significantly improving the quality of the final fused image. This embodiment effectively solves the technical problem of existing methods struggling to balance global dependency capture efficiency with local detail fidelity, reducing spatial distortion and spectral bias, improving spatial detail representation and spectral consistency of the fused image, and exhibiting good generalization ability and computational efficiency.

[0058] Furthermore, this embodiment also provides a remote sensing image fusion system based on a two-dimensional RWKV mechanism, including a microprocessor and a memory interconnected, wherein the microprocessor is programmed or configured to execute the remote sensing image fusion method based on the two-dimensional RWKV mechanism. This embodiment also provides a computer-readable storage medium storing a computer program or instructions programmed or configured to execute the remote sensing image fusion method based on the two-dimensional RWKV mechanism via a processor. This embodiment also provides a computer program product, including a computer program or instructions programmed or configured to execute the remote sensing image fusion method based on the two-dimensional RWKV mechanism via a processor.

[0059] Those skilled in the art will understand that the technical solutions provided by this invention may take the form of a method, system, or computer program product. Therefore, this invention may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this invention may take the form of a computer program product embodied on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, produce an implementation of the flowchart... Figure 1 One or more processes and / or boxes Figure 1 The computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable apparatus for implementing the process. Figure 1 One or more processes and / or boxes ​ The steps of the function specified in one or more boxes.

[0060] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A remote sensing image fusion method based on a two-dimensional RWKV mechanism, characterized in that, The process involves inputting a high-resolution panchromatic remote sensing image and a low-resolution multispectral remote sensing image to be fused into a deep fusion network RWPNet to obtain a high-resolution fused multispectral remote sensing image. The processing of the deep fusion network RWPNet includes: The upsampling operation is performed on the low-resolution multispectral remote sensing image by the upsampling module to make its spatial size consistent with that of the high-resolution panchromatic remote sensing image to be fused. In the shallow feature extraction stage, shallow feature extraction is performed on the high-resolution panchromatic remote sensing image to be fused and the upsampled multispectral remote sensing image, and the original image is mapped to shallow features. In the deep feature extraction stage, the shallow features of panchromatic remote sensing images and multispectral remote sensing images are extracted using a dual-dimensional RWKV module to extract deep global and local features. The dual-dimensional RWKV module includes a spatial blending unit, a local detail perception unit, and a channel blending unit. The spatial blending unit and the local detail perception unit are connected in sequence. The output features and input features of the spatial blending unit are used as the input features of the local detail perception unit after residual connection. The output features and input features of the channel blending unit are used as the output of the dual-dimensional RWKV module after residual connection. In the dual-branch fusion stage, deep features of the two branches, panchromatic remote sensing image and multispectral remote sensing image, are extracted and enhanced element-wise through channel shuffling and parallel dual-dimensional RWKV module to achieve deep interaction and obtain fused features. In the reconstruction stage, the fused features are reconstructed to obtain high-resolution multispectral remote sensing images.

2. The remote sensing image fusion method based on the dual-dimensional RWKV mechanism according to claim 1, characterized in that, The shallow feature extraction stage includes: A 3×3 convolutional layer is used to extract shallow features from both the high-resolution panchromatic remote sensing image to be fused and the upsampled multispectral remote sensing image, thus obtaining shallow features from both branches of the panchromatic and multispectral remote sensing images.

3. The remote sensing image fusion method based on the dual-dimensional RWKV mechanism according to claim 1, characterized in that, The spatial blending unit processes the input features as follows: It performs layer normalization on the input features; it maps the normalized input features into an acceptability matrix, a key matrix, and a value matrix using three 1×1 convolutional layers, preserving the two-dimensional spatial structure of the features; it activates the acceptability matrix using the Sigmoid activation function; it performs bidirectional WKV calculation on the key and value matrices using a bidirectional weighted key-value attention mechanism to obtain global spatial attention features; it performs a dot product operation on the global spatial attention features and the acceptability matrix activated by the Sigmoid activation function to obtain the dot product result; it performs channel-dimensional WKV attention calculation on the global spatial attention features to obtain channel WKV results; and it adds the output features obtained by multiplying the dot product result and the channel WKV result to the dot product result to obtain the output features of the spatial blending unit. The local detail perception unit processes the input features by: extracting local detail information from the input features of the local detail perception unit through a depthwise separable convolutional layer; adding the extracted local detail information to the input features of the local detail perception unit through a residual connection; and finally convolving the added features through a 3×3 convolutional layer to obtain the output features of the local detail perception unit. The channel mixing unit processes the input features by: performing layer normalization on the input features of the channel mixing unit; mapping the layer-normalized input features into an acceptance matrix and a key matrix through a linear mapping layer; activating the acceptance matrix using the Sigmoid activation function; activating the key matrix using the Squared ReLU activation function; and multiplying the two output features obtained from the activation process to obtain the output features of the channel mixing unit.

4. The remote sensing image fusion method based on the dual-dimensional RWKV mechanism according to claim 3, characterized in that, The bidirectional WKV computation includes: employing a horizontal zigzag scanning path strategy to spatially traverse the input two-dimensional key matrix and value matrix, rearranging them into a one-dimensional key sequence and value sequence; performing a first WKV computation on the one-dimensional key sequence and value sequence according to the horizontal order of the sequence to capture the long-distance dependencies of image features on the horizontal path; restoring the one-dimensional key sequence and using a vertical zigzag scanning path strategy as the vertically traversed key sequence; restoring the output features of the first WKV computation into a two-dimensional matrix and using a vertical zigzag scanning path strategy as the vertically traversed value sequence; and performing a second WKV computation on the vertically traversed key sequence and value sequence according to the vertical order of the sequence to capture the long-distance dependencies of image features on the vertical path to obtain global spatial attention features. The channel-dimensional WKV attention calculation for the global spatial attention features includes: compressing the input features of the channel WKV calculation module in the spatial dimension through average pooling to obtain a one-dimensional sequence of channel features; mapping the one-dimensional sequence of channel features to channel key matrices and value matrices through linear layers, and then performing WKV calculation on the key matrices and value matrices to capture long-distance dependencies between different spectral bands, while using a sigmoid activation function to generate enhanced channel feature weights; and multiplying the channel feature weights with the input features of the channel WKV calculation module to obtain the output features of the channel WKV calculation module.

5. The remote sensing image fusion method based on the dual-dimensional RWKV mechanism according to claim 1, characterized in that, The dual-branch fusion stage includes multiple cascaded dual-branch cross-fusion modules. The processing of input features by these modules includes: performing channel-dimensional shuffling on the features of the panchromatic and multispectral remote sensing images, alternating the channel dimensions of the features, which are then used as input features for the two fusion branches. Each of the two fusion branches includes a dual-dimensional RWKV module and a dot product module, and both branches share an addition and convolution module. The dual-dimensional RWKV module extracts features from the mixed feature maps of the corresponding branches and outputs them to the dot product and addition modules of the corresponding fusion branch. The addition module adds the features of the two fusion branches and then performs convolution processing through the convolution module. The dot product module performs a dot product operation on the features of the fusion branch and the convolutionally processed features to obtain the output features of the fusion branch. Finally, the output features of the two fusion branches are added together and then convolved by a convolution module to extract the fused features output by the dual-branch cross-fusion module.

6. The remote sensing image fusion method based on the dual-dimensional RWKV mechanism according to claim 1, characterized in that, The reconstruction stage includes: passing the fused features sequentially through a two-level dual-dimensional RWKV module, two 3×3 convolutional layers, and a residual module; and then using the residual module to add the output features of the last 3×3 convolutional layer to the upsampled multispectral remote sensing image to obtain the fused multispectral remote sensing image.

7. The remote sensing image fusion method based on the dual-dimensional RWKV mechanism according to claim 1, characterized in that, The loss function used during training of the deep fusion network RWPNet is expressed as follows: ； in, For loss function, To deeply fuse the network parameters of the RWPNet network, This represents the number of samples in the training batch. For the deep fusion network RWPNet, RWPNet, a deep fusion network based on network parameters right The generated fused multispectral remote sensing image; For the first Multispectral remote sensing images of a sample; For the first Panchromatic remote sensing image corresponding to each sample; For the first Reference multispectral remote sensing image of each sample, It is an L1 norm.

8. A remote sensing image fusion system based on a two-dimensional RWKV mechanism, comprising a microprocessor and a memory interconnected, characterized in that, The microprocessor is programmed or configured to execute the remote sensing image fusion method based on the two-dimensional RWKV mechanism as described in any one of claims 1 to 7.

9. A computer-readable storage medium storing a computer program or instructions, characterized in that, The computer program or instructions are programmed or configured to execute the remote sensing image fusion method based on the two-dimensional RWKV mechanism as described in any one of claims 1 to 7 via a processor.

10. A computer program product, comprising a computer program or instructions, characterized in that, The computer program or instructions are programmed or configured to execute the remote sensing image fusion method based on the two-dimensional RWKV mechanism as described in any one of claims 1 to 7 via a processor.

Images