Real-time arbitrary-scale remote sensing super-resolution method based on Mama and multi-resolution Hash
By combining Mamba and multi-resolution hashing, the problem of poor adaptability of existing models to different scale requirements is solved, and high-quality reconstruction of images at arbitrary super-resolution is achieved, improving the ability to capture global and local features of images.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-03
- Publication Date
- 2026-03-17
AI Technical Summary
Existing image processing models based on the Mamba learning architecture only support fixed super-resolution ratios, resulting in poor adaptability to different scale requirements and limiting their wide applicability.
By combining Mamba and multi-resolution hashing, and introducing a hierarchical interactive information strategy and multi-resolution hashing, we can capture global and local features of images and improve the reconstruction quality of images at any super-resolution.
It significantly improves the reconstruction quality of images at arbitrary super-resolution, enhances the high-frequency details and edge recovery capabilities of images, and also has scale generalization and real-time performance.
Smart Images
Figure CN121685261A_ABST
Abstract
Description
Technical Field
[0001] This application relates to a real-time arbitrary-scale remote sensing super-resolution method based on Mamba and multi-resolution hashing, a real-time arbitrary-scale remote sensing super-resolution device based on Mamba and multi-resolution hashing, an electronic device, and a computer-readable storage medium, belonging to the field of image processing technology. Background Technology
[0002] Currently, remote sensing image processing tasks can be implemented using the Mamba learning architecture. This architecture replaces the traditional self-attention mechanism with state-space equations, achieving long sequence modeling with linear complexity. The aim is to improve the processing efficiency of high-resolution images and the ability to capture global dependencies. However, image processing models trained based on the Mamba learning architecture typically only support fixed super-resolution ratios (e.g., ×2, ×3, ×4), exhibiting poor adaptability to different scale requirements, thus limiting their widespread application. Summary of the Invention
[0003] This application proposes a real-time arbitrary-scale remote sensing super-resolution method based on Mamba and multi-resolution hashing, a real-time remote sensing super-resolution device based on Mamba and multi-resolution hashing, an electronic device, and a computer-readable storage medium.
[0004] The real-time arbitrary-scale remote sensing super-resolution method based on Mamba and multi-resolution hashing in this application is adapted to image processing models under the Mamba architecture. The method specifically includes the following steps: Obtain shallow features of the image to be processed, so as to map the image to be processed into a feature space; Based on the shallow features, the deep features of the image to be processed are obtained; A reconstructed image is determined based on the shallow features and the deep features, wherein the reconstructed image has super-resolution of the image to be processed at any scale.
[0005] In some implementations, obtaining deep features of the image to be processed based on the shallow features, in order to perform high-order feature extraction and edge information enhancement on the feature space, includes: Based on multiple basic residual modules in the image processing model, new intermediate layer features or deep layer features are determined according to the shallow layer features or intermediate layer features.
[0006] In some implementations, the determination of new intermediate layer features or deep layer features based on the shallow layer features or intermediate layer features by the multiple basic residual modules in the feature extraction layer of the image processing model includes: Based on the shallow features or the intermediate features, the channel dimension is divided into multiple branch features, and the output feature corresponding to each branch feature is obtained; By fusing the output features, new intermediate layer features or deep layer features are determined.
[0007] In some implementations, the step of dividing the shallow features or the intermediate features into multiple branch features along the channel dimension and obtaining the output feature corresponding to each branch feature includes: Based on the shallow layer features or the intermediate layer features, the channel dimension is divided into first branch features, second branch features, third branch features and fourth branch features; The first branch feature itself is taken as the first output feature; Perform wavelet transform on the second branch feature to determine the second output feature; Based on the Mamba submodule included in the basic residual module, differential fusion is performed on the third branch feature and the fourth branch feature to determine the third output feature.
[0008] In some implementations, fusing the output features to determine new intermediate layer features or deep layer features includes: Based on the Mamba sub-module included in the basic residual module, the first output feature and the second output feature are fused to determine the enhanced feature; Based on the Mamba submodule included in the basic residual module, the enhanced features and the third output features are fused to determine the new intermediate layer features or the deep layer features.
[0009] In some implementations, determining the reconstructed image based on the shallow features and the deep features includes: Based on the upsampling unit of the image processing model, multiple pixel unit center coordinates and corresponding target feature maps are obtained according to the joint features formed by the shallow features and the deep features. Based on the target feature map, a feature query is performed on the hash grid to determine multi-resolution features; The reconstructed image is determined based on the multi-resolution features, the center coordinates of the pixel units, and the corresponding target feature map.
[0010] In some implementations, the step of performing a feature query on a hash grid based on the target feature map to determine multi-resolution features includes: Perform normalization mapping on the target feature map to determine the normalized target feature map; Based on the normalized target feature map, and using hash function indexing and bilinear interpolation, the resolution feature vector corresponding to each layer of the hash grid is determined under the multi-layer hash grid. The multi-resolution feature is determined based on multiple resolution feature vectors.
[0011] The real-time arbitrary-scale remote sensing super-resolution device based on Mamba and multi-resolution hashing in the embodiments of this application includes: A shallow feature extraction module is used to obtain shallow features of the image to be processed, so as to map the image to be processed into a feature space; The deep feature extraction module is used to obtain the deep features of the image to be processed based on the shallow features, so as to perform high-order feature extraction and edge information enhancement on the feature space; An image reconstruction module is used to determine a reconstructed image based on the shallow features and the deep features, wherein the reconstructed image has super-resolution of the image to be processed at any scale.
[0012] The electronic device in this application includes a memory and a processor. The memory stores a computer program, which, when executed by the processor, implements the real-time arbitrary-scale remote sensing super-resolution method based on Mamba and multi-resolution hashing in the above embodiments.
[0013] The computer-readable storage medium in the embodiments of this application stores a computer program that, when executed by one or more processors, implements the real-time arbitrary-scale remote sensing super-resolution method based on Mamba and multi-resolution hashing in the above embodiments.
[0014] The beneficial effects of this application are: the real-time remote sensing super-resolution device method based on Mamba and multi-resolution hashing in the embodiments of this application combines differential fusion Mamba and multi-resolution hashing coding. By introducing a hierarchical interactive information strategy and a multi-resolution hashing upsampling mechanism, it can effectively capture the global and local features of remote sensing images, while improving the high-frequency details and edge recovery capabilities of the images, and significantly improving the quality of image reconstruction under any super-resolution. Attached Figure Description
[0015] Figure 1 This is one of the flowcharts of the real-time arbitrary-scale remote sensing super-resolution method based on Mamba and multi-resolution hashing in the embodiments of this application; Figure 2 This is the second flowchart of the real-time arbitrary-scale remote sensing super-resolution method based on Mamba and multi-resolution hashing in the embodiments of this application. Figure 3This is the third flowchart of the real-time arbitrary-scale remote sensing super-resolution method based on Mamba and multi-resolution hashing in the embodiments of this application; Figure 4 This is the fourth flowchart of the real-time arbitrary-scale remote sensing super-resolution method based on Mamba and multi-resolution hashing in the embodiments of this application. Figure 5 This is the fifth flowchart of the real-time arbitrary-scale remote sensing super-resolution method based on Mamba and multi-resolution hashing in the embodiments of this application; Figure 6 This is the sixth flowchart of the real-time arbitrary-scale remote sensing super-resolution method based on Mamba and multi-resolution hashing in the embodiments of this application; Figure 7 This is the seventh flowchart of the real-time arbitrary-scale remote sensing super-resolution method based on Mamba and multi-resolution hashing in the embodiments of this application. Figure 8 This is one of the application scenario diagrams of the real-time arbitrary-scale remote sensing super-resolution method based on Mamba and multi-resolution hashing in the embodiments of this application; Figure 9 This is the second schematic diagram of the application scenario of the real-time arbitrary-scale remote sensing super-resolution method based on Mamba and multi-resolution hashing in the embodiments of this application. Detailed Implementation
[0016] Please see Figure 1 The real-time arbitrary-scale remote sensing super-resolution method based on Mamba and multi-resolution hashing in this application is adapted to image processing models under the Mamba architecture. Specifically, the real-time arbitrary-scale remote sensing super-resolution method based on Mamba and multi-resolution hashing includes: Step 01: Obtain shallow features of the image to be processed to map the image to the feature space.
[0017] Specifically, the execution of the real-time arbitrary-scale remote sensing super-resolution method based on Mamba and multi-resolution hashing in the embodiments of this application is generally implemented through an image processing model under the Mamba architecture. The input of this model is the image to be processed, and the output is a reconstructed image with super-resolution at any scale corresponding to the image to be processed. Generally, the image to be processed can be a remote sensing image. The real-time arbitrary-scale remote sensing super-resolution method based on Mamba and multi-resolution hashing in the embodiments of this application can be used to process the image to be processed to generate a reconstructed image with a higher resolution than the image to be processed.
[0018] After the image to be processed is input into the image processing model, the convolutional layer in the image processing model is first invoked to perform shallow feature extraction on the image to be processed. The purpose of this extraction is to map the image to be processed onto a latent feature space. The specific processing can be represented by the following formula:
[0019]
[0020] in Indicates shallow features. Indicates a convolutional layer. Let H represent the image to be processed, and W represent the height and width of the image to be processed. These shallow features are primarily used to represent the overall image features of the image to be processed, and can serve as the basic data for generating the reconstructed image.
[0021] Please continue reading. Figure 1 In some implementations, the real-time arbitrary-scale remote sensing super-resolution method based on Mamba and multi-resolution hashing further includes: Step 02: Obtain the deep features of the image to be processed based on the shallow features.
[0022] Specifically, based on the above implementation method, by way of example, after obtaining the shallow features of the image to be processed, multiple levels of continuous feature extraction can be performed based on the shallow features to obtain deep features that mainly represent the details and edge information of the image to be processed. These deep features can be used as the basic data for generating the reconstructed image. Using the deep features as the above-mentioned basic data to generate the reconstructed image can ensure that the edges and details of objects in the reconstructed image remain as clear as possible after the resolution is improved.
[0023] For specific instructions on step 02, please refer to [link / reference]. Figure 2 Specifically, it includes the following steps: Step 021: Based on multiple basic residual modules in the image processing model, determine new intermediate layer features or deep layer features according to shallow or intermediate layer features.
[0024] Specifically, based on the above implementation method, and according to the specific method of extracting deep features from shallow features, for example, the image processing model uses multiple basic residual modules stacked together to achieve continuous feature extraction at multiple levels to obtain deep features. Each basic residual module includes multiple convolutional-like lightweight differential fusion (DIMB) feature extraction layers to extract new intermediate layer features based on shallow or intermediate layer features, or to extract deep features based on intermediate layer features. Furthermore, each basic residual module also includes convolutional layers to aggregate local features. The feature extraction process at each level can be represented by the following formula:
[0025] In particular, when k When it is 1:
[0026] in, Indicates the first k A basic residual module, Indicates the first k -1 of the basic residual modules' output characteristics (also the first) k Input characteristics of each basic residual module). Indicates the first k The output characteristics of each basic residual module, when k When not at its maximum, , All are intermediate layer features, when k When it reaches its maximum, the corresponding These are the aforementioned deep features, using express, This indicates shallow features.
[0027] In this way, multi-level feature extraction, which is performed iteratively by multiple basic residual modules, can take shallow features as the overall input and obtain deep features through multiple intermediate features.
[0028] Please refer to further information. Figure 3 The execution process of step 021 specifically includes: Step 0211: Divide the shallow or intermediate features into multiple branch features along the channel dimension and obtain the output features corresponding to each branch feature; Step 0212: Fuse the output features to determine new intermediate or deep features.
[0029] Specifically, regarding the specific process of obtaining deep features in the above embodiments, for example, the single-level feature extraction process performed by each basic residual module is executed in the following manner: First, based on the input features of the current basic residual module (which can be shallow features or intermediate layer features), the input features are preferentially divided into multiple branch features along the channel dimension. Each branch feature is processed using its corresponding transformation method to obtain its corresponding output features. By dividing the input features into multiple branch features and performing different transformation processes on each, the backbone information of the image to be processed can be preserved, while the edges and details of the image can be enhanced. Then, the output features corresponding to each branch feature are further fused using the lightweight differential fusion module (DIMB) in the basic residual module to obtain the overall output features of the current basic residual module. When the overall input features are shallow or intermediate layer features, the overall output features of the basic residual module are new intermediate layer features. When the overall input features are intermediate layer features, the overall output features of the basic residual module can also be the final deep layer features. This allows for further enhancement of the detailed features in the image, ensuring the accuracy of the generated reconstructed image.
[0030] For step 0211, please refer to Figure 4 Specifically, it includes: Step 02111: Based on shallow or intermediate layer features, divide the channel dimension into first branch features, second branch features, third branch features, and fourth branch features; Step 02112: Use the first branch feature itself as the first output feature; Step 02113: Perform wavelet transform on the second branch feature to determine the second output feature; Step 02114: Based on the Mamba submodule included in the basic residual module, perform differential fusion on the third branch feature and the fourth branch feature to determine the third output feature.
[0031] Specifically, based on the above implementation method, the following example illustrates the specific method of dividing branch features and determining the output features corresponding to each branch feature.
[0032] For shallow features Shallow features in the channel dimension Divide into multiple branch features, for example, into first, second, third, and fourth branch features. , , as well as ).
[0033] Among them, the first branch features As an information authentication branch, its first output feature is itself. The purpose of this setting is to preserve the core information of the entire image to be processed.
[0034] Second branch features As a wavelet branch, it is necessary to specifically target the features of the second branch. The wavelet transform process is performed, and the features after the wavelet transform process are used as the corresponding second output features. The main function of the wavelet transform process is to enhance the edges and details in the image to be processed, so as to improve the accuracy of the generated reconstructed image.
[0035] Third branch characteristics and the fourth branch features Then, as a parallel Mamba branch, the Mamba submodule in the lightweight differential fusion module is used to perform differential fusion Mamba on the above two features, obtaining the third output feature that corresponds to both. The specific differential fusion Mamba process can be represented by the following set of formulas:
[0036] in, a , b , c , d For learnable scaling factor, M 1 and M 2 indicates different Mamba submodules. Features of the third branch The corresponding difference features, Features of the fourth branch The corresponding difference features, U For the third output feature, in the third branch feature Features of the fourth branch After differential and scaling combinations, the third output feature is formed by channel concatenation. U .
[0037] For step 0212, please refer to [link / reference]. Figure 5 Specifically, it includes: Step 02121: Based on the Mamba sub-modules included in the basic residual module, fuse the first output feature and the second output feature to determine the enhanced feature; Step 02122: Based on the Mamba sub-module included in the basic residual module, fuse the enhanced features and the third output features to determine new intermediate layer features or deep layer features.
[0038] Specifically, based on the above implementation method, after determining the first output feature, the second output feature and the third output feature, the fusion can be performed based on the above three to finally obtain the overall output feature of the current basic residual module, that is, the new intermediate layer feature or deep layer feature.
[0039] For the specific fusion process, the first output feature and the second output feature are first fused in state space using the Mamba submodule to determine the enhanced features based on the first output feature and the second output feature. The enhanced features can effectively enhance the edges and details in the image while maintaining the overall backbone information of the image to be processed.
[0040] Then, the Mamba submodule is used to perform differential fusion Mamba with the enhanced features obtained by fusion and the third output feature mentioned above. The specific execution method can be found in the formula group corresponding to the differential fusion Mamba process in the above implementation. The final fused feature is the overall output feature of the current basic residual module. Depending on the actual situation, it can be a new intermediate layer feature or the finally extracted deep layer feature. In this way, high-order feature extraction can be performed on the feature space while enhancing the edge information of the image, providing a data foundation for the subsequent generation of reconstructed images.
[0041] Please continue reading. Figure 1 In some implementations, the real-time arbitrary-scale remote sensing super-resolution method based on Mamba and multi-resolution hashing further includes: Step 03: Determine the reconstructed image based on shallow and deep features. The reconstructed image has super-resolution of the image to be processed at any scale.
[0042] Specifically, based on the above implementation method, after extracting the shallow and deep features of the image to be processed, the shallow and deep features can be used as the original data, and a coordinate channel splicing process based on hash feature encoding can be used to decode and reconstruct a super-resolution image corresponding to the image to be processed. The above process mainly relies on the reconstruction function module included in the image processing model, which includes an upsampling unit and a convolutional layer for aggregating features.
[0043] In some implementations, please refer to [link to relevant documentation] for the specific execution of step 03. Figure 6 Specifically, it includes the following steps: Step 031: Based on the upsampling unit of the image processing model, obtain the center coordinates of multiple pixel units and the corresponding target feature map according to the joint features formed by shallow features and deep features; Step 032: Based on the target feature map, perform feature query on the hash grid to determine multi-resolution features; Step 033: Determine the reconstructed image based on the multi-resolution features, pixel unit center coordinates, and the corresponding target feature map.
[0044] In some implementations, please refer to [link to relevant documentation] for the specific execution of step 032. Figure 7 Specifically, it includes the following steps: Step 0321: Perform normalization mapping on the target feature map to determine the normalized target feature map; Step 0322: Based on the normalized target feature map, and using hash function indexing and bilinear interpolation, determine the resolution feature vector corresponding to each layer of the hash grid under the multi-layer hash grid. Step 0323: Determine the multi-resolution features based on the multiple resolution feature vectors.
[0045] Specifically, in the process of reconstructing an image, the predefined parameters of the image processing model can first be used to determine the center coordinates of multiple pixel units in the image to be processed. The aforementioned pixel unit center coordinates serve as a scale factor when generating the reconstructed image, supplementing the relative position information of local pixels in the resulting reconstructed image. Furthermore, based on the upsampling unit included in the reconstruction function module of the image processing model, upsampling interpolation is performed on the joint features formed by the shallow and deep features of the image to be processed, determining the target feature map of the resulting reconstructed image. This target feature map is mainly used to determine the position of each pixel in the reconstructed image to be generated.
[0046] Next, using multi-resolution hash feature encoding, feature query is performed on a multi-layer hash grid at the target scale corresponding to the reconstructed image. The feature vectors corresponding to each pixel on each layer of the hash grid are calculated and merged to form multi-resolution features.
[0047] Finally, using the pixel unit center coordinates mentioned above Target feature map Based on multi-resolution features, a lightweight multilayer perceptron in the image processing model is used for decoding, ultimately generating a reconstructed image with super-resolution. Step 03 in the above implementation is the core of enabling the reconstructed image to have super-resolution at any scale, especially step 032. The core idea is to encode spatial location features at different scales using a multi-resolution hash table, and further decode to output a reconstructed image with continuous scale consistency and super-resolution. The entire process described above can be simply represented as follows:
[0048] in, Indicates the reconstructed image. This indicates the rebuild function module. This refers to the aforementioned joint features, also known as shallow features. with deep features The sum of.
[0049] Regarding the generation of multi-resolution features, in some examples, the target feature map at the target scale is first generated. A normalized mapping is performed, and based on the normalized target feature map obtained after the normalized mapping, feature queries are performed on a multi-layer hash grid. The hash grid satisfies the following conditions:
[0050]
[0051] in, p It is a preset scale growth factor. For the first The resolution of the layered hash grid, L The total number of layers in the hash grid. The minimum resolution among all hash grids is defined by a feature dimension for each layer of the hash grid.
[0052] Next, for the normalized target feature map, under the condition of each of the above hash grid layers, the bucket index of the normalized target feature map is calculated using a hash function, and then bilinear interpolation is performed on the calculated bucket index to obtain the resolution feature vector corresponding to each hash grid layer. Then, the feature vectors corresponding to each hash grid layer are concatenated according to the channel dimension to obtain the above multi-resolution feature, which can be referred to in the following representation:
[0053] in Represents multi-resolution features, This represents the resolution feature vector corresponding to the first layer of the hash grid. This represents the resolution feature vector corresponding to the second-layer hash grid. Indicates the first L The resolution feature vectors corresponding to the layered hash grids have the same dimension as the feature vectors of the corresponding layer of hash grids. This enables joint modeling of multi-layered spatial frequency features, allowing the image processing model to simultaneously depict local details and preserve global image structure.
[0054] In obtaining multi-resolution features In this case, further, the target feature map can be... Pixel unit center coordinates and multi-resolution features The vectors are concatenated along the channel dimension to form the input vector used to generate the reconstructed image. For details, please refer to the following notation:
[0055] Finally, the above input vector The image is input into a lightweight multilayer perceptron for decoding, enabling continuous mapping and outputting a reconstructed image with super-resolution at any scale. The generated reconstructed image satisfies the following conditions:
[0056] Where r is the target scale mentioned above.
[0057] Furthermore, since the magnification factor γ of the image to be processed is continuously variable during the generation of reconstructed images with super-resolution at arbitrary scales, and its value is not necessarily an integer (for example, γ can take the values of 2.5, 3.7, etc.), in order to enable the image processing model to perceive the current target scale, the magnification factor γ can be encoded through a positional encoding function. Embedded into the input vector This allows the image processing model to dynamically adjust the feature mapping relationship during the decoding stage according to different magnification levels, thus ensuring a smooth transition between continuous scales and preventing scale jumps or artifacts. Under this condition, the generated input vector Please refer to the following notation:
[0058] In some implementations, considering that the image processing model still needs further optimization, the optimization of the image processing model is generally aimed at minimizing the loss function, which is expressed as follows:
[0059] in, L 1 represents the loss function. This indicates the reconstructed image, while To process the image Correspondingly, and with a resolution greater than the image to be processed. The real image is used to calculate the pixel differences between the reconstructed image and the corresponding real image. L The minimum value is taken as the optimization target, and the parameters used in the feature aggregation, hash encoding and decoding processes in the above implementation are optimized in reverse, thereby improving the detail recovery capability of the image processing model.
[0060] The effectiveness of the real-time arbitrary-scale remote sensing super-resolution method based on Mamba and multi-resolution hashing in the embodiments of this application will be explained through experimental results: Using the AID public test dataset of natural images as training data, the image processing model used in the above implementation to perform the real-time arbitrary-scale remote sensing super-resolution method based on Mamba and multi-resolution hashing was trained. During training, the input low-resolution image was cropped to a 48×48 image patch. During testing, peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) were introduced as evaluation metrics. The above two metrics were calculated in the Y channel of the super-resolution result converted to YCbCr space.
[0061] Therefore, using various publicly available natural image test datasets (including AID, DOTA, and DIOR) as test data, the image processing model (DM-HashSR) described above is compared with various image super-resolution algorithms (including LIIF, LTR, CiaoSR, CLIT, LMF, and BliSR) in current related technologies within the training scale (specifically, 4x super-resolution), and the comparison results shown in Table 1 are obtained. For the AID image set, the image processing model (DM-HashSR) described above is compared with various image super-resolution algorithms (including LIIF, LTR, CiaoSR, CLIT, LMF, and BliSR) out of the training scale (specifically, 6x, 8x, 12x, 24x, and 30x super-resolution), and the results shown in Table 2 are obtained.
[0062] As shown in Table 1, compared with the image super-resolution methods in the current related technologies, the real-time arbitrary-scale remote sensing super-resolution method based on Mamba and multi-resolution hashing proposed in this application has achieved varying degrees of improvement on a variety of test datasets. The number of parameters has been significantly reduced, the processing time is also very short, and it can maintain a high level of structural similarity while maintaining a high peak signal-to-noise ratio.
[0063] Table 1. Comparison of the image processing model in this application with existing real-time arbitrary-scale remote sensing super-resolution methods based on Mamba and multi-resolution hashing within the training scale.
[0064] As shown in Table 2, compared with the image super-resolution methods in the current related technologies, the real-time arbitrary-scale remote sensing super-resolution method based on Mamba and multi-resolution hashing proposed in this application can keep the two indicators, PSNR and SSIM, at the highest or second highest level. Therefore, it has both scale generalization and real-time performance, which is of great significance for the deployment of real-time arbitrary-scale super-resolution of remote sensing images.
[0065] Table 2. Comparison of the image processing model in this application's implementation with existing real-time arbitrary-scale remote sensing super-resolution methods based on Mamba and multi-resolution hashing outside the training scale.
[0066] Additionally, for example, please refer to Figure 8 as well as Figure 9 , Figure 8 The image to be processed in the above embodiments is... Figure 9 The image shown is a reconstructed image obtained after processing by the real-time arbitrary-scale remote sensing super-resolution method based on Mamba and multi-resolution hashing in the above embodiments. It can be seen that the real-time arbitrary-scale remote sensing super-resolution method based on Mamba and multi-resolution hashing in the above embodiments improves the resolution of the image to be processed and restores the global contour and detailed texture of the image to be processed.
[0067] The real-time arbitrary-scale remote sensing super-resolution device based on Mamba and multi-resolution hashing in the embodiments of this application includes: The shallow feature extraction module is used to obtain shallow features of the image to be processed, so as to map the image to be processed into the feature space. The deep feature extraction module is used to obtain deep features of the image to be processed based on shallow features, so as to perform high-order feature extraction and edge information enhancement in the feature space. The image reconstruction module is used to determine the reconstructed image based on shallow and deep features, wherein the reconstructed image has super-resolution of the image to be processed at any scale.
[0068] The electronic device in this application includes a memory and a processor. The memory stores a computer program. When the computer program is executed by the processor, the real-time arbitrary-scale remote sensing super-resolution method based on Mamba and multi-resolution hashing in the above-described embodiments is implemented.
[0069] The computer-readable storage medium in the embodiments of this application stores a computer program that, when executed by one or more processors, implements the real-time arbitrary-scale remote sensing super-resolution method based on Mamba and multi-resolution hashing in the above embodiments.
[0070] The above description is merely a preferred embodiment of this application and is not intended to limit this application in any way. Although this application has disclosed the preferred embodiment as above, it is not intended to limit this application. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the technical solution of this application. Any simple modifications, equivalent substitutions, and improvements made to the above embodiments without departing from the technical solution of this application, based on the technical essence of this application and within the spirit and principles of this application, shall still fall within the protection scope of the technical solution of this application.
Claims
1. A real-time arbitrary scale remote sensing super-resolution method based on Mamba and multi-resolution hashing, characterized in that, The method is adapted to an image processing model under a Mamba architecture, and the method comprises: obtaining a shallow feature of a to-be-processed image to map the to-be-processed image to a feature space; obtaining a deep feature of the to-be-processed image according to the shallow feature; determining a reconstructed image according to the shallow feature and the deep feature, wherein the reconstructed image has a super-resolution of the to-be-processed image at an arbitrary scale.
2. The method of claim 1, wherein, The method of obtaining a deep feature of the to-be-processed image according to the shallow feature comprises: determining a new intermediate feature or the deep feature according to the shallow feature or the intermediate feature based on a plurality of basic residual modules in the image processing model.
3. The method of claim 2, wherein, The method of determining a new intermediate feature or the deep feature according to the shallow feature or the intermediate feature based on a plurality of basic residual modules in the feature extraction layer of the image processing model comprises: dividing the shallow feature or the intermediate feature into a plurality of branch features in a channel dimension, and obtaining an output feature corresponding to each branch feature; fusing the output features to determine a new intermediate feature or the deep feature.
4. The method of claim 3, wherein, The method of dividing the shallow feature or the intermediate feature into a plurality of branch features in a channel dimension and obtaining an output feature corresponding to each branch feature comprises: dividing the shallow feature or the intermediate feature into a first branch feature, a second branch feature, a third branch feature, and a fourth branch feature in the channel dimension; taking the first branch feature itself as a first output feature; performing wavelet transform on the second branch feature to determine a second output feature; performing difference fusion on the third branch feature and the fourth branch feature based on a Mamba submodule included in the basic residual module to determine a third output feature.
5. The method of claim 4, wherein, The method of fusing the output features to determine a new intermediate feature or the deep feature comprises: fusing the first output feature and the second output feature based on a Mamba submodule included in the basic residual module to determine an enhanced feature; fusing the enhanced feature and the third output feature based on a Mamba submodule included in the basic residual module to determine a new intermediate feature or the deep feature.
6. The method of claim 1, wherein, The method of determining a reconstructed image according to the shallow feature and the deep feature comprises: obtaining a plurality of pixel unit center coordinates and corresponding target features based on a joint feature formed by the shallow feature and the deep feature based on an upsampling unit of the image processing model; performing feature query on a hash grid based on the target feature map to determine a multi-resolution feature; determining the reconstructed image based on the multi-resolution feature, the pixel unit center coordinates, and the corresponding target feature map.
7. The method of claim 6, wherein, The method of performing feature query on a hash grid based on the target feature map to determine a multi-resolution feature comprises: performing normalization mapping on the target feature map to determine a normalized target feature map; According to the normalized target feature map, based on a hash function index and bilinear interpolation, a resolution feature vector corresponding to each layer of the hash grid is determined under the hash grid in multiple layers; According to a plurality of resolution feature vectors, the multi-resolution features are determined.
8. A real-time arbitrary scale remote sensing super-resolution device based on Mamba and multi-resolution hashing, characterized in that, The image processing device comprises: a shallow feature extraction module configured to obtain a shallow feature of a to-be-processed image, so as to map the to-be-processed image to a feature space; a deep feature extraction module configured to obtain a deep feature of the to-be-processed image according to the shallow feature, so as to perform high-order feature extraction and edge information enhancement on the feature space; an image reconstruction module configured to determine a reconstructed image according to the shallow feature and the deep feature, wherein the reconstructed image has super-resolution of the to-be-processed image in any scale.
9. An electronic device, comprising: The electronic device comprises a memory and a processor, and the memory stores a computer program. When the computer program is executed by the processor, the real-time arbitrary scale remote sensing super-resolution method based on Mamba and multi-resolution hash according to any one of claims 1-7 is implemented.
10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program. When the computer program is executed by one or more processors, the real-time arbitrary scale remote sensing super-resolution method based on Mamba and multi-resolution hash according to any one of claims 1-7 is implemented.