Super-resolution image reconstruction method based on implicit neural representation

Abstract

The invention provides a super-resolution image reconstruction method based on implicit neural representation. A high-frequency component and a low-frequency component are separated and reconstructed, a continuous up-sampling module is used for focusing on generation of high-frequency details, meanwhile, low-frequency basic information of an image is reserved through interpolation up-sampling, and finally fusion is achieved through component superposition. The design effectively overcomes the defect that in traditional implicit neural representation, a multi-layer perceptron deviates to learn low-frequency content and neglects high-frequency details, and it is ensured that a reconstructed image can keep a clear texture structure and edge integrity under any preset scale. Through an independent processing mechanism of high and low frequency components, the problem of structural distortion under a large scale factor is significantly suppressed, the visual quality of the image is improved, the advantage of self-adaption of a single model to any scale is maintained, and the storage and calculation cost of multi-model training is reduced.

Description

Technical Field

[0001] This application relates to the field of super-resolution image processing, and in particular to a super-resolution image reconstruction method based on implicit neural representation. Background Technology

[0002] Image super-resolution technology is a method that uses software algorithms to reconstruct low-resolution images (LRI) into super-resolution images (SRI). This technology not only increases the number of pixels in an image, but more importantly, it can restore or enhance the image's detail information, making the image clearer and more realistic.

[0003] Implicit neural representation (ILR) is a parameterized continuous field representation method based on a differentiable multilayer perceptron. Its mathematical essence lies in functionally approximating the mapping relationship from a low-dimensional spectroscopic space to a high-dimensional physical property field through a neural network. In image continuous super-resolution tasks, ILR inputs image coordinate information and depth feature information into the multilayer perceptron. The multilayer perceptron learns a mapping function from coordinates to pixel values, thereby achieving image representation and continuous super-resolution. ILR represents the image as an implicit function of coordinates, enabling a single model to achieve arbitrary-scale super-resolution, significantly reducing model storage space and training costs, and improving the flexibility of image generation. Specifically, image continuous super-resolution technology based on ILR constructs a continuous implicit function of the coordinate basis neural field to achieve continuous-scale adaptive super-resolution reconstruction. This method uses LRI and high-resolution image (HRI) coordinates as input, learns the nonlinear mapping from the low-resolution observation space to the high-resolution latent space end-to-end, achieves sub-pixel-level texture synthesis through an implicit differentiation mechanism, and finally outputs a super-resolution image SRI with arbitrary target resolution.

[0004] However, while implicit neural representation-based super-resolution methods address the data volume issue, they still have limitations. Standard MLP models in continuous upsampling modules tend to learn low-frequency content while neglecting high-frequency details, which can easily lead to image structural distortion, especially at large scales. Furthermore, although feature encoders show good performance in feature extraction, they lack a deep understanding of the spatial relationships of features within the image, resulting in low visual quality of the reconstructed image. Therefore, a super-resolution image reconstruction method based on implicit neural representations is needed to improve image reconstruction quality while ensuring reconstruction efficiency. Summary of the Invention

[0005] The purpose of this application is to at least address one of the aforementioned technical deficiencies, particularly the technical deficiency in the prior art where efficiency and quality in super-resolution image reconstruction cannot be simultaneously achieved.

[0006] In a first aspect, this application provides a super-resolution image reconstruction method based on implicit neural representation, the method comprising:

[0007] Acquire low-resolution images;

[0008] The low-resolution image is input into a feature encoder to generate a multi-scale fused feature map.

[0009] Based on the multi-scale fusion feature map, high-frequency components are generated through a continuous upsampling module;

[0010] Interpolation upsampling is performed on the low-resolution image to generate low-frequency components;

[0011] The high-frequency component and the low-frequency component are added element by element to output a target super-resolution image of a preset scale.

[0012] As an optional implementation, the feature encoder includes a preset number of cascaded multi-scale feature fusion modules. The step of inputting the low-resolution image into the feature encoder to generate a multi-scale fused feature map includes:

[0013] The low-resolution image is processed by a first-scale convolutional block to extract features, which are then input into a feature encoder. The multi-scale feature fusion modules fuse the multi-scale information, channel information, and spatial information corresponding to the low-resolution image to generate a multi-scale fused feature map.

[0014] As an optional implementation, each of the multi-scale feature fusion modules includes a first bottleneck attention module, a multi-level feature fusion module, a second-scale convolutional block, a layer-by-layer feature fusion module, and a second bottleneck attention module connected in sequence.

[0015] The first bottleneck attention module is used to perform spatial channel calibration on the input features, the multi-level feature fusion module is used to extract multi-scale features, the second scale convolution block is used to compress channel dimensions, the layer-by-layer feature fusion module is used to integrate detail features and semantic features, and the second bottleneck attention module is used to optimize feature weights.

[0016] In this process, each of the multi-scale feature fusion modules passes features through skip connections, and the multi-scale fused feature map is generated by superimposing the shallow features with the deep features output by each of the multi-scale feature fusion modules after the first-scale convolution block convolution operation.

[0017] And, the skip connection includes:

[0018] The input features and corresponding output features of each multi-scale feature fusion module are added together via a residual path and then input into the next multi-scale feature fusion module for processing.

[0019] As an optional implementation, both the first bottleneck attention module and the second bottleneck attention module include parallel channel branches and spatial branches.

[0020] The channel branch is used to generate channel weights by performing global average pooling on the input features and then passing them through a fully connected layer. The spatial branch is used to generate spatial weights by connecting two levels of wide receptive field convolution operations with preset dilation parameters through dimensionality reduction convolution. The channel weights and the spatial weights are fused and superimposed on the corresponding original input features as the output features of the corresponding modules.

[0021] As an optional implementation, the step of generating high-frequency components through a continuous upsampling module based on the multi-scale fused feature map includes:

[0022] Based on the coordinate information of the target super-resolution image and the multi-scale fusion feature map, the coordinate information is positionally encoded through a continuous upsampling module. The encoded coordinate information is then fused with the multi-scale fusion feature map in a high-dimensional manner to generate high-frequency components.

[0023] As an optional implementation, the specific method by which the continuous upsampling module generates high-frequency components includes:

[0024] The multi-scale fused feature map is convolved to generate a first high-dimensional feature and a second high-dimensional feature;

[0025] By using nearest neighbor interpolation, based on the coordinate information of the target super-resolution image, a first sampling feature and a second sampling feature corresponding to the first high-dimensional feature and the second high-dimensional feature are extracted;

[0026] The second sampling feature is combined with the coordinate information to perform an inner product operation to adjust the feature distribution and obtain the adjusted high-frequency regulation feature.

[0027] The cell features corresponding to the preset scale are processed by a fully connected layer and then concatenated with the high-frequency adjustment features to obtain high-frequency concatenated features;

[0028] The high-frequency splicing feature is multiplied by the first sampling feature to complete feature modulation, thereby obtaining the high-frequency modulation feature;

[0029] The modulation features are input into a multilayer perceptron to generate the high-frequency components.

[0030] Secondly, this application provides a super-resolution image reconstruction device based on implicit neural representation, the device comprising:

[0031] The acquisition module is used to acquire low-resolution images;

[0032] The processing module is used to input the low-resolution image into the feature encoder to generate a multi-scale fusion feature map;

[0033] The processing module is further configured to generate high-frequency components based on the multi-scale fusion feature map through a continuous upsampling module;

[0034] The processing module is also used to perform interpolation upsampling on the low-resolution image to generate low-frequency components;

[0035] The processing module is also used to add the high-frequency component and the low-frequency component element by element to output a target super-resolution image of a preset scale.

[0036] As an optional implementation, the feature encoder includes a preset number of cascaded multi-scale feature fusion modules. The specific method by which the processing module inputs the low-resolution image into the feature encoder to generate a multi-scale fused feature map includes:

[0037] The low-resolution image is processed by a first-scale convolutional block to extract features, which are then input into a feature encoder. The multi-scale feature fusion modules fuse the multi-scale information, channel information, and spatial information corresponding to the low-resolution image to generate a multi-scale fused feature map.

[0038] Thirdly, this application provides a computer device including one or more processors and a memory storing computer-readable instructions that, when executed by the one or more processors, perform the steps of the method described in the first aspect.

[0039] Fourthly, this application provides a storage medium storing computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of the method described in the first aspect.

[0040] As can be seen from the above technical solutions, the embodiments of this application have the following advantages:

[0041] Based on any of the above embodiments, the super-resolution reconstruction method proposed in this application systematically solves the core defects of traditional implicit neural representations through a feature encoder with multi-scale channel spatial triple fusion, a dual-branch bottleneck attention mechanism, and a continuous upsampling module for high- and low-frequency separation reconstruction. The feature encoder integrates spatial channel calibration, multi-scale extraction, channel compression, and cross-layer semantic fusion through a chain operation of cascaded multi-scale fusion modules, generating feature maps with both positional accuracy and semantic integrity, overcoming the visual quality degradation problem caused by insufficient spatial correlation. The dual-branch bottleneck attention module processes channel and spatial weights in parallel, jointly calibrating feature responses and significantly improving the model's perceptual sensitivity to key regions. The continuous upsampling module overcomes the low-frequency bias of multilayer perceptrons through a coordinate-feature high-dimensional fusion mechanism, accurately reconstructing high-frequency details while retaining basic information of low-frequency components through interpolation upsampling, achieving a balance between structural fidelity and detail enhancement through component superposition. The synergistic effect of these technologies enables a single model to adapt to different magnification factor requirements, achieving a dual improvement in visual quality and detail restoration while significantly reducing storage and computational resource consumption, especially effectively suppressing structural distortion in large-scale super-resolution tasks. Attached Figure Description

[0042] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0043] Figure 1 A flowchart illustrating a super-resolution image reconstruction method based on implicit neural representation provided in one embodiment of this application;

[0044] Figure 2 This is a schematic diagram of a feature encoder framework structure provided in one embodiment of this application;

[0045] Figure 3 This is a schematic diagram of a bottleneck attention module structure provided in one embodiment of this application;

[0046] Figure 4 A schematic diagram of a continuous upsampling module structure based on high and low frequency feature enhancement provided in one embodiment of this application;

[0047] Figure 5 This is an internal structural diagram of a computer device provided in an embodiment of this application. Detailed Implementation

[0048] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0049] Image super-resolution technology is a method that uses software algorithms to reconstruct LRI (Limited Image Representation) into SRI (Simplified Image Representation). This technology not only increases the number of pixels in an image, but more importantly, it can restore or enhance the image's detail information, making the image clearer and more realistic.

[0050] Implicit neural representation (ILR) is a parameterized continuous field representation method based on a differentiable multilayer perceptron. Its mathematical essence lies in functional approximation of the mapping relationship from a low-dimensional observation space to a high-dimensional physical property field through a neural network. In image continuous super-resolution tasks, ILR inputs image coordinate information and depth feature information into the multilayer perceptron. The multilayer perceptron learns a mapping function from coordinates to pixel values, thereby achieving image representation and continuous super-resolution. ILR represents the image as an implicit function of coordinates, enabling a single model to achieve arbitrary-scale super-resolution, significantly reducing model storage space and training costs, and improving the flexibility of image generation. Specifically, image continuous super-resolution technology based on ILR constructs a continuous implicit function of the coordinate basis neural field to achieve continuous-scale adaptive super-resolution reconstruction. This method uses LRI and HRI coordinates as input, learns the nonlinear mapping from the low-resolution observation space to the high-resolution latent space end-to-end, achieves sub-pixel-level texture synthesis through an implicit differentiation mechanism, and finally outputs an SRI of arbitrary target resolution.

[0051] Traditional super-resolution methods are mainly based on interpolation, reconstruction, and learning. While computationally fast, the generated images lack detail and have poor visual presentation. Deep learning-based super-resolution methods primarily rely on Convolutional Neural Networks (CNNs), Generative Adversarial Networks (GANs), Transformers, and diffusion models. CNN-based methods have indeed achieved good progress and visual effects, but compared to GANs, they still lack detail and realism. GAN-based methods also suffer from unstable training and high computational complexity. Transformer-based methods offer advantages in multi-task learning and superior performance, but also suffer from high model complexity and large data requirements. Diffusion model-based methods generate SRIs with rich and realistic details, and their models are robust to input noise, but they suffer from high computational cost, difficult training, low sampling efficiency, and slow generation speed.

[0052] While the aforementioned methods achieve significant visual results, they require training super-resolution models at specific scales based on specific magnification factors. Furthermore, during application, each model can only magnify the LRI by a fixed factor. Super-resolution at different magnification factors constitutes different and independent tasks, posing challenges to training costs and storage space. Therefore, some feasible implementations include arbitrary-scale super-resolution, also known as continuous super-resolution, which requires only a single super-resolution model to achieve arbitrary scale magnification of images while maintaining high image clarity.

[0053] In summary, some feasible remote sensing image super-resolution methods can only achieve a fixed magnification, requiring separate model training for each magnification super-resolution task, resulting in high training and storage costs. Furthermore, users need to frequently switch models when facing different magnification requirements, significantly impacting the user experience. Standard MLP models in continuous upsampling modules tend to learn low-frequency content while neglecting high-frequency details, which can easily lead to image structure distortion, especially at large scale factors, in image reconstruction. While feature encoders show good performance in feature extraction, they lack a deep understanding of the spatial relationships of features within the image, resulting in low visual quality of the reconstructed image.

[0054] This application utilizes the concept of implicit neural representation to design a continuous super-resolution method that adapts to various scales and achieves high-quality reconstruction. Specifically, based on the characteristics of remote sensing images and the magnification characteristics of continuous super-resolution, a multi-scale feature fusion attention module is employed to solve the problem of in-depth extraction of various types and scales of features in remote sensing images. Simultaneously, to deeply extract comprehensive information at various scales and match the structure and coordinate information of the continuous upsampling module, the feature encoder introduces key spatial location information into the channel information, forming a powerful feature representation with triple fusion of multi-scale, channel, and spatial information. In the continuous upsampling module, the concept of implicit neural representation is adopted. High- and low-frequency feature enhancement operations and a 256-dimensional multilayer perceptron are used to map the feature map into RGB values, and continuous super-resolution is performed by combining HRI coordinates. Furthermore, Bicubic interpolation is used to supplement the low-frequency components of SRI at different magnification factors, further optimizing the reconstruction effect and improving the smoothness and naturalness of the reconstructed image.

[0055] In summary, the technical concept of this application lies in the fact that the proposed super-resolution reconstruction method systematically solves the core defects of traditional implicit neural representations through a feature encoder that fuses multiple scales, channels, and spaces, a dual-branch bottleneck attention mechanism, and a continuous upsampling module for high- and low-frequency separation reconstruction. The feature encoder integrates spatial channel calibration, multi-scale extraction, channel compression, and cross-layer semantic fusion through a chain-like operation of cascaded multi-scale fusion modules, generating feature maps with both positional accuracy and semantic integrity, overcoming the visual quality degradation problem caused by insufficient spatial correlation. The dual-branch bottleneck attention module processes channel and spatial weights in parallel, jointly calibrating feature responses and significantly improving the model's perceptual sensitivity to key regions. The continuous upsampling module overcomes the low-frequency bias of multilayer perceptrons through a coordinate-feature high-dimensional fusion mechanism, accurately reconstructing high-frequency details while retaining basic information of low-frequency components through interpolation upsampling, achieving a balance between structural fidelity and detail enhancement through component superposition. The synergistic effect of these technologies enables a single model to adapt to different magnification factor requirements, achieving a dual improvement in visual quality and detail reproduction while significantly reducing storage and computing resource consumption, especially in large-scale super-resolution tasks, effectively suppressing structural distortion.

[0056] The methods provided in this application will be described in detail below based on the corresponding implementation methods in some practical application scenarios.

[0057] Please see Figure 1 , Figure 1 This is a flowchart illustrating a super-resolution image reconstruction method based on implicit neural representations according to an embodiment of this application. This application provides a super-resolution image reconstruction method based on implicit neural representations, the method comprising:

[0058] Acquire low-resolution images;

[0059] The low-resolution image is input into a feature encoder to generate a multi-scale fused feature map.

[0060] Based on the multi-scale fusion feature map, high-frequency components are generated through a continuous upsampling module;

[0061] Interpolation upsampling is performed on the low-resolution image to generate low-frequency components;

[0062] The high-frequency component and the low-frequency component are added element by element to output a target super-resolution image of a preset scale.

[0063] Furthermore, the main inventive point of this application at the model level is that it provides a model that integrates two core modules: a feature encoder and a continuous upsampling module.

[0064] The feature encoder, as the core of the model applied in this method, employs an efficient feature extraction architecture. This architecture consists of several multi-scale feature fusion attention modules, effectively capturing and integrating multi-scale information in the image. Furthermore, the encoder of this invention also introduces a spatial attention mechanism to accurately extract key spatial features. Thus, the encoder comprehensively maps the input image to a 2D feature map from three dimensions: multi-scale, channel, and spatial.

[0065] The continuous upsampling module employs the concept of implicit neural representation, leveraging the powerful modeling capabilities of multilayer perceptrons to accurately predict the RGB values ​​of each HRI coordinate based on 2D feature map representations. Simultaneously, it aims to preserve the original low-frequency components while effectively reconstructing high-frequency details. This design allows the continuous super-resolution method to flexibly address the needs of different scales, achieving arbitrary-scale super-resolution of the input LRI, greatly improving the method's practicality and generalization ability.

[0066] This implementation separates and reconstructs high-frequency and low-frequency components. A continuous upsampling module focuses on generating high-frequency details, while interpolation upsampling preserves the low-frequency fundamental information of the image. Finally, component superposition achieves fusion. This design effectively overcomes the shortcomings of traditional implicit neural representations, where multilayer perceptrons tend to learn low-frequency content while neglecting high-frequency details. This ensures that the reconstructed image maintains clear texture structure and edge integrity at any preset scale. Through the independent processing mechanism of high and low-frequency components, structural distortion at large scale factors is significantly suppressed. While improving image visual quality, it maintains the advantage of single-model adaptation to arbitrary scales and reduces the storage and computational costs of multi-model training.

[0067] As an optional implementation, the feature encoder includes a preset number of cascaded multi-scale feature fusion modules. The step of inputting the low-resolution image into the feature encoder to generate a multi-scale fused feature map includes:

[0068] The low-resolution image is processed by a first-scale convolutional block to extract features, which are then input into a feature encoder. The multi-scale feature fusion modules fuse the multi-scale information, channel information, and spatial information corresponding to the low-resolution image to generate a multi-scale fused feature map.

[0069] This implementation utilizes a cascaded multi-scale feature fusion module to progressively fuse multi-scale, channel, and spatial information based on shallow feature extraction. This chain-like structure overcomes the shortcomings of traditional encoders in modeling spatial relationships through repeated multi-dimensional feature interactions, enabling the generated fused feature map to accurately capture semantic relationships and local details at different scales within the image. This triple fusion mechanism provides rich and position-sensitive feature representations for subsequent high-frequency component reconstruction, significantly improving the visual consistency and detail restoration capabilities of the reconstructed image.

[0070] As an optional implementation, each of the multi-scale feature fusion modules includes a first bottleneck attention module, a multi-level feature fusion module, a second-scale convolutional block, a layer-by-layer feature fusion module, and a second bottleneck attention module connected in sequence.

[0071] The first bottleneck attention module is used to perform spatial channel calibration on the input features, the multi-level feature fusion module is used to extract multi-scale features, the second scale convolution block is used to compress channel dimensions, the layer-by-layer feature fusion module is used to integrate detail features and semantic features, and the second bottleneck attention module is used to optimize feature weights.

[0072] In this process, each of the multi-scale feature fusion modules passes features through skip connections, and the multi-scale fused feature map is generated by superimposing the shallow features with the deep features output by each of the multi-scale feature fusion modules after the first-scale convolution block convolution operation.

[0073] And, the skip connection includes:

[0074] The input features and corresponding output features of each multi-scale feature fusion module are added together via a residual path and then input into the next multi-scale feature fusion module for processing.

[0075] This implementation employs a modular, chain-like design, sequentially executing spatial channel calibration, multi-scale feature extraction, channel compression, cross-layer semantic integration, and weight optimization. The first bottleneck attention module suppresses irrelevant feature interference; the multi-level feature fusion module extracts cross-scale correlation information; the layer-by-layer feature fusion module enhances the hierarchical complementarity of details and semantics; and the second bottleneck attention module further optimizes feature distribution. Combined with the residual propagation mechanism of skip connections, this design achieves the progressive enhancement and fusion of shallow details and deep semantic features, significantly improving the feature encoder's ability to represent complex image structures.

[0076] Please see Figure 2 , Figure 2 This is a schematic diagram of a feature encoder framework provided in one embodiment of this application. This application provides a feature encoder based on spatial feature enhancement. On a multi-scale feature fusion framework, spatial attention is introduced to achieve a deeper understanding of the spatial positional relationships of features in the image, and to effectively complement the coordinate information incorporated by subsequent continuous upsampling modules. Specifically, through deep fusion of multi-scale features, channel information, and spatial key information, a multi-scale feature fusion attention module is designed to construct a feature encoder based on spatial feature enhancement (FE-SFE), achieving richer and more accurate feature representation. The implementation framework of this feature encoder is as follows: Figure 2 As shown.

[0077] Where LR is the input feature corresponding to LRI, serving as the input to the feature encoder. Conv 3x3 refers to a convolutional layer with a 3x3 kernel; MLFFAM is the Multi-level Feature Fusion Attention module (MLFFAM); Conv 1x1 is a convolutional layer operating with a 1x1 kernel; LLFFM is the Layer by Layer Feature Fusion module (LLFFM); and FeatureMap is the output feature map of the feature encoder. BAM-1 and BAM-2 are both bottleneck attention modules, containing spatial attention and channel attention. In BAM-2, the channel attention component replaces the original CA module, and its structure is as follows: Figure 2 As shown, MSFFAM is a novel multi-scale feature fusion attention module that incorporates spatial information.

[0078] In summary, the implementation process of the feature encoder based on spatial feature enhancement proposed in this invention is as follows: First, LRI undergoes a Conv 3x3 operation to obtain shallow features; then, the shallow features are processed by 24 multi-scale feature fusion attention modules (MSFFAM) for deep feature extraction. MSFFAM utilizes skip residual connections to sequentially perform BAM-1, MLFFAM, Conv 1x1, LLFFM, and BAM-2 operations, resulting in different depth features for each MSFFAM; finally, the deep features are combined with the shallow features after a Conv 3x3 operation to construct the output feature map.

[0079] As an optional implementation, both the first bottleneck attention module and the second bottleneck attention module include parallel channel branches and spatial branches.

[0080] The channel branch is used to generate channel weights by performing global average pooling on the input features and then passing them through a fully connected layer. The spatial branch is used to generate spatial weights by connecting two levels of wide receptive field convolution operations with preset dilation parameters through dimensionality reduction convolution. The channel weights and the spatial weights are fused and superimposed on the corresponding original input features as the output features of the corresponding modules.

[0081] The bottleneck attention module provided in this application uses BAM (Bottleneck Attention Module) placed at each bottleneck of the neural network. As the number of network layers increases, BAM will gradually focus on higher-level semantic features.

[0082] Please see Figure 3 , Figure 3 This diagram illustrates the bottleneck attention module structure of one embodiment of the present application, demonstrating the implementation principle of BAM. Inputs are the input feature maps, denoted by F; Outputs are the outputs; and BAM attention is the bottleneck attention map, denoted by M(F). In the channel attention branch, Global average pooling is the global average pooling operation; FC is the fully connected layer operation. This represents the number of channels; Channel attention represents the channel attention map. This section describes the representation and parameter information of the final channel attention map obtained from the channel attention branch. In the spatial attention branch, 1x1 Conv is a convolution operation with a 1x1 kernel; 3x3 Conv is a 3x3 dilated convolution operation with a dilation value of d=4 and a shrinkage rate of r=16; ×2 indicates the use of two dilated convolutions; Spatialattention represents the spatial attention map. This represents the spatial attention map obtained from the spatial attention branch, along with its parameter information.

[0083] This implementation employs a dual-branch attention mechanism to optimize channel and spatial features in parallel. The channel branch utilizes global statistical information to generate channel weights, highlighting important feature channels; the spatial branch captures long-range positional dependencies through wide receptive field convolution, generating a spatial attention map. The fusion and superposition of the weights from the two branches achieves joint spatial-channel calibration of the feature map, significantly enhancing the model's ability to perceive key regions such as edges and textures, suppressing background interference, and thus improving the structural integrity of the reconstructed image.

[0084] As an optional implementation, the step of generating high-frequency components through a continuous upsampling module based on the multi-scale fused feature map includes:

[0085] Based on the coordinate information of the target super-resolution image and the multi-scale fusion feature map, the coordinate information is positionally encoded through a continuous upsampling module. The encoded coordinate information is then fused with the multi-scale fusion feature map in a high-dimensional manner to generate high-frequency components.

[0086] This implementation maps the target image coordinates to a high-dimensional space through positional encoding and performs deep fusion with multi-scale fusion features. This high-dimensional interaction mechanism breaks the low-frequency tendency of the standard multilayer perceptron, forcing the model to learn a fine-grained mapping relationship between coordinates and features, thereby effectively recovering sub-pixel-level high-frequency details in coordinate-driven reconstruction. Combined with the local feature adaptation capability of coordinate information, it significantly improves texture clarity and edge sharpness under large-scale factors.

[0087] As an optional implementation, the specific method by which the continuous upsampling module generates high-frequency components includes:

[0088] The multi-scale fused feature map is convolved to generate a first high-dimensional feature and a second high-dimensional feature;

[0089] By using nearest neighbor interpolation, based on the coordinate information of the target super-resolution image, a first sampling feature and a second sampling feature corresponding to the first high-dimensional feature and the second high-dimensional feature are extracted;

[0090] The second sampling feature is combined with the coordinate information to perform an inner product operation to adjust the feature distribution and obtain the adjusted high-frequency regulation feature.

[0091] The cell features corresponding to the preset scale are processed by a fully connected layer and then concatenated with the high-frequency adjustment features to obtain high-frequency concatenated features;

[0092] The high-frequency splicing feature is multiplied by the first sampling feature to complete feature modulation, thereby obtaining the high-frequency modulation feature;

[0093] The modulation features are input into a multilayer perceptron to generate the high-frequency components.

[0094] This implementation refines the generation of high-frequency components through a phased feature interaction process. Inner product operations adjust the feature distribution to adapt to coordinate positions; cell feature concatenation introduces scale adaptability; modulation operations fuse position and feature information; and a multilayer perceptron ultimately completes the high-frequency mapping. This process, through the step-by-step adjustment and fusion of high-dimensional features, significantly enhances the model's ability to capture local texture changes, ensuring that the output components present rich detail information at any scale.

[0095] Please see Figure 4 , Figure 4 This is a schematic diagram of a continuous upsampling module structure based on high- and low-frequency feature enhancement provided in one embodiment of this application. Before inputting into the MLP, this continuous upsampling module first maps the input coordinates to a high-dimensional space through position encoding, and then upscales the low-dimensional feature representation to a high-dimensional space. This allows for deep fusion of the high-dimensional coordinate information and high-dimensional features, enabling the MLP model to capture more high-frequency details. Furthermore, a Bicubic method is used to directly upsample the LRI to high resolution to supplement the low-frequency components. Thus, while effectively reconstructing high-frequency details, the original low-frequency components are preserved, ensuring that the continuous super-resolution model proposed in this invention performs excellently in super-resolution tasks of any scale. The structure of this continuous upsampling module is as follows: Figure 4 As shown.

[0096] Wherein, LR is the input feature corresponding to LRI, and SR is the output feature corresponding to SRI; FE-SFE is the feature encoder proposed in this invention, responsible for multi-scale feature extraction and fusion; Bicubic is the bicubic interpolation upsampling operation; Conv 3x3 is the convolution operation with a kernel size of 3*3; Sampling is the nearest neighbor interpolation feature map upsampling operation; Concat is the feature merging operation; FC represents a fully connected layer; MLP is a 5-layer multilayer perceptron with ReLU activation and 256 hidden dimensions.

[0097] In summary, the implementation process of the continuous upsampling module based on high and low frequency feature enhancement is as follows: First, the LRI features encoded by FE-SFE are received as input, and high-dimensional feature maps HF1_0 and HF2_0 are generated through Conv 3x3 convolution. Sampling operations are then used to sample HF1_0 and HF2_0 to obtain feature values ​​HF1_1 and HF2_1 corresponding to the query coordinates. Second, the inner product of HF2_1 and the HRI position coordinates (coordinate) is used to obtain HF2_2, which adjusts the feature weights and captures more high-frequency details. Next, the cell feature containing scale information (cell) is processed by a fully connected layer (FC) and concatenated with HF2_2 to fuse spatial location and scale information, generating a new feature HF2_3. Then, HF2_3 is multiplied by HF1_1 for feature modulation. The modulated feature is input into a multilayer perceptron (MLP), where the MLP learns the deep LRI features and HRI coordinate information to obtain the SRI. The mapping relationship is used to obtain the output of the continuous upsampling module; finally, the module output is added element-wise to the low-frequency component obtained by the LR through the Bicubic operation to generate the final continuous super-resolution network output feature SR.

[0098] In summary, the main inventive aspects of this application lie in the provided optimized feature encoder, which employs an optimized multi-scale feature fusion attention module to sequentially perform BAM-1, MLFFAM, Conv 1x1, LLFFM, and BAM-2 operations, repeating this process 24 times to obtain the deepest features, thus achieving richer and more accurate feature representations. The introduction of implicit neural representation enables the mapping of feature maps and coordinate information to the RGB values ​​of the reconstructed image. The optimized continuous upsampling module first fuses high-dimensional coordinate information with high-dimensional features, allowing the MLP model to capture more high-frequency details. Secondly, the Bicubic method is used to supplement low-frequency components, achieving high- and low-frequency feature enhancement and achieving higher-quality super-resolution at any scale.

[0099] The continuous super-resolution method for remote sensing images provided in this application exhibits superior reconstruction performance and visual effects compared to current mainstream continuous super-resolution methods. The proposed feature encoder incorporates scale attention, satisfying the requirement for rich scale information in arbitrary-scale super-resolution, and introduces spatial attention, fusing spatial details based on coordinate information. Both of these elements significantly enhance the reconstruction results. The proposed continuous upsampling module effectively reconstructs high-frequency details while preserving the original low-frequency components, ensuring superior performance compared to existing mainstream methods when handling arbitrary-scale super-resolution tasks.

[0100] This application also provides a super-resolution image reconstruction device based on implicit neural representation, the device comprising:

[0101] The acquisition module is used to acquire low-resolution images;

[0102] The processing module is used to input the low-resolution image into the feature encoder to generate a multi-scale fusion feature map;

[0103] The processing module is further configured to generate high-frequency components based on the multi-scale fusion feature map through a continuous upsampling module;

[0104] The processing module is also used to perform interpolation upsampling on the low-resolution image to generate low-frequency components;

[0105] The processing module is also used to add the high-frequency component and the low-frequency component element by element to output a target super-resolution image of a preset scale.

[0106] This implementation separates and reconstructs high-frequency and low-frequency components. A continuous upsampling module focuses on generating high-frequency details, while interpolation upsampling preserves the low-frequency fundamental information of the image. Finally, component superposition achieves fusion. This design effectively overcomes the shortcomings of traditional implicit neural representations, where multilayer perceptrons tend to learn low-frequency content while neglecting high-frequency details. This ensures that the reconstructed image maintains clear texture structure and edge integrity at any preset scale. Through the independent processing mechanism of high and low-frequency components, structural distortion at large scale factors is significantly suppressed. While improving image visual quality, it maintains the advantage of single-model adaptation to arbitrary scales and reduces the storage and computational costs of multi-model training.

[0107] As an optional implementation, the feature encoder includes a preset number of cascaded multi-scale feature fusion modules. The specific method by which the processing module inputs the low-resolution image into the feature encoder to generate a multi-scale fused feature map includes:

[0108] The low-resolution image is processed by a first-scale convolutional block to extract features, which are then input into a feature encoder. The multi-scale feature fusion modules fuse the multi-scale information, channel information, and spatial information corresponding to the low-resolution image to generate a multi-scale fused feature map.

[0109] This implementation utilizes a cascaded multi-scale feature fusion module to progressively fuse multi-scale, channel, and spatial information based on shallow feature extraction. This chain-like structure overcomes the shortcomings of traditional encoders in modeling spatial relationships through repeated multi-dimensional feature interactions, enabling the generated fused feature map to accurately capture semantic relationships and local details at different scales within the image. This triple fusion mechanism provides rich and position-sensitive feature representations for subsequent high-frequency component reconstruction, significantly improving the visual consistency and detail restoration capabilities of the reconstructed image.

[0110] As an optional implementation, each of the multi-scale feature fusion modules includes a first bottleneck attention module, a multi-level feature fusion module, a second-scale convolutional block, a layer-by-layer feature fusion module, and a second bottleneck attention module connected in sequence.

[0111] The first bottleneck attention module is used to perform spatial channel calibration on the input features, the multi-level feature fusion module is used to extract multi-scale features, the second scale convolution block is used to compress channel dimensions, the layer-by-layer feature fusion module is used to integrate detail features and semantic features, and the second bottleneck attention module is used to optimize feature weights.

[0112] In this process, each of the multi-scale feature fusion modules passes features through skip connections, and the multi-scale fused feature map is generated by superimposing the shallow features with the deep features output by each of the multi-scale feature fusion modules after the first-scale convolution block convolution operation.

[0113] And, the skip connection includes:

[0114] The input features and corresponding output features of each multi-scale feature fusion module are added together via a residual path and then input into the next multi-scale feature fusion module for processing.

[0115] This implementation employs a modular, chain-like design, sequentially executing spatial channel calibration, multi-scale feature extraction, channel compression, cross-layer semantic integration, and weight optimization. The first bottleneck attention module suppresses irrelevant feature interference; the multi-level feature fusion module extracts cross-scale correlation information; the layer-by-layer feature fusion module enhances the hierarchical complementarity of details and semantics; and the second bottleneck attention module further optimizes feature distribution. Combined with the residual propagation mechanism of skip connections, this design achieves the progressive enhancement and fusion of shallow details and deep semantic features, significantly improving the feature encoder's ability to represent complex image structures.

[0116] As an optional implementation, both the first bottleneck attention module and the second bottleneck attention module include parallel channel branches and spatial branches.

[0117] The channel branch is used to generate channel weights by performing global average pooling on the input features and then passing them through a fully connected layer. The spatial branch is used to generate spatial weights by connecting two levels of wide receptive field convolution operations with preset dilation parameters through dimensionality reduction convolution. The channel weights and the spatial weights are fused and superimposed on the corresponding original input features as the output features of the corresponding modules.

[0118] This implementation employs a dual-branch attention mechanism to optimize channel and spatial features in parallel. The channel branch utilizes global statistical information to generate channel weights, highlighting important feature channels; the spatial branch captures long-range positional dependencies through wide receptive field convolution, generating a spatial attention map. The fusion and superposition of the weights from the two branches achieves joint spatial-channel calibration of the feature map, significantly enhancing the model's ability to perceive key regions such as edges and textures, suppressing background interference, and thus improving the structural integrity of the reconstructed image.

[0119] As an optional implementation, the specific method by which the processing module generates high-frequency components based on the multi-scale fused feature map through a continuous upsampling module includes:

[0120] Based on the coordinate information of the target super-resolution image and the multi-scale fusion feature map, the coordinate information is positionally encoded through a continuous upsampling module. The encoded coordinate information is then fused with the multi-scale fusion feature map in a high-dimensional manner to generate high-frequency components.

[0121] This implementation maps the target image coordinates to a high-dimensional space through positional encoding and performs deep fusion with multi-scale fusion features. This high-dimensional interaction mechanism breaks the low-frequency tendency of the standard multilayer perceptron, forcing the model to learn a fine-grained mapping relationship between coordinates and features, thereby effectively recovering sub-pixel-level high-frequency details in coordinate-driven reconstruction. Combined with the local feature adaptation capability of coordinate information, it significantly improves texture clarity and edge sharpness under large-scale factors.

[0122] As an optional implementation, the specific method by which the processing module triggers the continuous upsampling module to generate high-frequency components includes:

[0123] The multi-scale fused feature map is convolved to generate a first high-dimensional feature and a second high-dimensional feature;

[0124] By using nearest neighbor interpolation, based on the coordinate information of the target super-resolution image, a first sampling feature and a second sampling feature corresponding to the first high-dimensional feature and the second high-dimensional feature are extracted;

[0125] The second sampling feature is combined with the coordinate information to perform an inner product operation to adjust the feature distribution and obtain the adjusted high-frequency regulation feature.

[0126] The cell features corresponding to the preset scale are processed by a fully connected layer and then concatenated with the high-frequency adjustment features to obtain high-frequency concatenated features;

[0127] The high-frequency splicing feature is multiplied by the first sampling feature to complete feature modulation, thereby obtaining the high-frequency modulation feature;

[0128] The modulation features are input into a multilayer perceptron to generate the high-frequency components.

[0129] This implementation refines the generation of high-frequency components through a phased feature interaction process. Inner product operations adjust the feature distribution to adapt to coordinate positions; cell feature concatenation introduces scale adaptability; modulation operations fuse position and feature information; and a multilayer perceptron ultimately completes the high-frequency mapping. This process, through the step-by-step adjustment and fusion of high-dimensional features, significantly enhances the model's ability to capture local texture changes, ensuring that the output components present rich detail information at any scale.

[0130] It should be noted that the division of the various modules in the above device is merely a logical functional division. In actual implementation, they can be fully or partially integrated into a single physical entity, or they can be physically separated. Furthermore, these modules can be implemented entirely in software via processing element calls; they can be fully implemented in hardware; or some modules can be implemented by processing element calls to software, while others are implemented in hardware. For example, a processing module can be a separate processing element, or it can be integrated into a chip within the device. Alternatively, it can be stored as program code in the device's memory, and its functions can be called and executed by a processing element. The implementation of other modules is similar. Moreover, these modules can be fully or partially integrated together, or they can be implemented independently. The processing element here can be an integrated circuit with signal processing capabilities. During implementation, each step of the above method or each of the above modules can be completed through integrated logic circuits in the hardware of the processor element or through software instructions.

[0131] Indicatively, such as Figure 5 As shown, Figure 5 This is a schematic diagram of the internal structure of a computer device 300 provided in an embodiment of this application. The computer device 300 can be provided as a server. (Refer to...) Figure 5 The computer device 300 includes a processing component 302, which further includes one or more processors, and memory resources represented by memory 301 for storing instructions, such as application programs, that can be executed by the processing component 302. The application programs stored in memory 301 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 302 is configured to execute instructions to perform the methods of any of the embodiments described above.

[0132] The computer device 300 may also include a power supply component 303 configured to perform power management of the computer device 300, a wired or wireless network interface 304 configured to connect the computer device 300 to a network, and an input / output (I / O) interface 305. The computer device 300 may operate on an operating system stored in memory 301, such as Windows Server™, Mac OS X™, Unix™, Linux™, Free BSD™, or similar.

[0133] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0134] This application provides a storage medium storing computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the method provided in any embodiment.

[0135] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0136] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can be referred to each other.

[0137] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A super-resolution image reconstruction method based on implicit neural representation, characterized in that, The method includes: Acquire low-resolution images; The low-resolution image is input into a feature encoder to generate a multi-scale fused feature map. Based on the multi-scale fusion feature map, high-frequency components are generated through a continuous upsampling module; Interpolation upsampling is performed on the low-resolution image to generate low-frequency components; The high-frequency component and the low-frequency component are added element by element to output a target super-resolution image of a preset scale.

2. The method according to claim 1, characterized in that, The feature encoder includes a preset number of cascaded multi-scale feature fusion modules. The step of inputting the low-resolution image into the feature encoder to generate a multi-scale fused feature map includes: The low-resolution image is processed by a first-scale convolutional block to extract features, which are then input into a feature encoder. The multi-scale feature fusion modules fuse the multi-scale information, channel information, and spatial information corresponding to the low-resolution image to generate a multi-scale fused feature map.

3. The method according to claim 2, characterized in that, Each of the multi-scale feature fusion modules includes a first bottleneck attention module, a multi-level feature fusion module, a second-scale convolutional block, a layer-by-layer feature fusion module, and a second bottleneck attention module connected in sequence. The first bottleneck attention module is used to perform spatial channel calibration on the input features, the multi-level feature fusion module is used to extract multi-scale features, the second scale convolution block is used to compress the channel dimension, the layer-by-layer feature fusion module is used to integrate detail features and semantic features, and the second bottleneck attention module is used to optimize feature weights. In this process, each of the multi-scale feature fusion modules passes features through skip connections, and the multi-scale fused feature map is generated by superimposing the shallow features with the deep features output by each of the multi-scale feature fusion modules after the first-scale convolution block convolution operation. And, the skip connection includes: The input features and corresponding output features of each multi-scale feature fusion module are added together via a residual path and then input into the next multi-scale feature fusion module for processing.

4. The method according to claim 2, characterized in that, Both the first bottleneck attention module and the second bottleneck attention module include parallel channel branches and spatial branches; The channel branch is used to generate channel weights by performing global average pooling on the input features and then passing them through a fully connected layer. The spatial branch is used to generate spatial weights by connecting two levels of wide receptive field convolution operations with preset dilation parameters through dimensionality reduction convolution. The channel weights and the spatial weights are fused and superimposed on the corresponding original input features as the output features of the corresponding modules.

5. The method according to claim 1, characterized in that, The step of generating high-frequency components based on the multi-scale fused feature map through a continuous upsampling module includes: Based on the coordinate information of the target super-resolution image and the multi-scale fusion feature map, the coordinate information is positionally encoded through a continuous upsampling module. The encoded coordinate information is then fused with the multi-scale fusion feature map in a high-dimensional manner to generate high-frequency components.

6. The method according to claim 5, characterized in that, The specific methods by which the continuous upsampling module generates high-frequency components include: The multi-scale fused feature map is convolved to generate a first high-dimensional feature and a second high-dimensional feature; By using nearest neighbor interpolation, based on the coordinate information of the target super-resolution image, a first sampling feature and a second sampling feature corresponding to the first high-dimensional feature and the second high-dimensional feature are extracted; The second sampling feature is combined with the coordinate information to perform an inner product operation to adjust the feature distribution and obtain the adjusted high-frequency regulation feature. The cell features corresponding to the preset scale are processed by a fully connected layer and then concatenated with the high-frequency adjustment features to obtain high-frequency concatenated features; The high-frequency splicing feature is multiplied by the first sampling feature to complete feature modulation, thereby obtaining the high-frequency modulation feature; The modulation features are input into a multilayer perceptron to generate the high-frequency components.

7. A super-resolution image reconstruction device based on implicit neural representation, characterized in that, The device includes: The acquisition module is used to acquire low-resolution images; The processing module is used to input the low-resolution image into the feature encoder to generate a multi-scale fusion feature map; The processing module is further configured to generate high-frequency components based on the multi-scale fusion feature map through a continuous upsampling module; The processing module is also used to perform interpolation upsampling on the low-resolution image to generate low-frequency components; The processing module is also used to add the high-frequency component and the low-frequency component element by element to output a target super-resolution image of a preset scale.

8. The apparatus according to claim 7, characterized in that, The feature encoder includes a preset number of cascaded multi-scale feature fusion modules. The processing module inputs the low-resolution image into the feature encoder to generate a multi-scale fused feature map in the following specific ways: The low-resolution image is processed by a first-scale convolutional block to extract features, which are then input into a feature encoder. The multi-scale feature fusion modules fuse the multi-scale information, channel information, and spatial information corresponding to the low-resolution image to generate a multi-scale fused feature map.

9. A computer device, characterized in that, The method includes one or more processors and a memory storing computer-readable instructions that, when executed by the one or more processors, perform the steps of the method as described in any one of claims 1-6.

10. A storage medium, characterized in that, The storage medium stores computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of the method as described in any one of claims 1-6.

Images