Image optimization method and device, electronic equipment, storage medium and product
By introducing an attention extraction module with local enhancement windows and an image optimization model with deep convolutional layers, the problem of poor edge accuracy in remote sensing image denoising is solved, achieving higher image detail extraction accuracy and edge information preservation.
Patent Information
- Application Number
- CN202510974479.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-15
- Publication Date
- 2025-10-31
AI Technical Summary
Existing technologies struggle to effectively guarantee the accuracy of image edge regions during image denoising, leading to a decline in image quality. This is especially true in remote sensing image processing, where traditional methods are prone to detail distortion and blurred edge information.
An image optimization model is constructed by employing an attention extraction module with local enhancement windows and a self-attention mechanism with non-overlapping windows, combined with deep convolutional layers in a feedforward enhancement network. This model optimizes image quality through local feature extraction and contextual information fusion.
It improves the accuracy of image detail feature extraction, ensures sufficient extraction of edge information, reduces model computational complexity, and enhances the accuracy of image edge regions.
Smart Images
Figure CN120876288A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of image processing technology, specifically to an image optimization method, apparatus, electronic device, storage medium, and product. Background Technology
[0002] Remote sensing images are image data of various land features acquired through various remote sensing technologies, and are widely used in fields such as natural resource management, environmental monitoring, and urban planning. High-quality remote sensing images can yield more realistic and accurate analysis results. However, due to hardware limitations, weather conditions, and the influence of random signals during image transmission at various stages, the final remote sensing images are often affected by noise, leading to a decline in image quality. Traditional denoising methods mainly rely on various filters or time-frequency conversion reconstruction, but images denoised using these methods are prone to problems such as detail distortion and blurred edge information.
[0003] Therefore, how to ensure the accuracy of image edge regions during image denoising has become an urgent problem to be solved. Summary of the Invention
[0004] In view of this, this application provides an image optimization method, apparatus, electronic device, storage medium and product, the main purpose of which is to solve the problem of poor accuracy of image edge regions during image denoising.
[0005] In a first aspect, this application provides an image optimization method, including:
[0006] Obtain the image to be optimized, which includes a noise-contaminated image;
[0007] The noise-contaminated image is input into a pre-trained image optimization model, which includes an attention extraction module for local enhancement windows, and extracts local features through the self-attention mechanism of non-overlapping windows.
[0008] The image optimization model is used to process the noise-contaminated image and output an optimized image.
[0009] Optionally, the image optimization model further includes a feedforward enhancement network, which introduces deep convolutional layers into a multilayer perceptron to fuse contextual information while extracting local features.
[0010] Optionally, the training steps of the image optimization model include: constructing an initial model, wherein the encoder and decoder of the initial model both employ a local enhancement window attention extraction module; obtaining an image training set, wherein the image training set includes original remote sensing images and noise-contaminated images corresponding to the original remote sensing images; using the noise-contaminated images as input features and the original remote sensing images as target labels to train the initial model; repeating the training until the loss function value of the initial model is less than a preset threshold, thereby obtaining the image optimization model.
[0011] Optionally, the feedforward enhancement network in the initial model includes deep convolutional layers to enhance local context modeling capabilities.
[0012] Optionally, the step of using the noise-contaminated image as input features and the original remote sensing image as target labels to train the initial model includes: inputting the preprocessed noise-contaminated image into the encoding module of the initial model, and sequentially performing k-level encoding and downsampling processing to obtain image feature information; inputting the original remote sensing image into the decoding module of the initial model for inverse processing, and restoring the resolution through feature comparison and upsampling processing, and finally outputting an optimized image.
[0013] Optionally, the image training set includes several image training pairs, and the original remote sensing image and the noise-contaminated image corresponding to the original remote sensing image constitute a set of image training pairs; obtaining the image training pairs includes: adding Gaussian noise and salt-and-pepper noise to the original remote sensing image to obtain the noise-contaminated image corresponding to the original remote sensing image, so as to obtain the image training pairs.
[0014] Secondly, this application provides an image optimization apparatus, comprising:
[0015] The acquisition unit is configured to acquire an image to be optimized, the image to be optimized including a noise-contaminated image;
[0016] The processing unit is configured to input the noise-contaminated image into a pre-trained image optimization model, the image optimization model including a local enhancement window attention extraction module, which extracts local features through a self-attention mechanism of non-overlapping windows;
[0017] The output unit is configured to process the noise-contaminated image using the image optimization model and output an optimized image.
[0018] Thirdly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the image optimization method described in the first aspect.
[0019] Fourthly, this application provides an electronic device including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the image optimization method described in the first aspect.
[0020] Fifthly, this application provides a computer program product having a computer program stored thereon, which, when executed by a processor, implements the image optimization method described in the first aspect.
[0021] By employing the above technical solutions, this application provides an image optimization method, apparatus, electronic device, storage medium, and product. First, an image to be optimized is acquired, including a noisy image. The noisy image is then input into a pre-trained image optimization model. The image optimization model includes a local enhancement window attention extraction module, which extracts local features through a self-attention mechanism using non-overlapping windows. The image optimization model processes the noisy image and outputs an optimized image. Compared to related technologies, this application improves the accuracy of image detail feature extraction by introducing a local enhancement window attention extraction module and performing self-attention using non-overlapping windows instead of global attention, thereby ensuring sufficient extraction of image edge information.
[0022] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description
[0023] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.
[0024] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0025] Figure 1 A flowchart illustrating an image optimization method provided in an embodiment of this application is shown.
[0026] Figure 2 A flowchart illustrating an image optimization model training method provided in an embodiment of this application is shown.
[0027] Figure 3 This paper shows a schematic diagram of the structure of a ULT network provided in an embodiment of this application;
[0028] Figure 4 This illustration shows a flowchart of an image restoration and denoising process provided in an embodiment of this application;
[0029] Figure 5 This paper shows a schematic diagram of the structure of a Transformer model provided in an embodiment of this application;
[0030] Figure 6 A schematic diagram of a feedforward enhancement network provided in an embodiment of this application is shown;
[0031] Figure 7 A schematic diagram of the structure of an image optimization device provided in an embodiment of this application is shown. Detailed Implementation
[0032] To better understand the above-mentioned objectives, features, and advantages of this application, the solution of this application will be further described below. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.
[0033] As mentioned in the background, remote sensing images are image data of various land features acquired through various remote sensing technologies, and are widely used in fields such as natural resource management, environmental monitoring, and urban planning. Acquiring high-quality remote sensing images allows us to obtain more realistic and accurate analysis results. However, due to hardware limitations of imaging equipment, weather conditions, and the influence of random signals during image transmission at various stages, the final obtained remote sensing images are often affected by noise, leading to a decline in image quality. Among related methods, traditional image denoising methods are mainly based on signal processing and statistical theory, and can be divided into two main categories: spatial domain methods and transform domain methods. Spatial domain methods are mainly based on various filters; transform domain methods are mainly based on Fourier transform and sparsity reconstruction. Although these two types of methods can accomplish most denoising tasks, the results are generally limited, and the denoised images are prone to problems such as detail distortion and blurred edge information. Some methods also use deep neural networks to train models to restore blurred images caused by object motion to clear images. However, with the development of imaging equipment, the resolution of acquired remote sensing images is getting higher and higher. When the Transformer-based model (a deep neural network structure with self-attention mechanism) extracts information using the global attention mechanism, it may ignore the details of edge regions, and the global window will also increase the computational complexity of the model.
[0034] The image optimization method proposed in this embodiment is applied to an image optimization device or electronic device, which can be installed or integrated into some data processing or image processing systems, and can execute any of the image optimization methods mentioned below during operation.
[0035] To address the issue of poor accuracy in maintaining image edge regions during current image denoising processes, this embodiment proposes an image optimization method, such as... Figure 1 As shown, the method includes:
[0036] S101, Obtain the image to be optimized;
[0037] Among these, the images to be optimized include noise-contaminated images, which are images with poor clarity due to noise and require precision optimization. In practical applications, the images to be optimized can be remote sensing images that have acquired image data of various land features using various remote sensing technologies.
[0038] S102, Input the noisy contaminated image into the pre-trained image optimization model;
[0039] The image optimization model includes an attention extraction module with local enhancement windows, which extracts local features through a self-attention mechanism using non-overlapping windows. Unlike a typical Transformer, this pre-trained image optimization model uses local enhancement windows in its attention extraction module. Compared to traditional global attention extraction modules, this module extracts local features through a self-attention mechanism using non-overlapping windows, ensuring sufficient extraction of edge information. In one feasible implementation, the network has a U-shaped shape, and the same Transformer block structure is used in both the encoding and decoding modules.
[0040] S103 processes the noise-contaminated image using an image optimization model and outputs an optimized image.
[0041] In this embodiment, the image to be optimized, including a noisy image, is first acquired. The noisy image is then input into a pre-trained image optimization model. This model includes a local enhancement window attention extraction module, which extracts local features through a self-attention mechanism using non-overlapping windows. The image optimization model processes the noisy image and outputs an optimized image. Compared to related technologies, this embodiment improves the accuracy of image detail feature extraction by introducing a local enhancement window attention extraction module and performing self-attention using non-overlapping windows instead of global attention, thus ensuring sufficient extraction of image edge information.
[0042] Optionally, the image optimization model also includes a feedforward enhancement network, which introduces deep convolutional layers into the multilayer perceptron to fuse contextual information while extracting local features.
[0043] In this embodiment, the Feed-Forward Enhancement Network (FFEN) is a neural network architecture for image processing and computer vision tasks. It improves image quality through neural networks, focusing on local feature extraction and contextual information fusion. The Multilayer Perceptron (MLP) is a feedforward artificial neural network model consisting of at least three layers: an input layer, hidden layers, and an output layer. Each layer is fully connected to the layer below, meaning each node is connected to every node in every other layer. This structure enables MLPs to learn complex patterns in data and can be applied to various fields such as classification and regression. Deep convolutional layers refer to a stack of multiple convolutions, each layer extracting local features (such as edges and textures) through a convolutional kernel. Deep convolutions can capture more complex patterns. Traditional MLPs are stacks of fully connected layers, lacking locality. By introducing deep convolutional layers, MLPs are modified to handle the spatial structure of images. Convolutional operations are naturally suitable for processing local neighborhood information in images and can be used for local feature extraction. By stacking multiple convolutional layers, the network can gradually expand its receptive field, ensuring that local features are extracted while also taking into account the context.
[0044] Optionally, the training steps of the image optimization model include: constructing an initial model, wherein the encoder and decoder of the initial model both adopt the attention extraction module of the local enhancement window; obtaining an image training set, which includes the original remote sensing image and the noise-contaminated image corresponding to the original remote sensing image; using the noise-contaminated image as input features and the original remote sensing image as target label to train the initial model; repeating the training until the loss function value of the initial model is less than a preset threshold to obtain the image optimization model.
[0045] This embodiment describes the training steps of the image optimization model. First, an initial model is constructed, and an initial network structure is designed, specifically adopting a U-shaped encoder-decoder architecture. Both the encoding and decoding parts use attention extraction modules with local enhancement windows. Next, an image training set is prepared, including the original remote sensing image and a corresponding noisy contaminated image. It should be noted that the noisy contaminated image referred to here is the one corresponding to the original remote sensing image, used for training. Specifically, the corresponding noisy contaminated image can be obtained by adding noise to the original remote sensing image. For example, a random original remote sensing image can be taken, and noise can be added to it to obtain the corresponding noisy contaminated image, thus giving the two images the characteristic of "same image content but different noise content," which can then be used to train the model, enabling the model to acquire the ability to denoise and optimize images. In another feasible implementation, a high-resolution image can be obtained from an image database, and noise can be artificially added to the high-resolution image to form a pair of images with "same image content but different noise content." The applied noise can be selected according to the training requirements. The aforementioned noisy image in the context of the image to be optimized refers to an image that inherently contains noise and has an insufficient signal-to-noise ratio. In other words, during the application phase after model training, an image that needs optimization due to noise is input into the model for optimization. While both are noisy images, their meanings are different. One involves noise inherent in the image to be optimized during the application phase, while the other involves noise artificially added during model training.
[0046] Furthermore, the model is trained using the noisy image corresponding to the original remote sensing image as input features and the original remote sensing image as the target label. That is, the noisy image is considered a low signal-to-noise ratio (SNR) image, and the original remote sensing image is considered a high SNR image. In each training round, the loss function L1 is calculated for both the generated image and the high SNR image, serving as a reference standard for the model's training level. This process continues iteratively until L1 is less than a preset threshold, completing the training process. A local enhancement window attention extraction module is used, specifically extracting feature information locally within non-overlapping windows, ensuring sufficient extraction of edge information. Specific details are as follows:
[0047] 1. Construct the initial model
[0048] The encoder is responsible for extracting features from noisy images, while the decoder reconstructs the optimized image.
[0049] Local enhancement windows refer to block processing (window attention in Transformers) and local receptive design, used to focus on small-region features in an image. Compared to traditional global attention extraction modules, local enhancement windows extract feature information locally through non-overlapping windows during feature extraction, ensuring sufficient extraction of edge information while significantly reducing the computational complexity of the model. For example, features can be dynamically weighted using self-attention or cross-attention to enhance the representation of important regions (such as edges and textures).
[0050] 2. Obtain the image training set
[0051] Input data: Noise-contaminated image corresponding to the original remote sensing image. For example, low signal-to-noise ratio data can be prepared by adding Gaussian noise and salt-and-pepper noise to the original remote sensing image. For instance, Gaussian noise with a variance of 0.05 and salt-and-pepper noise with a variance of 0.01 can be manually added to the original remote sensing image, and image cropping can be performed to obtain a noise-contaminated image.
[0052] Target label: The corresponding original remote sensing image.
[0053] 3. Supervised training
[0054] (1) Input-output pair: Noisy image → Encoder → Decoder → Output optimized image.
[0055] (2) The loss function uses the L1 loss value (i.e., mean squared error MSE) to measure the difference between the output and the sharp image.
[0056] (3) Optimization objective: Minimize the loss function and update the model parameters through backpropagation.
[0057] 4. Termination Conditions
[0058] (1) Iteration stop: Training stops when the loss value on the validation set is lower than the preset threshold (e.g., 0.01) for several consecutive rounds.
[0059] (2) Model saving: Select the model version with the lowest loss as the final image optimization model.
[0060] Optionally, the feedforward enhancement network in the initial model includes deep convolutional layers to enhance local context modeling capabilities.
[0061] In this embodiment, the feedforward enhancement network improves feature quality gradually through nonlinear transformations in the image optimization task. Specifically, the feedforward enhancement network is embedded as an auxiliary module in the main network to refine feature representations. A deep convolutional layer refers to a stack of multiple convolutional layers. Specifically, deep convolutional layers extract local features by capturing local patterns (such as edges and textures) of the image through a sliding window of the convolutional kernel; by stacking multiple convolutional layers, deeper features gradually fuse with a wider range of contextual information.
[0062] Optionally, the noise-contaminated image is used as the input feature and the original remote sensing image is used as the target label to train the initial model. This includes: inputting the preprocessed noise-contaminated image into the encoding module of the initial model, and obtaining image feature information by sequentially performing k-level encoding and downsampling processing; inputting the original remote sensing image into the decoding module of the initial model for inverse processing, and restoring the resolution by feature comparison and upsampling processing, and finally outputting an optimized image.
[0063] In this embodiment, the training process of the image optimization model is divided into three key stages: 1. Input preprocessing and encoding: The noisy image is processed by the encoding module to extract multi-scale features; 2. Label guidance and decoding: The original remote sensing image is used as a label to guide the decoding module to restore the resolution; 3. Feature comparison and optimization: Through encoding-decoding feature alignment and upsampling, the optimized image is output. Specifically, the details are as follows:
[0064] 1. Input preprocessing and encoding
[0065] Input data: Noise-contaminated images corresponding to the original remote sensing images.
[0066] Preprocessing includes normalization (scaling pixel values to [0,1] or [-1,1]) and channel alignment (such as band matching for multispectral images).
[0067] Encoding module:
[0068] (1) k-level encoding: Through multi-layer convolution and attention mechanism, features are extracted step by step from low level to high level. Where k is a positive integer.
[0069] (2) Downsampling: After each encoding stage, the spatial resolution is reduced by strided convolution, while the number of channels is increased (feature channel expansion). Downsampling can compress redundant information, retain key features, and reduce computation.
[0070] 2. Tag guidance and decoding
[0071] Target label: The corresponding original remote sensing image.
[0072] Decoding module:
[0073] (1) Inverse processing: Symmetrical to the encoding process, the resolution is gradually restored through upsampling and feature fusion.
[0074] (2) Upsampling: Transposed convolution is used to expand the low-resolution feature map to its original size. Spatial details (such as edges and textures) are gradually reconstructed.
[0075] (3) Feature comparison: During the decoding process, intermediate features (such as skip connections) of the encoding module are concatenated or added to the decoding features to retain more low-level details. Feature comparison can avoid blurring caused by direct upsampling and combines local information from the encoder.
[0076] 3. Supervised training and loss function
[0077] Supervised learning: The model takes a noisy image as input and outputs an optimized image, and calculates the loss by comparing it with the original clear image.
[0078] Loss function: L1 loss, which measures the difference or error between the model's prediction and the target label. For example, it is calculated as follows: For each corresponding pixel in the high signal-to-noise ratio (SNR) image output by the model (predicted value) and the true high SNR image (target value), calculate the absolute difference between their values. Then sum these absolute differences for all pixels (or all samples). Finally, divide this sum by the total number of pixels (or the total number of samples) to obtain the average absolute error. The formula is: L1_loss=(1 / N)*Σ|y_pred[i]-y_true[i]|, where y_pred is the model-predicted high SNR image (a pixel value matrix); y_true is the true high SNR image (a pixel value matrix of the same size); and N is the total number of pixels in the image.
[0079] Optimization objective: Minimize the loss function and update model parameters (such as convolutional kernel weights and upsampling parameters) through backpropagation.
[0080] Optionally, the image training set contains several image training pairs, with the original remote sensing image and the noise-contaminated image corresponding to the original remote sensing image constituting a set of image training pairs; obtaining the image training pairs includes: adding Gaussian noise and salt-and-pepper noise to the original remote sensing image to obtain the noise-contaminated image corresponding to the original remote sensing image, so as to obtain the image training pairs.
[0081] In this embodiment, the image training pair consists of a pair of data comprising a clear original image and a corresponding noisy image. In supervised learning, the original image serves as the target label, and the corresponding noisy image serves as the input. The model learns the mapping relationship from noise to clarity. The original remote sensing image is a high-resolution surface image acquired by sensors such as satellites and drones. It typically contains multispectral or hyperspectral information and has high spatial resolution. Although it may be affected by noise (such as sensor noise or atmospheric interference), it only needs to have a signal-to-noise ratio higher than that of the noisy image to enable the model to train its image optimization capabilities. The corresponding noisy image is an image with noise artificially added to the original remote sensing image, used to simulate image degradation in real-world scenarios. Through supervised learning, the model learns the ability to recover the original clear image from the noisy image, enabling optimization of the input image during application.
[0082] Adding Gaussian noise involves adding random values following a Gaussian (normal) distribution to each pixel value of an image to simulate sensor noise or random interference in signal transmission. Adding salt-and-pepper noise involves randomly setting some pixel values of the image to their minimum (salt noise, such as white) or maximum (pepper noise, such as black) values to simulate impulse noise (such as sensor malfunction or data transmission errors) during image acquisition or transmission. The original remote sensing image and its corresponding noise-contaminated image are paired by filename or index and saved as image training pairs.
[0083] Furthermore, the image optimization method used in this embodiment will be described, such as... Figure 2 The diagram shows a flowchart of another image optimization method proposed in this embodiment.
[0084] First, low signal-to-noise ratio (SNR) data is prepared by adding Gaussian noise and salt-and-pepper noise to the original remote sensing image, thus obtaining the noise-contaminated image corresponding to the original remote sensing image. Then, it is used together with the corresponding high SNR image (the original remote sensing image, where high SNR means a higher SNR than its corresponding noise-contaminated image) as the training set.
[0085] After obtaining the data for training, it is fed into the ULT network for training.
[0086] In neural network training, the loss function serves as a core evaluation metric, and its value exhibits an inverse correlation with the model's current performance—a higher loss value indicates poor predictive performance. The training mechanism dynamically adjusts network parameters through backpropagation, ensuring that parameter updates consistently align with the decreasing direction of the loss function, thereby continuously improving model performance. This parameter optimization process proceeds iteratively, with each training round guiding the network towards lower loss states through gradient calculations until convergence is achieved. Ultimately, the maturely trained ULT network architecture, leveraging this optimization mechanism, can be deployed for practical tasks such as image reconstruction, enabling the mapping from degraded inputs to high-quality outputs.
[0087] This embodiment uses a deep convolutional neural network to perform image restoration on a single low-resolution remote sensing image of the sample. Its overall architecture is as follows: Figure 3 As shown, the network has a U-shaped shape, and both the encoding and decoding modules use Transformer blocks with the same structure. According to the architecture diagram of the ULT network in this embodiment, in one feasible approach, the ULT is built using the PyTorch framework, trained on an NVIDIA ARTX 306016G GPU, and the optimizer is Adam with parameters set to 0.5 and 0.999. The learning rate is set to 2e-4, and the learning rate is kept constant for the first 200 epochs, gradually decreasing to 0 in the last 200 epochs, for a total of 400 epochs.
[0088] Finally, the ULT network model obtained through the above training method can restore and denoise low signal-to-noise ratio remote sensing images to obtain corresponding high signal-to-noise ratio images. This allows remote sensing images to be restored to normal or even higher precision images even if they are affected by noise. This saves a lot of time and economic costs for staff in fields such as natural resource management, environmental monitoring, and urban planning, and provides a new method for the further development of the remote sensing field.
[0089] In this embodiment, during the model verification phase, as follows: Figure 4 As shown, the generated high signal-to-noise ratio image needs to be as close as possible to the corresponding original remote sensing image, and it needs to meet the requirements of conventional denoising tasks in terms of evaluation indicators such as peak signal-to-noise ratio and structural similarity.
[0090] like Figure 5 As shown, unlike a typical Transformer, this embodiment uses a local enhancement window in its attention extraction module. Compared to the traditional global attention extraction module, this method extracts feature information locally through non-overlapping windows during feature extraction, ensuring sufficient edge information extraction while significantly reducing the model's computational complexity. Furthermore, this embodiment employs a feedforward enhancement network to obtain contextual relationships, such as... Figure 6As shown, by introducing deep convolutional layers into a regular feedforward network, its ability to utilize local contextual content is enhanced, ensuring that local feature extraction takes into account contextual relationships.
[0091] In this network, both a low signal-to-noise ratio (SNR) image and its corresponding high SNR image are used as input. For the low SNR image, the image size is first adjusted by a preprocessing module before being input into the encoding module for feature extraction and learning. The encoding stage in the ULT network consists of k encoding modules and k downsampling operations. After each encoding module processes the image information, its height and width are halved, and the number of channels is doubled. The resolution is then restored through downsampling, and this process is repeated until the image size becomes H / 2 after passing through the kth encoding module. k W / 2 k 2 k C; After passing through a bottleneck connection with the same structure as the encoding module, the image feature information learned by the encoding module is input into the decoding module; for the input high signal-to-noise ratio image, the feature information output after each round of encoding is directly output to the corresponding subsequent decoding module through the superposition algorithm; the decoding process also includes k decoding modules and k upsampling operations. The working principle of the decoding module is the reverse process of the encoding module. It combines the low signal-to-noise ratio image information learned by decoding with the high signal-to-noise ratio image information of the corresponding layer to generate the required two-dimensional image, then restores its resolution through upsampling, and finally sorts out the image format and size through the output module to generate the corresponding high-resolution image.
[0092] Based on the training method described above, this embodiment uses a low signal-to-noise ratio (SNR) image as input, calls a ULT model with convergent L1 loss to process the input image, and finally outputs a corresponding high SNR image, thereby achieving image restoration. In addition, it includes the preparation of the low SNR image, which includes manually adding Gaussian noise with a variance of 0.05 and salt-and-pepper noise with a variance of 0.01 to the original remote sensing image, as well as image cropping.
[0093] Furthermore, as Figures 1 to 6 The specific implementation of the method shown in this embodiment provides an image optimization device, such as... Figure 7 As shown, the device includes: an acquisition unit 701, a processing unit 702, and an output unit 703.
[0094] The acquisition unit 701 is configured to acquire an image to be optimized, the image to be optimized including a noise-contaminated image;
[0095] The processing unit 702 is configured to input the noise-contaminated image into a pre-trained image optimization model, the image optimization model including a local enhancement window attention extraction module, which extracts local features through a self-attention mechanism of non-overlapping windows;
[0096] The output unit 703 is configured to process the noise-contaminated image using the image optimization model and output an optimized image.
[0097] In specific application scenarios, the processing unit 702 is further configured such that the image optimization model also includes a feedforward enhancement network, which introduces a deep convolutional layer in a multilayer perceptron to fuse contextual information while extracting local features.
[0098] In a specific application scenario, the processing unit 702 is further configured to: construct an initial model, wherein the encoder and decoder of the initial model both employ a local enhancement window attention extraction module; acquire an image training set, which includes original remote sensing images and noise-contaminated images corresponding to the original remote sensing images; use the noise-contaminated images as input features and the original remote sensing images as target labels to train the initial model; repeat the training until the loss function value of the initial model is less than a preset threshold to obtain the image optimization model.
[0099] In specific application scenarios, the processing unit 702 is further configured such that the feedforward enhancement network in the initial model includes deep convolutional layers to enhance local context modeling capabilities.
[0100] In specific application scenarios, the output unit 703 is further configured to input the preprocessed noise-contaminated image into the encoding module of the initial model, and obtain image feature information by sequentially performing k-level encoding and downsampling processing; input the original remote sensing image into the decoding module of the initial model for reverse processing, and restore the resolution by feature comparison and upsampling processing, and finally output an optimized image.
[0101] In specific application scenarios, the processing unit 702 is further configured to add Gaussian noise and salt-and-pepper noise to the original remote sensing image to obtain a noise-contaminated image corresponding to the original remote sensing image, so as to obtain the image training pair.
[0102] It should be noted that other corresponding descriptions of the functional units involved in the image optimization device provided in this embodiment can be found in [reference]. Figures 1 to 6 The corresponding descriptions in [the document] will not be repeated here.
[0103] Based on the above, Figures 1 to 6Accordingly, this embodiment also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described method. Figures 1 to 6 The method shown.
[0104] Based on the above, Figures 1 to 6 Accordingly, this embodiment also provides a computer program product on which a computer program is stored, which, when executed by a processor, implements the above-described method. Figures 1 to 6 The method shown.
[0105] Based on this understanding, the technical solution of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as CD-ROM, USB flash drive, mobile hard drive, etc.) and includes several instructions to cause a computer device (such as personal computer, server, or network device, etc.) to execute the methods of various implementation scenarios of this application.
[0106] Based on the above, Figures 1 to 6 The method shown, and Figure 7 To achieve the above objectives, this application also provides an electronic device, which can be configured on a computer side, etc. This device includes a memory, a processor, and a computer program stored in the memory. The processor executes the computer program to achieve the above-described virtual device embodiments. Figures 1 to 6 The method shown.
[0107] Based on the above, Figures 1 to 6 The method shown, and Figure 7 To achieve the above objectives, the present application also provides a chip in the illustrated virtual device embodiment, including one or more interface circuits and one or more processors; the interface circuits are used to receive signals from the memory of an electronic device and send the signals to the processors, the signals including computer instructions stored in the memory; when the processor executes the computer instructions, it causes the electronic device to perform the above-described... Figures 1 to 6 The method shown.
[0108] Optionally, the aforementioned physical devices may also include a user interface, a network interface, a camera, radio frequency (RF) circuitry, sensors, audio circuitry, a Wi-Fi module, etc. The user interface may include a display screen, input units such as a keyboard, etc., and optional user interfaces may also include USB interfaces, card reader interfaces, etc. The network interface may optionally include standard wired interfaces, wireless interfaces (such as Wi-Fi interfaces), etc.
[0109] Those skilled in the art will understand that the physical device structure provided in this embodiment does not constitute a limitation on the physical device, and may include more or fewer components, or combine certain components, or have different component arrangements.
[0110] The storage medium may also include an operating system and a network communication module. The operating system is a program that manages the hardware and software resources of the aforementioned physical device, supporting the operation of information processing programs and other software and / or programs. The network communication module is used to enable communication between the various components within the storage medium, as well as communication with other hardware and software in the information processing physical device.
[0111] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms, or it can be implemented by hardware. By applying the solution of this embodiment, compared with related technologies, by introducing a local enhancement window attention extraction module, self-attention of non-overlapping windows is performed instead of global attention, thereby improving the extraction accuracy of image detail features and ensuring sufficient extraction of image edge information.
[0112] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the term "comprising" or any other variations thereof is intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0113] The above description is merely a specific embodiment of this application, enabling those skilled in the art to understand or implement this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments described herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.
Claims
1. An image optimization method, characterized in that, include: Obtain the image to be optimized, which includes a noise-contaminated image; The noise-contaminated image is input into a pre-trained image optimization model, which includes an attention extraction module for local enhancement windows, and extracts local features through the self-attention mechanism of non-overlapping windows. The image optimization model is used to process the noise-contaminated image and output an optimized image.
2. The method according to claim 1, characterized in that, The image optimization model also includes a feedforward enhancement network, which introduces deep convolutional layers into the multilayer perceptron to fuse contextual information while extracting local features.
3. The method according to claim 1, characterized in that, The training steps of the image optimization model include: An initial model is constructed, wherein both the encoder and decoder of the initial model adopt the attention extraction module of the local enhancement window; Obtain an image training set, which includes original remote sensing images and noise-contaminated images corresponding to the original remote sensing images; The initial model is trained using the noise-contaminated image as input features and the original remote sensing image as target label. Repeat the training until the loss function value of the initial model is less than a preset threshold to obtain the image optimization model.
4. The method according to claim 3, characterized in that, The feedforward enhancement network in the initial model contains deep convolutional layers to enhance local context modeling capabilities.
5. The method according to claim 3, characterized in that, The step of training the initial model using the noise-contaminated image as input features and the original remote sensing image as target labels includes: The noise-contaminated image is preprocessed and then input into the encoding module of the initial model. The image feature information is obtained by sequentially performing k-level encoding and downsampling. The original remote sensing image is input into the decoding module of the initial model for reverse processing. Through feature comparison and upsampling, the resolution is restored, and the optimized image is finally output.
6. The method according to claim 3, characterized in that, The image training set contains several image training pairs, and the original remote sensing image and the noise-contaminated image corresponding to the original remote sensing image constitute a set of the image training pairs; The acquisition of image training pairs includes: Gaussian noise and salt-and-pepper noise are added to the original remote sensing image to obtain a noise-contaminated image corresponding to the original remote sensing image, thus obtaining the image training pair.
7. An image optimization device, characterized in that, include: The acquisition unit is configured to acquire an image to be optimized, the image to be optimized including a noise-contaminated image; The processing unit is configured to input the noise-contaminated image into a pre-trained image optimization model, the image optimization model including a local enhancement window attention extraction module, which extracts local features through a self-attention mechanism of non-overlapping windows; The output unit is configured to process the noise-contaminated image using the image optimization model and output an optimized image.
8. An electronic device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the method of any one of claims 1 to 6.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 6.
10. A computer program product having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 6.