Remote sensing image stripe denoising method and system based on generative adversarial network

Abstract

The present invention relates to the field of remote sensing image processing technology, and in particular to a remote sensing image strip denoising method and system based on a generative adversarial network. The present invention proposes the following scheme: collect remote sensing image data; construct an image detection model, extract the spatial distribution characteristics of strip noise through a first processing network to generate an abnormal node labeling map, extract the temporal dynamic characteristics of strip noise through a second processing network to generate a temporal feature map; construct a generative adversarial network, a generator generates a denoised image according to joint optimization of spatial and temporal characteristics, a discriminator verifies the authenticity of the denoised image and feeds back the optimized generator parameters; finally, through an image fusion algorithm, the denoised image is fused with the original noise-free area to generate an optimized remote sensing image. The present invention can efficiently remove wide strips, thin strips and dynamic strip noise, while retaining image details and spectral information, and is suitable for multi-scene remote sensing image processing.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing image processing, and in particular to a remote sensing image strip denoising method and system based on a generative adversarial network. Background Art

[0002] In remote sensing image processing, stripe noise is a common type of noise. This noise usually appears as regular or irregular stripes, which significantly interferes with the quality of the image and subsequent analysis. In the prior art, spatial domain filtering, frequency domain filtering or statistical model-based methods are mainly used to remove stripe noise, such as mean filtering, Gaussian filtering, Fourier transform and matrix decomposition-based methods. These methods have certain effects in removing simple stripe noise, but there are significant shortcomings. How to efficiently remove complex stripe noise while retaining image details and spectral characteristics has become a technical problem that needs to be solved urgently in the field of remote sensing image processing.

[0003] For example, the Chinese patent with the authorization announcement number CN103020913B provides a method for removing stripe noise from remote sensing images based on segmented correction: the image can be divided into uniform and non-uniform areas according to the distribution of different objects, and the area scanned by the detection element can be divided into different grayscale areas according to the different grayscale values; through these two methods, an image (scanned by row or column) can be divided into different intervals by column or row, and these different interval segments are segmented using common spatial domain denoising methods such as moment matching and histogram matching to remove stripe noise, taking into account the categories of different objects and the grayscale value effects caused by external radiation changes. The results obtained take into account both the object type and spectral characteristics, which are closer to real data, and have high computational efficiency and are more robust. This invention can be well applied to the removal of stripe noise in remote sensing images.

[0004] The above patents all have the problems raised by this background technology: the processing effect of irregular strips or dynamic strips is poor, and there is a lack of modeling ability for time evolution laws, resulting in low consistency of multi-temporal images. To solve the above problems, this application designs a remote sensing image strip denoising method and system based on a generative adversarial network. Summary of the invention

[0005] The technical problem to be solved by the present invention is to address the deficiencies of the prior art, and provide a remote sensing image strip denoising method and system based on a generative adversarial network, collect remote sensing image data; construct an image detection model, extract the spatial distribution characteristics of strip noise through a first processing network to generate an abnormal node labeling map, and extract the temporal dynamic characteristics of strip noise through a second processing network to generate a temporal feature map; construct a generative adversarial network, the generator generates a denoised image based on joint optimization of spatial and temporal characteristics, the discriminator verifies the authenticity of the denoised image and feeds back the optimized generator parameters; finally, through an image fusion algorithm, the denoised image is fused with the original noise-free area to generate an optimized remote sensing image. The present invention can efficiently remove wide strips, thin strips and dynamic strip noise, while retaining image details and spectral information, and is suitable for multi-scene remote sensing image processing.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] A remote sensing image strip denoising method based on a generative adversarial network, the method comprising:

[0008] Collecting remote sensing image data, wherein the remote sensing image data is a serialized image;

[0009] Constructing an image detection model, the image detection model comprising a first processing network and a second processing network, the first processing network being used to distinguish strip noise areas from noise-free areas, extracting spatial distribution characteristics of strip noise areas, and generating an abnormal node labeling graph, and the second processing network being used to extract time transformation characteristics of strip noise and generate a time feature map;

[0010] Constructing a generative adversarial network, the network comprising a generator and a discriminator, the generator performs joint optimization according to spatial and temporal characteristics to generate a denoised image, the discriminator verifies the authenticity of the denoised image and updates the generator parameters;

[0011] According to the denoised image and the noise-free area, optimized remote sensing image data is generated through an image fusion algorithm.

[0012] The first processing network comprises:

[0013] The graph structure construction layer is used to treat the pixels in the remote sensing image data as graph nodes, calculate the spectral similarity between the pixels according to the spatial neighborhood, construct the edges of the graph, and generate the initial graph structure;

[0014] The graph feature extraction layer is used to extract local features and global features of each graph node, and aggregate the local features and global features through the attention mechanism to generate multi-dimensional features;

[0015] The abnormal node detection layer is used to perform feature mapping through frequency convolution according to the frequency information of the multi-dimensional features, obtain node characteristics, calculate node anomaly scores according to the node characteristics, mark abnormal nodes according to the node anomaly scores, and perform regional expansion in combination with their neighboring nodes to generate an abnormal node marking graph;

[0016] The regional protection layer is used to segment the remote sensing image data according to the abnormal node marking map, generate a protection mask for the remaining area, and smooth the mask boundary of the protection mask to generate a noise-free area.

[0017] The graph feature extraction layer includes a local feature extraction branch, a global feature extraction branch and a feature aggregation branch;

[0018] The local feature extraction branch is used to calculate the local connection density of the node through the edge of the node according to the initial graph structure, divide the neighborhood set of each node according to the local connection density, perform feature mapping on each node and its neighborhood nodes through spatial convolution, and calculate local features;

[0019] The global feature extraction branch is used to convert the initial graph structure into a graph Laplace matrix, perform eigenvalue decomposition on the graph Laplace matrix, calculate an eigenvalue matrix and an eigenvector matrix, perform global characteristic modeling on the eigenvalue matrix and the eigenvector matrix, and calculate global features;

[0020] The feature aggregation branch is used to perform weighted fusion of local features and global features according to the relative importance of nodes in local and global graph structures, analyze the correlation between node features and the importance of nodes in the graph, and enhance the fused features through an attention mechanism.

[0021] The second processing network comprises:

[0022] The time series capture layer is used to analyze remote sensing image data in different time spans through wavelet decomposition, and decompose the remote sensing image data into short-time scale components and long-time scale components;

[0023] The temporal feature extraction layer is used to perform temporal dependency modeling on the segmented remote sensing image data through a bidirectional long short-term memory network to extract dynamic time features, wherein the temporal dependency modeling includes forward modeling and reverse modeling, wherein the forward modeling is used to capture the changing trend of the strip noise in the time series over time, and the reverse modeling is used to analyze the historical time dependency of the strip noise, predict the characteristics of the strip noise at the current time point, and provide feedback to the normal modeling according to the prediction results;

[0024] The time series graph generation layer is used to analyze the frequency domain of the remote sensing image data according to Fourier transform, calculate the periodic features, combine the periodic features with the dynamic time features, and construct a time feature map.

[0025] The generative adversarial network comprises:

[0026] A generator, configured to receive the abnormal node labeling graph and the time feature map, construct the initial characteristics of the denoised image, separate the spatial distribution characteristics and the time evolution characteristics of the stripe noise according to the initial characteristics, process the spatial distribution characteristics and the time evolution characteristics according to the residual network, and generate the denoised image;

[0027] A discriminator is used to discriminate the authenticity of the denoised image output by the generator and the real bandless image according to a discrimination criterion, and generate a residual noise labeling map according to the discrimination result, wherein the discrimination criterion includes spatial structure integrity, spectral similarity and temporal dynamic consistency;

[0028] The adversarial training layer is used to generate a weighted mask of the residual noise area according to the residual noise labeling map, calculate the frequency domain loss, pixel-level reconstruction loss and weighted loss according to the noise area of ​​the weighted mask, calculate the loss value according to the frequency domain loss, pixel-level reconstruction loss and weighted loss, perform gradient backpropagation according to the loss value, and update the generator parameters.

[0029] The generator comprises:

[0030] An input layer, used to calculate the spatial distribution range of the strip noise according to the abnormal node labeling graph, and calculate the dynamic change information of the strip noise according to the time feature map;

[0031] A spatial characteristic separation layer is used to perform spatial pattern deconstruction on the strip noise area in the abnormal node marking map according to the spatial distribution range, separate the spatial characteristics of the strip noise through the geometric distribution of the strip position and the intensity change, perform parameterized description on the width, spacing and direction of the strip according to the spatial characteristics, and output a spatial characteristic map of the strip noise;

[0032] A time characteristic separation layer is used to calculate the phase change of the strip noise in the time dimension according to the dynamic change information, generate a time correlation matrix, model the time correlation matrix according to the autoregressive model, predict the change trend of the strip noise at a future time point, perform nonlinear fitting according to the actual trend and the change trend, and output a time characteristic diagram of the strip noise;

[0033] A regional reconstruction layer, used for dividing the strip noise region into a plurality of sub-regions according to the spatial characteristic map and the temporal characteristic map, calculating the temporal correlation weight of each sub-region, generating a weight matrix, and assigning a temporal dynamic correction factor to the pixel points in each sub-region according to the weight matrix;

[0034] The corrected sub-region pixel points are input into the residual network, and the low-frequency component and high-frequency component of the stripe noise are calculated through the residual network. In combination with the spatial characteristics of the sub-region, the low-frequency component is compensated and the high-frequency component is suppressed.

[0035] The calculation formula of the loss value is:

[0036] ,

[0037] in, represents the loss value, represents the frequency domain loss, represents the pixel-level reconstruction loss, represents the weighted loss, represents the mean gradient of the residual noise labeling map, represents the temporal dynamics factor of the residual noise signature, , and Represents the loss weight coefficient.

[0038] The method of generating optimized remote sensing image data by using an image fusion algorithm includes:

[0039] According to the noise-free area protection mask, the area without stripe noise in the original image is calibrated;

[0040] The geometric characteristics of the stripe noise removal area in the denoised image and the noise-free area in the original image are matched, and the texture and intensity of the edge area are aligned according to the matching results;

[0041] According to the alignment results, edge smoothing is performed through bilinear interpolation to output optimized remote sensing image data.

[0042] A remote sensing image strip denoising system based on a generative adversarial network, the system comprising a noise detection module, a noise denoising module and an image fusion module;

[0043] The noise detection module is used to construct an image detection model, which includes a first processing network and a second processing network. The first processing network is used to distinguish between strip noise areas and noise-free areas, extract the spatial distribution characteristics of the strip noise areas, and generate an abnormal node labeling map. The second processing network is used to extract the time transformation characteristics of the strip noise and generate a time feature map.

[0044] The noise denoising module is used to construct a generative adversarial network, the network includes a generator and a discriminator, the generator performs joint optimization according to spatial and temporal characteristics to generate a denoised image, and the discriminator verifies the authenticity of the denoised image and updates the generator parameters;

[0045] The image fusion module is used to fuse the denoised image with the original image area without stripe noise, align the texture and intensity of the edge area according to the geometric characteristic matching result, and perform edge smoothing to generate the final optimized remote sensing image data.

[0046] The noise detection module comprises:

[0047] A spatial characteristic analysis unit, used for extracting the spatial distribution characteristics of strip noise in the remote sensing image through the first processing network, generating an abnormal node marking graph, and marking the spatial range and distribution characteristics of the strip noise;

[0048] The time characteristic analysis unit is used to extract the time variation pattern of the strip noise through the second processing network, generate a time characteristic map, and characterize the evolution characteristics of the strip noise in the time dimension.

[0049] The noise denoising module comprises:

[0050] An image denoising unit is used to construct the initial characteristics of the denoised image, separate the spatial distribution characteristics and the time evolution characteristics of the stripe noise according to the initial characteristics, and process the spatial distribution characteristics and the time evolution characteristics according to the residual network to generate a denoised image;

[0051] The parameter updating unit is used to distinguish the authenticity of the denoised image and the real strip-free image output by the generator according to the discrimination standard, generate a residual noise labeling map according to the discrimination result, and update the generator parameters according to the residual noise labeling map.

[0052] Compared with the prior art, the present invention has the following beneficial effects:

[0053] 1. The present invention constructs an image detection model to accurately extract the spatial distribution characteristics and temporal dynamic characteristics of stripe noise, and combines the generative adversarial network to achieve accurate removal of stripe noise and retention of image details;

[0054] 2. The present invention can process wide strip, thin strip and dynamic strip noise at the same time, improves the spectral consistency, spatial detail integrity and temporal dimension consistency of the denoised image, and is suitable for high-precision remote sensing image processing of complex scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0056] Figure 1 It is a flow chart of a remote sensing image strip denoising method based on a generative adversarial network according to Embodiment 1 of the present invention;

[0057] Figure 2 This is a first processing network structure diagram of Embodiment 1 of the present invention;

[0058] Figure 3 This is a structural diagram of the feature extraction layer of Embodiment 1 of the present invention;

[0059] Figure 4 This is a diagram of the second processing network structure of Example 1 of the present invention;

[0060] Figure 5 This is a diagram of the generative adversarial network structure of Example 1 of the present invention;

[0061] Figure 6 This is a module diagram of a remote sensing image strip denoising system based on a generative adversarial network according to Example 2 of the present invention. DETAILED DESCRIPTION

[0062] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0063] Example 1

[0064] See also Figure 1 The present invention provides an embodiment: a remote sensing image strip denoising method based on a generative adversarial network, wherein the specific steps of the method are as follows:

[0065] S1: Collect remote sensing image data;

[0066] S2: Build an image detection model to generate abnormal node labeling graph and temporal feature map;

[0067] In this step, a graph-based analysis method is used. The pixels of the remote sensing image are regarded as nodes in the graph structure. The marker graph is used to locate the strip noise area and highlight its spatial distribution range. The time series analysis technology is used to generate a time feature map to represent the change pattern of the strip noise in the time dimension. The combination of spatial and temporal characteristics can accurately locate and describe the complex distribution of strip noise.

[0068] S3: Build a generative adversarial network to generate denoised images;

[0069] S4: Generate optimized remote sensing image data through image fusion algorithm.

[0070] Specifically, existing technologies mainly rely on filtering methods in the spatial domain or frequency domain to remove stripe noise in remote sensing images, such as reducing the impact of stripe noise through mean filtering, Gaussian filtering or Fourier transform. However, these methods are often only effective for regular stripe noise. For irregular stripe noise or situations where stripe noise is highly coupled with image detail information, it is easy to cause the loss of image details or over-smoothing, and fail to retain the true information of the image well. In addition, for multi-temporal remote sensing images, traditional methods lack the ability to model the temporal dynamic changes of stripe noise and have difficulty in processing dynamic stripe noise. By pre-processing remote sensing images, the distribution and changes of noise are identified, and this is used as input to jointly optimize the generator and discriminator of the generative adversarial network, so as to remove stripe noise in a targeted manner while retaining the spectral, texture and detail information of the image.

[0071] The image detection model includes a first processing network and a second processing network, wherein the first processing network is used to distinguish strip noise areas from noise-free areas, extract spatial distribution characteristics of strip noise areas, and generate an abnormal node labeling graph, and the second processing network is used to extract time transformation characteristics of strip noise and generate a time feature map;

[0072] See also Figure 2 , the first processing network structure diagram of the embodiment of the present invention, the first processing network is intended to achieve accurate detection and segmentation of strip noise areas through steps such as graph structure construction, feature extraction, abnormal node detection and area protection, and provide clear input targets and protection mechanisms for subsequent denoising and fusion processes. The first processing network includes:

[0073] The graph structure construction layer is used to treat the pixels in the remote sensing image data as graph nodes, calculate the spectral similarity between the pixels according to the spatial neighborhood, construct the edges of the graph, and generate the initial graph structure;

[0074] Because stripe noise usually has certain continuity and directional characteristics in spatial distribution, and the graph structure can well describe the local and global relationship between pixels, each pixel in the remote sensing image is regarded as a graph node. Specifically, by calculating the spectral angle between each pixel and its spatial neighboring pixels, the edge weight of the graph is determined, and then the initial graph structure is constructed. The initial graph structure can clearly describe the spectral correlation between different pixels in the remote sensing image, ensuring that the stripe noise area has clear connection characteristics in the graph structure. The remote sensing image is projected from two-dimensional space to high-dimensional space, which provides a richer feature description for subsequent stripe noise detection.

[0075] The graph feature extraction layer is used to extract local features and global features of each graph node through a convolutional network, and aggregate the local features and global features through an attention mechanism to generate multi-dimensional features;

[0076] Specifically, the graph neural network extracts local and global features by aggregating the neighborhood information of each node. Local features are obtained by weighted aggregation of features of neighborhood nodes, representing the direct environmental features of each node; while global features are calculated by multi-hop neighborhood aggregation combined with the global information of the entire graph structure. In order to enhance the effect of graph feature extraction, an attention mechanism is introduced. This mechanism makes the node features of the strip noise area more prominent in the feature space by dynamically adjusting the importance weights of local and global features. The multi-dimensional features finally outputted can accurately describe the importance of each node in the spectral dimension, spatial distribution dimension, and its relative neighborhood.

[0077] The abnormal node detection layer is used to perform feature mapping through frequency convolution according to the frequency information of the multi-dimensional features, obtain node characteristics, calculate node anomaly scores according to the node characteristics, mark abnormal nodes according to the node anomaly scores, and perform regional expansion in combination with their neighboring nodes to generate an abnormal node marking graph;

[0078] Frequency convolution uses the periodic characteristics of strip noise to generate a frequency characteristic vector of the node by weighted mapping the frequency components of the spectral characteristic vector. Subsequently, the abnormal nodes of the strip noise are marked by calculating the node anomaly score. The calculation formula of the anomaly score combines the mean difference of the spectral characteristics and the spatial gradient characteristics and the cosine similarity between the neighborhoods, thereby mathematically realizing a comprehensive evaluation of the directionality and intensity changes of the strip. The higher the anomaly score of the node, the more likely it is to be located in the strip noise area. According to the scoring results, combined with its neighboring nodes, the abnormal node marking map is generated through the region expansion algorithm. This process ensures the integrity and coherence of the strip noise area.

[0079] The calculation formula for the anomaly score is:

[0080] ,

[0081] in, represents the abnormality score of the node, k represents a single frequency channel of the node, K represents the total number of frequency channels of the node, Represents the spectral characteristic vector corresponding to the kth frequency channel node, Represents the kth frequency channel node The corresponding spatial gradient vector, Representation Node The neighborhood set of represents the mean of the spectral characteristic vector corresponding to the neighborhood set of the k-th frequency channel, represents the mean of the spatial gradient vector corresponding to the neighborhood set of the kth frequency channel, represents the spatial distribution difference corresponding to the kth frequency channel node, represents the logarithm with base 2, Different from the node in the neighborhood set , n represents the total number of nodes in the neighborhood set, Representation Node and nodes The cosine similarity of the strip directional characteristics, Representation Node The strip direction angle, Representation Node The strip direction angle;

[0082] A regional protection layer is used to segment the remote sensing image data according to the abnormal node marking map, generate a protection mask for the remaining area, and smooth the mask boundary of the protection mask to generate a noise-free area;

[0083] The principle of mask generation is to extract the area marked as noise-free from the original image, and at the same time smooth the mask boundary to eliminate boundary artifacts caused by segmentation. Gaussian blur technology is often used for smoothing, which optimizes the gradient change of the boundary area to ensure a natural transition between the texture of the noise-free area and the stripe noise area, and to maximize the integrity and texture details of the noise-free area.

[0084] See also Figure 3 , a structural diagram of a graph feature extraction layer in an embodiment of the present invention, wherein the graph feature extraction layer includes a local feature extraction branch, a global feature extraction branch and a feature aggregation branch;

[0085] The local feature extraction branch is used to calculate the local connection density of the node through the edge of the node according to the initial graph structure, divide the neighborhood set of each node according to the local connection density, perform feature mapping on each node and its neighborhood nodes through spatial convolution, and calculate local features;

[0086] Specifically, the local connection density of the node is calculated based on the edge weight of each node in the initial graph structure to measure the closeness between the node and its neighborhood. Subsequently, the nodes are divided into multiple neighborhood sets according to the local connection density, and a mapping method based on spatial convolution is used to perform convolution operations on each node and its neighborhood nodes. This convolution operation transforms the spatial neighborhood characteristics of the node into a high-dimensional feature representation, thereby generating local characteristics. This process emphasizes the precise description of the local structure, so that the boundary and directional characteristics of the stripe noise can be captured at the detail level.

[0087] The global feature extraction branch is used to convert the initial graph structure into a graph Laplacian matrix, which can reflect the global topological relationship of the entire graph structure, perform eigenvalue decomposition on the graph Laplacian matrix, and calculate the eigenvalue matrix and eigenvector matrix. The eigenvalue matrix is ​​used to describe the overall relationship strength between nodes, and the eigenvector matrix is ​​used to represent the importance of nodes in the global graph structure. Global characteristic modeling of the eigenvalue matrix and eigenvector matrix can reveal the overall pattern of strip noise from a global perspective, such as the periodic distribution characteristics of wide strips and the global impact of irregular strips;

[0088] The feature aggregation branch is used to perform weighted fusion of local features and global features according to the relative importance of nodes in local and global graph structures, analyze the correlation between node features and the importance of nodes in the graph, enhance the fused features through the attention mechanism, and dynamically adjust the fused feature representation by learning the weight coefficient of each node, so that the significant area of ​​the strip noise is given a higher weight.

[0089] See also Figure 4 , a structural diagram of a second processing network according to an embodiment of the present invention, wherein the second processing network comprises:

[0090] The time series capture layer is used to analyze remote sensing image data in different time spans through wavelet decomposition, and decompose the remote sensing image data into short-time scale components and long-time scale components;

[0091] The short-time scale component can capture the fast-changing pattern of strip noise, such as the fast-fluctuating thin strip noise; while the long-time scale component extracts the overall trend and stable change pattern of strip noise, which is suitable for the analysis of wide strip or periodic strip noise. The introduction of wavelet decomposition can effectively solve the problem that traditional time series analysis methods cannot handle local rapid changes and global long-term trends at the same time. In addition, during the decomposition process, by performing specific threshold processing on the wavelet coefficients, the interference of high-frequency noise can be further reduced, providing a clearer time signal for subsequent modeling.

[0092] The time series feature extraction layer is used to perform time series dependency modeling on the segmented remote sensing image data through a bidirectional long short-term memory network to extract dynamic time features, wherein the time series dependency modeling includes forward modeling and reverse modeling, wherein the forward modeling is used to capture the changing trend of the strip noise in the time series over time, such as the intensity growth or attenuation of the strip noise, and the reverse modeling is based on the historical time series data, analyzes the source characteristics of the strip noise and its impact on the current time point, predicts the characteristics of the strip noise at the current time point, and provides feedback to the normal modeling according to the prediction results;

[0093] Specifically, the bidirectional long short-term memory network introduces forward and reverse temporal information flows to capture the evolution trend of stripe noise while compensating for the information omission in time series modeling.

[0094] The time series graph generation layer is used to analyze the frequency domain of remote sensing image data according to Fourier transform to identify the periodic characteristics and frequency distribution of strip noise, calculate the periodic characteristics, combine the periodic characteristics with the dynamic time characteristics, and construct a time feature map.

[0095] Specifically, the Fourier transform maps the time series from the time domain to the frequency domain, making the periodic behavior of the strip noise appear as a specific peak in frequency. By analyzing the frequency amplitude distribution in the spectrum diagram, the main frequency components of the strip noise can be located, and further combined with the dynamic time features to generate a time feature map. The dynamic time feature provides the distribution law of the strip noise in the time dimension, while the frequency domain information supplements the periodicity and repetitive characteristics of the strip noise. Combining the two to generate a time feature map can clearly describe the time evolution pattern and frequency characteristics of the strip noise.

[0096] See also Figure 5 , a structure diagram of a generative adversarial network according to an embodiment of the present invention, wherein the generative adversarial network comprises:

[0097] A generator, configured to receive the abnormal node labeling graph and the time feature map, construct the initial characteristics of the denoised image, separate the spatial distribution characteristics and the time evolution characteristics of the strip noise according to the initial characteristics, so as to achieve accurate modeling of the strip noise area, and process the spatial distribution characteristics and the time evolution characteristics according to the residual network to generate the denoised image;

[0098] Specifically, in the extraction of spatial distribution characteristics, the abnormal node labeling map is used to calibrate the spatial range of strip noise, and the width, direction, and density distribution of strip noise are extracted through geometric distribution analysis and regional intensity changes. The spatial characteristics are realized through image segmentation and regional decomposition, which makes the spatial characteristics of the strip noise area more refined, which is conducive to the subsequent local optimization processing. In the extraction of time evolution characteristics, the time feature map is used to capture the dynamic changes of strip noise, and the image is decomposed into short-time scale and long-time scale components through wavelet decomposition. The short-time scale component is used to analyze the rapid change characteristics of strip noise, and the long-time scale component is used to capture the stable mode of strip noise. Subsequently, the temporal characteristics are modeled through the autoregressive integrated moving average model to predict the change trend of strip noise at future time points, thereby achieving precise control of strip noise in the time dimension.

[0099] The residual network effectively avoids the gradient vanishing problem by designing the feature connection structure of the shallow and deep layers, ensuring that the generator can compensate for the low-frequency components of the stripe noise and suppress the high-frequency components. The discriminator is used to distinguish the authenticity of the denoised image output by the generator and the real stripe-free image according to the discrimination criteria, and generate a residual noise labeling map according to the discrimination results, wherein the discrimination criteria include spatial structural integrity, spectral similarity and temporal dynamic consistency;

[0100] The spatial structural integrity is achieved by comparing the edge structures of the denoised image and the real image. The Canny edge detection operator is used to extract the edge characteristics of the image, and the similarity of the two images is calculated through the structural similarity index to ensure that the denoised image does not lose details in the processing of the stripe boundary. The spectral similarity is achieved by comparing the spectral distribution of the denoised image and the real image. The spectral characteristics of the image are quantitatively analyzed through the spectral characteristic histogram matching algorithm to ensure that the denoising process does not damage the spectral information of the image. The temporal dynamic consistency is achieved based on the comparative analysis of the temporal characteristic mapping diagram. The spectral characteristics of the stripe noise in the time series are extracted using Fourier transform, and the difference in the spectrum between the denoised image and the real image is compared to verify the temporal dynamic consistency.

[0101] The result of the judgment is based on the residual noise to determine whether it meets a preset noise threshold. If the noise threshold is not met, a denoised image is output if the residual noise is less than the noise threshold.

[0102] The adversarial training layer is used to generate a weighted mask of the residual noise area according to the residual noise labeling map, calculate the frequency domain loss, pixel-level reconstruction loss and weighted loss according to the noise area of ​​the weighted mask, calculate the loss value according to the frequency domain loss, pixel-level reconstruction loss and weighted loss, perform gradient backpropagation according to the loss value, and update the generator parameters.

[0103] Specifically, the adversarial training layer builds a closed-loop optimization mechanism between the generator and the discriminator, generates a weighted mask of the residual noise area through the residual noise labeling map, and focuses on optimizing the denoising effect of the generator in the strip noise area. The weighted mask is dynamically generated based on the discrimination result of the discriminator, in which the residual noise area is given a higher optimization weight to guide the targeted optimization of the generator. In the design of the loss function, the frequency domain loss, pixel-level reconstruction loss and weighted loss are combined to achieve a comprehensive improvement in the performance of the generator in a multi-objective optimization manner. The frequency domain loss calculates the difference in frequency spectrum between the denoised image and the real image through Fourier transform to ensure that the periodic characteristics of the strip noise are effectively suppressed. The pixel-level reconstruction loss calculates the error between the denoised image and the real image at the pixel level to ensure that the basic pixel characteristics of the generated image are consistent with the real image. The weighted loss performs enhanced optimization on the residual noise area according to the weighted mask to ensure the key removal of the strip noise.

[0104] During the gradient backpropagation process, the generator parameters are updated according to the total loss value calculated by the above loss function. In order to improve the optimization efficiency, the adversarial training layer introduces a dynamic learning rate adjustment mechanism to dynamically adjust the generator's learning rate according to the changing trend of the loss value to ensure the stability and convergence of the training process.

[0105] The generator comprises:

[0106] An input layer, used to calculate the spatial distribution range of the strip noise according to the abnormal node labeling graph, and calculate the dynamic change information of the strip noise according to the time feature map;

[0107] The calculation of the spatial distribution range is based on the characteristics of each node in the abnormal node labeling map, including its spectral information, brightness gradient, and spectral similarity with neighboring nodes. By constructing a neighborhood relationship graph of pixel points, each node is classified using a weighted distance function to calibrate the main distribution area of ​​stripe noise. At the same time, in order to ensure the integrity and continuity of the identified stripe area, a clustering algorithm is introduced to correct the boundaries of the noise area and generate a complete spatial distribution range.

[0108] The temporal feature map shows the intensity change and direction change trend of the strip noise in the time dimension. Combining multiple frames of images in the time series, short-time Fourier transform is used to extract the frequency information of the strip noise and calculate the dynamic change information.

[0109] A spatial characteristic separation layer is used to perform spatial pattern deconstruction on the strip noise area in the abnormal node marking map according to the spatial distribution range, separate the spatial characteristics of the strip noise through the geometric distribution of the strip position and the intensity change, perform parameterized description on the width, spacing and direction of the strip according to the spatial characteristics, and output a spatial characteristic map of the strip noise;

[0110] Specifically, the width, spacing and direction of the stripe area are separated by morphological analysis methods. These parameters are parameterized and modeled through the stripe characteristic description formula. For example, the stripe width and spacing are used as independent variables to construct a distribution model of the stripe pattern and form a spatial characteristic map of the stripe noise. The intensity change of the stripe position is fitted by the spectral gradient and pixel value fluctuation to describe the spatial characteristics of the stripe noise in the image.

[0111] A time characteristic separation layer is used to calculate the phase change of the strip noise in the time dimension according to the dynamic change information, generate a time correlation matrix, model the time correlation matrix according to the autoregressive model, predict the change trend of the strip noise at a future time point, perform nonlinear fitting according to the actual trend and the change trend, and output a time characteristic diagram of the strip noise;

[0112] In the specific calculation, the strip noise characteristics of each frame of the image are used to extract the phase information by time spectrum analysis, and the time dependency is restored by the phase recovery algorithm. Then, a time autoregressive model is constructed to model the time correlation matrix and predict the change trend of the strip noise in the future. On the basis of the prediction, nonlinear fitting is performed according to the actual change trend of the strip noise and the time trend predicted by the model to ensure the accuracy of the time characteristic description and output the time characteristic diagram.

[0113] A regional reconstruction layer, used for dividing the strip noise region into a plurality of sub-regions according to the spatial characteristic map and the temporal characteristic map, and calculating the temporal correlation weight of each sub-region to generate a weight matrix, and assigning a temporal dynamic correction factor to the pixels in each sub-region according to the weight matrix, wherein the calculation of the dynamic correction factor takes into account the strip change amplitude in the temporal characteristic map and the regional position difference in the spatial characteristic map, thereby allocating an adaptive adjustment parameter to each pixel;

[0114] The corrected sub-region pixels are input into the residual network, which extracts the low-frequency and high-frequency components of the stripe noise through multi-layer calculations. The low-frequency component represents the overall distribution trend of the stripe noise and is adjusted through the compensation model; the high-frequency component represents the stripe edge and detail characteristics and is optimized through the suppression mechanism. Finally, the output of the residual network is overall corrected in combination with the spatial characteristics of the sub-region to generate an optimized image without stripe noise.

[0115] The calculation formula of the loss value is:

[0116] ,

[0117] in, represents the loss value, represents the frequency domain loss, represents the pixel-level reconstruction loss, represents the weighted loss, represents the mean gradient of the residual noise labeling map, represents the temporal dynamics factor of the residual noise signature, , and Represents the loss weight coefficient.

[0118] The pixel-level reconstruction loss is achieved by comparing the difference between the denoised image and the true noise-free image pixel by pixel. In the technical implementation, the denoised image and the reference noise-free image are first registered to ensure that the geometric positions of the two images are consistent. Then, the difference in intensity values ​​between the two images is calculated pixel by pixel, with particular attention paid to the pixel deviation in the stripe noise area. Through this pixel-by-pixel comparison, the accuracy of the denoised image in restoring the original characteristics can be quantified.

[0119] The calculation of frequency domain loss is based on the significant characteristics of strip noise in the spectrum. First, the denoised image and the noise-free image are Fourier transformed to convert the image from the spatial domain to the frequency domain. Strip noise usually manifests as an enhancement of frequency components in a specific direction. Therefore, by analyzing the intensity changes in the corresponding frequency direction in the spectrum, it is possible to evaluate whether the denoised image has effectively suppressed the frequency characteristics of strip noise. This step emphasizes the optimization of the energy distribution of the denoised image in the frequency domain, making it closer to the real image without strip noise.

[0120] The weighted loss is calculated based on the residual noise marker map, which is used to focus on optimizing the strip noise areas. In the specific implementation, the generated residual noise marker map marks the areas where strip noise still exists, and assigns higher weight values ​​to the pixels in these areas. When calculating the loss, the weight values ​​of the strip noise areas are included in the difference calculation, ensuring that the loss function can provide a greater penalty for these areas, thereby guiding the generator to further optimize the removal of strip noise.

[0121] The mean gradient reflects whether the details of the denoised image are natural. In the calculation, the edge detection algorithm is used to extract the gradient distribution of the image, focusing on whether the edge characteristics of the stripe noise area are destroyed. By comparing the gradient distribution of the denoised image and the real image, it is evaluated whether the denoised image has removed the stripe noise and retained the edge details of the image.

[0122] The calculation of the temporal dynamic factor is based on the temporal feature map, which is mainly used to measure the consistency of the denoised image in the time series. In the technical implementation, the temporal variation pattern of the strip noise in the denoised image is first extracted by short-time Fourier transform, and then compared with the reference temporal feature map to analyze whether the attenuation trend of the strip noise in the time dimension is consistent with the real noise-free image.

[0123] The method of generating optimized remote sensing image data by using an image fusion algorithm includes:

[0124] According to the noise-free area protection mask, the area without stripe noise in the original image is calibrated;

[0125] The geometric characteristics of the stripe noise removal area in the denoised image and the noise-free area in the original image are matched, and the texture and intensity of the edge area are aligned according to the matching results;

[0126] According to the alignment results, edge smoothing is performed through bilinear interpolation to output optimized remote sensing image data.

[0127] Example 2

[0128] See also Figure 6,The present invention provides an embodiment: a remote sensing image strip denoising system based on a generative adversarial network, the system comprising a noise detection module, a noise denoising module and an image fusion module;

[0129] The noise detection module is used to construct an image detection model, which includes a first processing network and a second processing network. The first processing network is used to distinguish between strip noise areas and noise-free areas, extract the spatial distribution characteristics of the strip noise areas, and generate an abnormal node labeling map. The second processing network is used to extract the time transformation characteristics of the strip noise and generate a time feature map.

[0130] The noise denoising module is used to construct a generative adversarial network, the network includes a generator and a discriminator, the generator performs joint optimization according to spatial and temporal characteristics to generate a denoised image, and the discriminator verifies the authenticity of the denoised image and updates the generator parameters;

[0131] The image fusion module is used to fuse the denoised image with the original image area without stripe noise, align the texture and intensity of the edge area according to the geometric characteristic matching result, and perform edge smoothing to generate the final optimized remote sensing image data.

[0132] The noise detection module comprises:

[0133] A spatial characteristic analysis unit, used for extracting the spatial distribution characteristics of strip noise in the remote sensing image through the first processing network, generating an abnormal node marking graph, and marking the spatial range and distribution characteristics of the strip noise;

[0134] The time characteristic analysis unit is used to extract the time variation pattern of the strip noise through the second processing network, generate a time characteristic map, and characterize the evolution characteristics of the strip noise in the time dimension.

[0135] The noise denoising module comprises:

[0136] An image denoising unit is used to construct the initial characteristics of the denoised image, separate the spatial distribution characteristics and the time evolution characteristics of the stripe noise according to the initial characteristics, and process the spatial distribution characteristics and the time evolution characteristics according to the residual network to generate a denoised image;

[0137] The parameter updating unit is used to distinguish the authenticity of the denoised image and the real strip-free image output by the generator according to the discrimination standard, generate a residual noise labeling map according to the discrimination result, and update the generator parameters according to the residual noise labeling map.

[0138] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A remote sensing image strip denoising method based on a generative adversarial network, characterized in that: The method comprises: Collecting remote sensing image data, wherein the remote sensing image data is a serialized image; Constructing an image detection model, the image detection model includes a first processing network and a second processing network, the first processing network is used to distinguish the strip noise area and the noise-free area of ​​the remote sensing image data, and extract the spatial distribution characteristics of the strip noise area to generate an abnormal node labeling map, and the second processing network is used to extract the time transformation characteristics of the remote sensing image data to generate a time feature map; Constructing a generative adversarial network, the generative adversarial network includes a generator and a discriminator, the generator jointly optimizes the abnormal node labeling map and the time feature map to generate a denoised image, and the discriminator verifies the authenticity of the denoised image and updates the generator parameters; According to the denoised image and the noise-free area, optimized remote sensing image data is generated through an image fusion algorithm.

2. The remote sensing image strip denoising method based on generative adversarial network according to claim 1, characterized in that: The first processing network comprises: The graph structure construction layer is used to treat the pixels in the remote sensing image data as graph nodes, calculate the spectral similarity between the pixels according to the spatial neighborhood, construct the edges of the graph, and generate the initial graph structure; The graph feature extraction layer is used to extract local features and global features of each graph node, and aggregate the local features and global features through the attention mechanism to generate multi-dimensional features; The abnormal node detection layer is used to perform feature mapping through frequency convolution according to the frequency information of the multi-dimensional features, obtain node characteristics, calculate node anomaly scores according to the node characteristics, mark abnormal nodes according to the node anomaly scores, and perform regional expansion in combination with their neighboring nodes to generate an abnormal node marking graph; The regional protection layer is used to segment the remote sensing image data according to the abnormal node marking map, generate a protection mask for the remaining area, and smooth the mask boundary of the protection mask to generate a noise-free area.

3. The remote sensing image strip denoising method based on generative adversarial network according to claim 2 is characterized in that: The graph feature extraction layer includes a local feature extraction branch, a global feature extraction branch and a feature aggregation branch; The local feature extraction branch is used to calculate the local connection density of the node through the edge of the node according to the initial graph structure, divide the neighborhood set of each node according to the local connection density, perform feature mapping on each node and its neighborhood nodes through spatial convolution, and calculate local features; The global feature extraction branch is used to convert the initial graph structure into a graph Laplace matrix, perform eigenvalue decomposition on the graph Laplace matrix, calculate an eigenvalue matrix and an eigenvector matrix, perform global characteristic modeling on the eigenvalue matrix and the eigenvector matrix, and calculate global features; The feature aggregation branch is used to perform weighted fusion of local features and global features according to the relative importance of nodes in local and global graph structures, analyze the correlation between node features and the importance of nodes in the graph, and enhance the fused features through an attention mechanism.

4. The remote sensing image strip denoising method based on generative adversarial network according to claim 1, characterized in that: The second processing network comprises: The time series capture layer is used to analyze remote sensing image data in different time spans through wavelet decomposition, and decompose the remote sensing image data into short-time scale components and long-time scale components; The temporal feature extraction layer is used to perform temporal dependency modeling on the segmented remote sensing image data through a bidirectional long short-term memory network to extract dynamic time features, wherein the temporal dependency modeling includes forward modeling and reverse modeling, wherein the forward modeling is used to capture the changing trend of the strip noise in the time series over time, and the reverse modeling is used to analyze the historical time dependency of the strip noise, predict the characteristics of the strip noise at the current time point, and provide feedback to the normal modeling according to the prediction results; The time series graph generation layer is used to analyze the frequency domain of the remote sensing image data according to Fourier transform, calculate the periodic features, combine the periodic features with the dynamic time features, and construct a time feature map.

5. The remote sensing image strip denoising method based on generative adversarial network according to claim 1, characterized in that: The generative adversarial network comprises: A generator, configured to receive the abnormal node labeling graph and the time feature map, construct the initial characteristics of the denoised image, separate the spatial distribution characteristics and the time evolution characteristics of the stripe noise according to the initial characteristics, process the spatial distribution characteristics and the time evolution characteristics according to the residual network, and generate the denoised image; A discriminator is used to discriminate the authenticity of the denoised image output by the generator and the real bandless image according to a discrimination criterion, and generate a residual noise labeling map according to the discrimination result, wherein the discrimination criterion includes spatial structure integrity, spectral similarity and temporal dynamic consistency; The adversarial training layer is used to generate a weighted mask of the residual noise area according to the residual noise labeling map, calculate the frequency domain loss, pixel-level reconstruction loss and weighted loss according to the noise area of ​​the weighted mask, calculate the loss value according to the frequency domain loss, pixel-level reconstruction loss and weighted loss, perform gradient backpropagation according to the loss value, and update the generator parameters.

6. The remote sensing image strip denoising method based on generative adversarial network according to claim 5, characterized in that: The generator comprises: An input layer, used to calculate the spatial distribution range of the strip noise according to the abnormal node labeling graph, and calculate the dynamic change information of the strip noise according to the time feature map; A spatial characteristic separation layer is used to perform spatial pattern deconstruction on the strip noise area in the abnormal node marking map according to the spatial distribution range, separate the spatial characteristics of the strip noise through the geometric distribution of the strip position and the intensity change, perform parameterized description on the width, spacing and direction of the strip according to the spatial characteristics, and output a spatial characteristic map of the strip noise; A time characteristic separation layer is used to calculate the phase change of the strip noise in the time dimension according to the dynamic change information, generate a time correlation matrix, model the time correlation matrix according to the autoregressive model, predict the change trend of the strip noise at a future time point, perform nonlinear fitting according to the actual trend and the change trend, and output a time characteristic diagram of the strip noise; A regional reconstruction layer, used for dividing the strip noise region into a plurality of sub-regions according to the spatial characteristic map and the temporal characteristic map, calculating the temporal correlation weight of each sub-region, generating a weight matrix, and assigning a temporal dynamic correction factor to the pixel points in each sub-region according to the weight matrix; The corrected sub-region pixel points are input into the residual network, and the low-frequency component and high-frequency component of the stripe noise are calculated through the residual network. In combination with the spatial characteristics of the sub-region, the low-frequency component is compensated and the high-frequency component is suppressed.

7. The remote sensing image strip denoising method based on generative adversarial network according to claim 1, characterized in that: The method of generating optimized remote sensing image data by using an image fusion algorithm includes: According to the noise-free area protection mask, the area without stripe noise in the original image is calibrated; The geometric characteristics of the stripe noise removal area in the denoised image and the noise-free area in the original image are matched, and the texture and intensity of the edge area are aligned according to the matching results; According to the alignment results, edge smoothing is performed through bilinear interpolation to output optimized remote sensing image data.

8. A remote sensing image strip denoising system based on a generative adversarial network, used to implement the remote sensing image strip denoising method based on a generative adversarial network as claimed in any one of claims 1 to 7, characterized in that: The system includes a noise detection module, a noise denoising module and an image fusion module; The noise detection module is used to construct an image detection model, which includes a first processing network and a second processing network. The first processing network is used to distinguish between strip noise areas and noise-free areas of remote sensing image data, and extract the spatial distribution characteristics of the strip noise areas to generate an abnormal node labeling map. The second processing network is used to extract the time transformation characteristics of the remote sensing image data to generate a time feature map. The noise denoising module is used to construct a generative adversarial network, which includes a generator and a discriminator. The generator jointly optimizes the abnormal node labeling map and the time feature map to generate a denoised image, and the discriminator verifies the authenticity of the denoised image and updates the generator parameters. The image fusion module is used to fuse the denoised image with the original image area without stripe noise, align the texture and intensity of the edge area according to the geometric characteristic matching result, and perform edge smoothing to generate the final optimized remote sensing image data.

9. The remote sensing image strip denoising system based on generative adversarial network according to claim 8, characterized in that: The noise detection module comprises: A spatial characteristic analysis unit, used for extracting the spatial distribution characteristics of strip noise in the remote sensing image through the first processing network, generating an abnormal node marking graph, and marking the spatial range and distribution characteristics of the strip noise; The time characteristic analysis unit is used to extract the time variation pattern of the strip noise through the second processing network, generate a time characteristic map, and characterize the evolution characteristics of the strip noise in the time dimension.

10. The remote sensing image strip denoising system based on generative adversarial network according to claim 8, characterized in that: The noise denoising module comprises: An image denoising unit is used to construct the initial characteristics of the denoised image, separate the spatial distribution characteristics and the time evolution characteristics of the stripe noise according to the initial characteristics, and process the spatial distribution characteristics and the time evolution characteristics according to the residual network to generate a denoised image; The parameter updating unit is used to distinguish the authenticity of the denoised image and the real strip-free image output by the generator according to the discrimination standard, generate a residual noise labeling map according to the discrimination result, and update the generator parameters according to the residual noise labeling map.

Images