A method and system for removing strip noise from hyperspectral remote sensing images

Abstract

The present application provides a method and system for removing stripe noise from hyperspectral remote sensing images, the method comprising: preprocessing an original training data set to obtain a training data set; training a semi-supervised decoupling transformation network model based on the training data set to obtain a trained semi-supervised decoupling transformation network model; inputting the hyperspectral remote sensing image to be processed into the trained semi-supervised decoupling transformation network model to obtain a hyperspectral remote sensing image with stripe noise removed. The present application uses image decoupling and noise transformation to encourage the model to learn the true stripe noise distribution. Compared with using a handcrafted stripe noise generation model, directly learning physical degradation from real images can implicitly provide a more accurate approximation of the real data. The present application improves the quality of removing stripe noise and generating images, especially in effectively removing stripe noise from real images.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method and system for removing stripe noise from hyperspectral remote sensing images. Background Art

[0002] Hyperspectral remote sensing imagery, rich in spatial and spectral information, is a valuable resource. However, imaging system instability, influenced by various factors such as stray light interference, slit contamination, and instrument instability, can cause variations in the data obtained from adjacent detectors, even when capturing the same ground material. This inconsistency results in banding noise in the acquired data, significantly reducing data quality and undermining the effectiveness of subsequent missions. Therefore, banding removal plays a crucial role in remote sensing, improving data quality and facilitating the advancement of subsequent implementations.

[0003] To address this interference problem in hyperspectral remote sensing images, various methods have been developed. These methods can be broadly categorized into four groups: statistical matching-based methods, filtering-based techniques, optimization-based methods, and deep learning-based methods. Nevertheless, traditional de-banding methods often face limitations, including poor recovery performance when dealing with complex ground materials, inability to handle images with prominent features in the transform domain, and reliance on rigidity assumptions that may invalidate the effectiveness of de-banding. In recent years, deep learning-based techniques have emerged as a promising solution, leveraging the representation power of convolutional neural networks (CNNs) to achieve improved de-banding performance.

[0004] Among existing methods, deep learning-based de-banding methods have emerged as the best at achieving superior de-banding performance. These models leverage the robust feature extraction capabilities of networks to effectively learn the mapping between input and desired output images, thereby generating high-quality clean images. Currently, most deep learning-based methods focus on enhancing de-banding performance by incorporating various modules to improve the network's learning capabilities. During the training process of these de-banding methods, a large number of simulated samples are typically used. These samples are generated by introducing striping noise to clean the images. Because the simulated images are artificially contaminated with striping noise, the noise distribution in the simulated test data is consistent with the training data, facilitating striping noise removal by deep learning-based methods. The complexity of spectral imaging systems and the numerous interfering factors during the imaging process make it challenging to design an accurate mathematical model for the degradation process of hyperspectral images. Specifically, when testing on real images, ensuring consistent noise distributions between test and training data becomes difficult, which can lead to failure of deep models. Therefore, designing an unsupervised method to allow the model to learn the noise distribution of real data during model training is crucial and an effective means to address these issues. Summary of the Invention

[0005] In order to overcome the above-mentioned problems existing in the prior art, this application provides a method and system for removing stripe noise from hyperspectral remote sensing images, which adopts the following technical solutions:

[0006] In a first aspect, the present application provides a method for removing stripe noise from a hyperspectral remote sensing image, comprising:

[0007] Preprocess the original training data set to obtain the training data set;

[0008] Training the semi-supervised decoupling transformation network model based on the training data set to obtain a trained semi-supervised decoupling transformation network model;

[0009] The hyperspectral remote sensing image to be processed is input into the trained semi-supervised decoupling transform network model to obtain a hyperspectral remote sensing image with stripe noise removed.

[0010] Furthermore, the preprocessing of the original training data set to obtain the training data set includes: cropping the original training data set into images of a preset size and normalizing the grayscale values of the cropped images, wherein the original training data set includes a fitting image and a real image.

[0011] Furthermore, the semi-supervised decoupled transformation network model is trained based on the training data set to obtain a trained semi-supervised decoupled transformation network model, wherein the semi-supervised decoupled transformation network model includes a fitting image branch and a real image branch, the fitting image branch is subject to supervised constraints, and the real image branch is subject to unsupervised constraints.

[0012] Furthermore, the fitting image branch is subject to supervised constraints, and the real image branch is subject to unsupervised constraints, including:

[0013] During training, the fitted image branch is guided by a supervised constraint using the ground truth mapping, while the real image branch is guided by an unsupervised constraint via the discriminative model.

[0014] Furthermore, the semi-supervised decoupled transformation network model includes four generators G1, G2, G3, and G4 and two discriminators D1 and D2.

[0015] Furthermore, the semi-supervised decoupling transformation network model learns the real stripe noise distribution through decoupling and noise transformation, which is specifically manifested as follows:

[0016] Based on the generator G1 and generator G3 in the semi-supervised decoupled transform network model, the clean background components are decoupled respectively, and the fitting image stripe noise component and the real image stripe noise component are further extracted from the fitting image and the real image;

[0017] Generate stripe noise from the fitting domain to the real domain based on generator G2, and generate stripe noise from the real domain to the fitting domain based on generator G4;

[0018] The discriminator D1 is trained to distinguish the fitted image O s and the generated degraded image B r -N s True and false instances, encouraging the generator G2 to reduce the fitting image O s and the generated degraded image B r -N s The discriminator D2 is used to enhance the ability of the generator G4 to transfer the stripe noise from the real domain to the fitting domain.

[0019] Furthermore, a combination of unsupervised adversarial loss, self-consistency loss and supervised mean square error loss is adopted in the intermediate stage of the semi-supervised decoupled transformation network model;

[0020] The unsupervised adversarial loss includes: for the input fitting image O s and the real image O r , respectively for the degraded image B generated in the first stage s -N r and B r -N s The adversarial constraint is applied to achieve the transformation generation of the striped noise image in an unsupervised manner. The unsupervised adversarial loss can be expressed as:

[0021]

[0022]

[0023] in, and denote the adversarial loss constraints of the fitted image and real image branches, E Os Indicates the mathematical expectation of the results of the fitting image branch, E Or Indicates the mathematical expectation of the result of the real image branch; D1 and D2 represent two discriminators, D1(O s ) represents the fitting image O of the discriminator D1 to the input s To judge, D2(O r ) represents the discriminator D2's response to the input real image O r Make a judgment; G r→s (O r ) and G s→r (O s ) represent the first stage generated image B r -N s and Bs -N r The process of G r→s Including G3 and G4, G s→r Including G1 and G2; D1 (G r→s (O r ) represents the discriminator D1's response to the input image B r -N s To judge, D2(G s→r (O s ) represents the discriminator D2's response to the input image B s -N r Make a judgment;

[0024] The supervised mean squared error loss includes: utilizing the correspondence between the true label and the fitted image, implementing supervised constraints within the fitted image branch, and generating a clean image by introducing stripe noise based on the fitted image, thereby establishing a reliable supervision basis and enhancing the performance of the model; the supervised mean squared error loss can be expressed as:

[0025] L mse =||G1(O s )-B|| 2 +||G1(G s→r (O s )-B)|| 2 =||B s -B|| 2 +||B′ s -B|| 2 (2)

[0026] Among them, L mse represents the supervised loss constraint of the fitting image branch, G1(O s ) represents the fitting image O of the generator G1 to the input s Generate, B represents the real label, G s→r (O s ) represents the first stage generated image B s -N r The process of G1(G s→r (O s ) represents the generator G1 for image B s -N r The regenerated fitting image background B′ is generated s , B s represents the generated fitting image background, || || 2 It means calculating the average of the sum of squares of the differences between the true value and the predicted value;

[0027] The self-consistency loss includes: applying consistency constraints to ensure the consistency of information between background images, maintaining consistency between the input image and the final generated image, maintaining a clean background image, and enhancing the semi-supervised decoupled transformation network model's attention to stripe noise generation and its conversion. The self-consistency loss can be expressed as:

[0028]

[0029] For the input and output of the model, both the fitted image and the real image must remain unchanged, which can be expressed as:

[0030]

[0031] in, and denote the consistency loss constraints on background information in the fitting and real image branches, and denote the consistency loss constraints on image information in the fitting and real image branches respectively; E Os Indicates the mathematical expectation of the results of the fitting image branch, Indicates the mathematical expectation of the result of the real image branch; s represents the input fitting image, O r represents the real input image; G r→s (O r ) and G s→r (O s ) represent the first stage generated image B r -N s and B s -N r The process of F r→s and F s→r Denote the second generation stage of fitting image and real image branches respectively; s ' and O r 'represent the fitted degraded image and the real degraded image generated after the two stages respectively; || ||1 represents the L1 norm, that is, the sum of the absolute values of each element.

[0032] Furthermore, the semi-supervised decoupled transformation network model also includes a comprehensive objective function, wherein the comprehensive objective function includes unsupervised adversarial loss, self-consistency loss and supervised mean square error loss, and the comprehensive objective function can be expressed as:

[0033]

[0034] Where L represents the total loss constraint, and denote the adversarial loss constraints of the fitted image and real image branches, L mse represents the supervised loss constraint for fitting the image branch, and denote the consistency loss constraints on background information in the fitting and real image branches, and denotes the consistency loss constraints on image information in the fitting and real image branches respectively; λ adv ,λ mse ,λ cyc and denote the balance parameters of adversarial loss, supervised mean squared error loss and self-consistency loss, respectively.

[0035] Furthermore, the hyperspectral remote sensing image to be processed is input into the trained semi-supervised decoupling transform network model to obtain a hyperspectral remote sensing image with stripe noise removed, including: removing the stripe noise in the fitting image by the generator G1 in the trained semi-supervised decoupling transform network model, and removing the stripe noise in the real image by the generator G3 in the trained semi-supervised decoupling transform network model.

[0036] In a second aspect, the present application also provides a hyperspectral remote sensing image stripe noise removal system, comprising:

[0037] The original training data preprocessing module is used to preprocess the original training data set to obtain the training data set;

[0038] A model training module is used to train the semi-supervised decoupling transformation network model based on the training data set to obtain a trained semi-supervised decoupling transformation network model;

[0039] The stripe noise removal module is used to input the hyperspectral remote sensing image to be processed into the trained semi-supervised decoupling transformation network model to obtain a hyperspectral remote sensing image with stripe noise removed.

[0040] In a third aspect, the present application provides an electronic device, comprising:

[0041] One or more processors; a memory; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and the one or more computer programs include instructions that, when executed by the device, cause the device to perform the method as described in the first aspect.

[0042] In a fourth aspect, the present application provides a computer-readable storage medium, which stores a computer program. When the computer-readable storage medium is run on a computer, the computer executes the method described in the first aspect.

[0043] In a fifth aspect, the present application provides a computer program, which, when executed by a computer, is used to execute the method described in the first aspect.

[0044] In one possible design, the program in the fifth aspect may be stored in whole or in part on a storage medium packaged with the processor, or may be stored in whole or in part on a memory not packaged with the processor.

[0045] This application has the following beneficial effects:

[0046] 1. This application proposes a new hyperspectral remote sensing image destriping algorithm, a semi-supervised decoupled transform network. The algorithm mainly encourages the model to learn the real stripe noise distribution through image decoupling and noise transformation. Compared with using a hand-crafted stripe noise generation model, learning physical degradation directly from real images can implicitly provide a more accurate approximation to the real data.

[0047] 2. This application proposes a decoupling strategy that effectively retains the consistent background by combining self-consistency loss and adversarial loss. Unlike directly converting the fitted image to the real image, this application specifically targets the conversion of the striped noise component while discarding the clean background component. The method provided by this application significantly reduces the parity between the fitted image and the real image, thereby bridging the gap between the two fields.

[0048] 3. This application adopts a combination of unsupervised adversarial loss, self-consistency loss and supervised mean square error loss in each intermediate stage of the semi-supervised decoupled transform network model, which can not only remove stripe noise but also generate high-quality images. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 is an exemplary system architecture diagram to which the embodiments of the present application can be applied;

[0050] Figure 2 This is a flow chart of a method for removing stripe noise from a hyperspectral remote sensing image according to an embodiment of the present application;

[0051] Figure 3 This is a schematic diagram of the semi-supervised decoupling transformation network structure of an embodiment of the present application;

[0052] Figure 4 Schematic diagram of the structures of Examples G1 and G3 of the present application;

[0053] Figure 5 Schematic diagram of the structures of Examples G2 and G4 of the present application;

[0054] Figure 6 Schematic diagram of the structures of Examples D1 and D2 of the present application;

[0055] Figure 7 This is a system flow chart of an embodiment of the present application;

[0056] Figure 8 It is a schematic diagram of a computer device according to an embodiment of the present application. DETAILED DESCRIPTION

[0057] Unless otherwise defined, all technical and scientific terms used in this application have the same meanings as commonly understood by those skilled in the art to which this application belongs. The terms used in the specification and description of this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The terms "including" and "having" and any variations thereof in the specification and claims of this application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0058] References to "embodiments" in this application mean that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described in this application may be combined with other embodiments.

[0059] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0060] like Figure 1 As shown, system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. Network 104 is a medium for providing communication links between terminal devices 101, 102, 103 and server 105. Network 104 may include various connection types, such as wired or wireless communication links or fiber optic cables.

[0061] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, and 103, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0062] Terminal devices 101, 102, and 103 can be various electronic devices with display screens and support web browsing, including but not limited to smartphones, tablet computers, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III, Moving Picture Experts Group Audio Layer 3), MP4 (Moving Picture Experts Group Audio Layer IV, Moving Picture Experts Group Audio Layer 4) players, laptop computers, desktop computers, etc.

[0063] The server 105 may be a server that provides various services, such as a background server that provides support for web pages displayed on the terminal devices 101 , 102 , and 103 .

[0064] It should be noted that the hyperspectral remote sensing image stripe noise removal method provided in the embodiment of the present application is generally executed by a server / terminal device. Accordingly, the hyperspectral remote sensing image stripe noise removal system is generally provided in the server / terminal device.

[0065] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is merely illustrative. Any number of terminal devices, networks and servers may be provided as required.

[0066] Continue to refer Figure 2 , the figure shows a flow chart of a method for removing stripe noise from a hyperspectral remote sensing image of the present application, the method comprising the following steps:

[0067] Step S1: preprocess the original training data set to obtain the training data set.

[0068] The preprocessing of the original training data set to obtain the training data set includes: cropping the original training data set into images of a preset size and normalizing the grayscale values of the cropped images, wherein the original training data set includes a fitting image and a real image.

[0069] Step S2: training the semi-supervised decoupling transformation network model based on the training data set to obtain a trained semi-supervised decoupling transformation network model.

[0070] For the specific structure of the semi-supervised decoupled transformation network SDTNet, please refer to Figure 3This application proposes a method for removing stripe noise from hyperspectral remote sensing images based on a semi-supervised decoupled transform network SDTNet. The semi-supervised decoupled transform network SDTNet includes two branches, namely the fitted image branch and the real image branch, which aims to bridge the gap between the two fields. Instead of relying on direct conversion, this application integrates the removal and generation of stripe noise in each module, thereby enhancing the overall robustness; in addition, this application introduces self-consistency loss and adversarial loss to better maintain the identity of the clean background component.

[0071] The goal of this application is to estimate the clean background B from the real degraded image O; however, fitting the training image O s and the real test image O r The significant difference between will seriously hinder the performance of methods based on fully supervised learning. To overcome this challenge, this application introduces SDTNet, which effectively approximates the stripe noise distribution in real data through an implicit method. Figure 3 shown. Figure 3 The corresponding alphabet explanation is shown in Table 1. In SDTNet, two branches are used to perform image decoupling and noise transfer on the fitted data and real data respectively.

[0072] Table 1 Description of letters in SDTNet

[0073]

[0074]

[0075] This application provides two key benefits: first, SDTNet establishes a robust relationship between the real and fitted degraded images by iteratively decoupling and generating stripe noise components, strengthening the connection between the two image domains; second, the overall architecture promotes feature propagation in the two tasks, resulting in a mutually reinforcing effect, thereby enhancing the performance of the semi-supervised disentangled transform network model.

[0076] In one possible implementation, the semi-supervised decoupled transformation network model is trained based on a training data set to obtain a trained semi-supervised decoupled transformation network model, wherein the semi-supervised decoupled transformation network model includes a fitting image branch and a real image branch, the fitting image branch is subject to supervised constraints, and the real image branch is subject to unsupervised constraints.

[0077] In one possible implementation, the fitted image branch is subject to supervised constraints, and the real image branch is subject to unsupervised constraints, including: during the training process, the fitted image branch is guided by supervised constraints using ground truth mapping, while the real image branch is subject to unsupervised constraints to guide the real image branch through a discriminant model.

[0078] In a possible implementation, the semi-supervised decoupled transformation network model includes four generators G1, G2, G3, and G4 and two discriminators D1 and D2.

[0079] In one possible implementation, the generators G1 and G3 use a U-net structure to generate clean background components from degraded input images (fitted images and real images). The network structure is as follows: Figure 4 As shown in Figure 1, the input image undergoes an encoding process that involves a series of downsampling operations to extract deep features and expand the network's receptive field. Subsequently, the decoding process continues with a series of upsampling operations to generate a clean background image layer that matches the input size. Within the network, information flow is enhanced by concatenating features of the same size from the encoding and decoding processes.

[0080] In a possible implementation, the generators G2 and G4 use the ResNet structure to generate the strip noise components between the fitted data and the real data; the network structure is as follows: Figure 5 As shown in the figure, the input degraded image passes through the convolution layer to expand the receptive field, and the residual module deepens the network without changing the size of the input features; finally, upsampling produces the features of the output strip noise layer with the same size as the original input, and the abstraction is enhanced. The residual module improves the model's ability to extract meaningful features by establishing a connection between the forward and backward feature information flows.

[0081] In one possible implementation, in the semi-supervised decoupled transformation network model of this application, the discriminators D1 and D2 use a PatchGAN structure that is different from the traditional GAN discriminator. The traditional GAN discriminator consists of a convolutional layer and a fully connected layer, and outputs a scalar value for overall discrimination. The network structure is as follows Figure 6 As shown in Figure 1, PatchGAN uses a fully convolutional network structure to generate an n×n matrix, where each element represents the discrimination of a local region of the input image (a specific area in the receptive field). The discrimination of the entire image is obtained by calculating the average of the matrix. This local focus allows PatchGAN to capture fine-grained details and generate high-quality generated images.

[0082] In one possible implementation, the semi-supervised decoupling transformation network model learns the true stripe noise distribution through decoupling and noise transformation, specifically as follows:

[0083] Based on the generator G1 and generator G3 in the semi-supervised decoupled transform network model, the clean background components are decoupled respectively, and the fitting image stripe noise component and the real image stripe noise component are further extracted from the fitting image and the real image;

[0084] Generate stripe noise from the fitting domain to the real domain based on generator G2, and generate stripe noise from the real domain to the fitting domain based on generator G4;

[0085] The discriminator D1 is trained to distinguish the fitted image O s and the generated degraded image B r -N s True and false instances, encouraging the generator G2 to reduce the fitting image O s and the generated degraded image B r -N s The discriminator D2 is used to enhance the ability of the generator G4 to transfer the stripe noise from the real domain to the fitting domain.

[0086] A combination of unsupervised adversarial loss, self-consistency loss, and supervised mean square error loss is adopted in the intermediate stage of the semi-supervised decoupled transformer network model;

[0087] The unsupervised adversarial loss includes: for the input fitting image O s and the real image O r , respectively for the degraded image B generated in the first stage s -N r and B r -N s The adversarial constraint is applied to achieve the transformation generation of the striped noise image in an unsupervised manner. The unsupervised adversarial loss can be expressed as:

[0088]

[0089]

[0090] in, and denote the adversarial loss constraints of the fitted image and real image branches, E Os Indicates the mathematical expectation of the results of the fitting image branch, E Or Indicates the mathematical expectation of the result of the real image branch; D1 and D2 represent two discriminators, D1(O s ) represents the fitting image O of the discriminator D1 to the input s To judge, D2(O r ) represents the discriminator D2's response to the input real image O r Make a judgment; G r→s (O r ) and G s→r (O s ) represent the first stage generated image B r -N s and B s -N rThe process of G r→s Including G3 and G4, G s→r Including G1 and G2; D1 (G r→s (O r ) represents the discriminator D1's response to the input image B r -N s To judge, D2(G s→r (O s ) represents the discriminator D2's response to the input image B s -N r Make a judgment;

[0091] The supervised mean squared error loss includes: utilizing the correspondence between the true label and the fitted image, implementing supervised constraints within the fitted image branch, and generating a clean image by introducing stripe noise based on the fitted image, thereby establishing a reliable supervision basis and enhancing the performance of the model; the supervised mean squared error loss can be expressed as:

[0092] L mse =||G1(O s )-B|| 2 +||G1(G s→r (O s )-B)|| 2 =||B s -B|| 2 +||B′ s -B|| 2 (2)

[0093] Among them, L mse represents the supervised loss constraint of the fitting image branch, G1(O s ) represents the fitting image O of the generator G1 to the input s Generate, B represents the real label, G s→r (O s ) represents the first stage generated image B s -N r The process of G1(G s→r (O s ) represents the generator G1 for image B s -N r The regenerated fitting image background B′ is generated s , B s represents the generated fitting image background, |||| 2 It means calculating the average of the sum of squares of the differences between the true value and the predicted value;

[0094] The self-consistency loss includes: applying consistency constraints to ensure the consistency of information between background images, maintaining consistency between the input image and the final generated image, maintaining a clean background image, and enhancing the semi-supervised decoupled transformation network model's attention to stripe noise generation and its conversion. The self-consistency loss can be expressed as:

[0095]

[0096] For the input and output of the model, both the fitted image and the real image must remain unchanged, which can be expressed as:

[0097]

[0098] in, and denote the consistency loss constraints on background information in the fitting and real image branches, and denote the consistency loss constraints on image information in the fitting and real image branches respectively; E Os Indicates the mathematical expectation of the results of the fitting image branch, E Or Indicates the mathematical expectation of the result of the real image branch; s represents the input fitting image, O r represents the real input image; G r→s (O r ) and G s→r (O s ) represent the first stage generated image B r -N s and B s -N r The process of F r→s and F s→r Denote the second generation stage of fitting image and real image branches respectively; s ' and O r 'represent the fitted degraded image and the real degraded image generated after the two stages respectively; ||||1 represents the L1 norm, that is, the sum of the absolute values of each element.

[0099] In one possible implementation, the semi-supervised decoupled transformation network model further includes a comprehensive objective function, wherein the comprehensive objective function includes unsupervised adversarial loss, self-consistency loss, and supervised mean square error loss. The comprehensive objective function can be expressed as:

[0100]

[0101] Where L represents the total loss constraint, and denote the adversarial loss constraints of the fitted image and real image branches, L mse represents the supervised loss constraint for fitting the image branch, and denote the consistency loss constraints on background information in the fitting and real image branches, and denotes the consistency loss constraints on image information in the fitting and real image branches respectively; λ adv ,λ mse ,λ cyc and denote the balance parameters of adversarial loss, supervised mean squared error loss and self-consistency loss, respectively.

[0102] The comprehensive objective function included in the semi-supervised disentangled transform network model presented in this application addresses the challenge of lacking paired, real-world degraded images. To preserve content information and maintain the desired output domain, each disentangled and generative subnetwork in this semi-supervised disentangled transform network model is subject to at least one unsupervised adversarial loss and one self-supervised self-consistency loss. These losses play a crucial role in leveraging clean background and stripe noise components, guiding the overall process, and ensuring the desired results.

[0103] Step S3: input the hyperspectral remote sensing image to be processed into the trained semi-supervised decoupled transform network model to obtain a hyperspectral remote sensing image with stripe noise removed.

[0104] In a possible implementation, the hyperspectral remote sensing image to be processed is input into a trained semi-supervised decoupled transform network model to obtain a hyperspectral remote sensing image with stripe noise removed, including: removing the stripe noise in the fitted image by the generator G1 in the trained semi-supervised decoupled transform network model, and removing the stripe noise in the real image by the generator G3 in the trained semi-supervised decoupled transform network model.

[0105] In the process of training the semi-supervised decoupled transform network model, a possible implementation detail of this application is as follows: a total of 3600 images are cropped from the dataset of the University of Houston, each image is 256×256 in size, to train the semi-supervised decoupled transform network model, and the grayscale values of the training images are normalized to the range of [0, 1]; the balance weight λ is empirically set to adv ,λ cyc ,λ mseThe learning rates of the G1 and G3 networks in the model were set to 2, 10, and 20, respectively. The Adam optimizer was used during model training. Before training SDTNet, a U-Net model was pre-trained on the fitted data with a learning rate of 0.00001 and 20 epochs. The G1 and G3 networks in the model were then initialized using the trained parameters of the U-Net. This initialization significantly accelerated the training of the entire model. The learning rates of G1 and G3 were set to 0.00001, while those of G2, G4, D1, and D2 were set to 0.0001. By assigning smaller learning rates to networks initialized with pre-trained parameters and larger learning rates to other networks, the stability of model training was improved. The entire model was trained for 40 epochs, ensuring comprehensive convergence and improved performance.

[0106] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When executed, the program can include the processes in the above-described method embodiments. The aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0107] It should be understood that although the steps in the flowcharts of the accompanying drawings are shown in sequence as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this application, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the flowcharts of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0108] Continue to refer Figure 7 The hyperspectral remote sensing image stripe noise removal system described in this embodiment includes:

[0109] The original training data preprocessing module 701 is used to preprocess the original training data set to obtain the training data set;

[0110] A model training module 702 is used to train the semi-supervised decoupling transformation network model based on the training data set to obtain a trained semi-supervised decoupling transformation network model;

[0111] The stripe noise removal module 703 is used to input the hyperspectral remote sensing image to be processed into the trained semi-supervised decoupled transform network model to obtain a hyperspectral remote sensing image with stripe noise removed.

[0112] To solve the above technical problems, the present application also provides a computer device. Figure 8 , Figure 8 This is a basic structural block diagram of the computer device in this embodiment.

[0113] The computer device 8 includes a memory 8a, a processor 8b, and a network interface 8c that are interconnected through a system bus. It should be noted that the figure only shows a computer device 8 having components 8a-8c, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to a microprocessor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a digital signal processor (DSP), an embedded device, etc.

[0114] The computer device may be a desktop computer, notebook computer, PDA, cloud server, etc. The computer device may interact with the user via a keyboard, mouse, remote control, touchpad, or voice control device.

[0115] The memory 8a includes at least one type of readable storage medium, including flash memory, a hard disk, a multimedia card, a card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic storage, a magnetic disk, an optical disk, etc. In some embodiments, the memory 8a may be an internal storage unit of the computer device 8, such as the hard disk or memory of the computer device 8. In other embodiments, the memory 8a may also be an external storage device of the computer device 8, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash memory card, etc. equipped on the computer device 8. Of course, the memory 8a may also include both the internal storage unit of the computer device 8 and its external storage device. In this embodiment, the memory 8a is generally used to store the operating system and various application software installed on the computer device 8, such as the program code of the hyperspectral remote sensing image stripe noise removal method. In addition, the memory 8a can also be used to temporarily store various types of data that have been output or are to be output.

[0116] In some embodiments, the processor 8b can be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip. The processor 8b is generally used to control the overall operation of the computer device 8. In this embodiment, the processor 8b is used to execute program code stored in the memory 8a or process data, such as executing the program code for the hyperspectral remote sensing image stripe noise removal method.

[0117] The network interface 8c may include a wireless network interface or a wired network interface. The network interface 8c is generally used to establish a communication connection between the computer device 8 and other electronic devices.

[0118] The present application also provides another embodiment, namely, providing a non-volatile computer-readable storage medium, which stores a program of a method for removing stripe noise from a hyperspectral remote sensing image. The method for removing stripe noise from a hyperspectral remote sensing image can be executed by at least one processor, so that the at least one processor performs the steps of the method for removing stripe noise from a hyperspectral remote sensing image as described above.

[0119] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0120] Obviously, the embodiments described above are only some of the embodiments of the present application, rather than all of the embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application has been described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions described in the aforementioned specific embodiments, or to make equivalent replacements for some of the technical features therein. Any equivalent structure made using the contents of the present application specification and the accompanying drawings, directly or indirectly used in other related technical fields, is also within the scope of patent protection of the present application.

Claims

1. A method for removing stripe noise from a hyperspectral remote sensing image, characterized in that: include: Preprocess the original training data set to obtain the training data set; The semi-supervised decoupling transformation network model is trained based on the training data set to obtain a trained semi-supervised decoupling transformation network model; the semi-supervised decoupling transformation network model learns the real stripe noise distribution through decoupling and noise transformation, which is specifically manifested as follows: based on the generator G1 and the generator G3 in the semi-supervised decoupling transformation network model, the clean background component is decoupled respectively, and the fitting image stripe noise component and the real image stripe noise component are further extracted from the fitting image and the real image; based on the generator G2, stripe noise from the fitting domain to the real domain is generated, and based on the generator G4, stripe noise from the real domain to the fitting domain is generated; the discriminator D1 is trained to distinguish the fitting image O s and the generated degraded image B r -N s True and false instances, the generator G2 reduces the fitting image O s and the generated degraded image B r -N s The difference in noise distribution between them; the discriminator D2 is used to enhance the ability of the generator G4 to transfer the stripe noise from the real domain to the fitting domain; the intermediate stage of the semi-supervised decoupled transformation network model adopts a combination of unsupervised adversarial loss, self-consistency loss and supervised mean square error loss, that is: for the input fitting image O s and the real image O r , respectively for the degraded image B generated in the first stage s -N r and B r -N s Perform adversarial constraints; apply consistency constraints to ensure the consistency of information between background images, and maintain consistency between the input image and the final generated image, maintain a clean background image, and enhance the semi-supervised decoupled transform network model's attention to stripe noise generation and its conversion; through supervised mean square error loss, use the correspondence between the true label and the fitted image, implement supervised constraints within the fitted image branch, and generate a clean image by introducing stripe noise based on the fitted image; The hyperspectral remote sensing image to be processed is input into the trained semi-supervised decoupling transform network model to obtain a hyperspectral remote sensing image with stripe noise removed.

2. The method for removing stripe noise from hyperspectral remote sensing images according to claim 1, wherein: The preprocessing of the original training data set to obtain the training data set includes: cropping the original training data set into images of a preset size and normalizing the grayscale values of the cropped images, wherein the original training data set includes a fitting image and a real image.

3. The method for removing stripe noise from hyperspectral remote sensing images according to claim 1, wherein: The semi-supervised decoupling transformation network model is trained based on the training data set to obtain a trained semi-supervised decoupling transformation network model, wherein the semi-supervised decoupling transformation network model includes a fitting image branch and a real image branch, the fitting image branch is subject to supervised constraints, and the real image branch is subject to unsupervised constraints.

4. The method for removing stripe noise from hyperspectral remote sensing images according to claim 3, characterized in that: The fitting image branch is subject to supervised constraints, and the real image branch is subject to unsupervised constraints, including: During training, the fitted image branch is guided by a supervised constraint using the ground truth mapping, while the real image branch is guided by an unsupervised constraint via the discriminative model.

5. The method for removing stripe noise from hyperspectral remote sensing images according to claim 4, characterized in that: The semi-supervised decoupled transformation network model includes four generators G1, G2, G3, and G4 and two discriminators D1 and D2.

6. The method for removing stripe noise from hyperspectral remote sensing images according to claim 1, wherein: A combination of unsupervised adversarial loss, self-consistency loss, and supervised mean square error loss is adopted in the intermediate stage of the semi-supervised decoupled transformer network model; The unsupervised adversarial loss includes: for the input fitting image O s and the real image O r , respectively for the degraded image B generated in the first stage s -N r and B r -N s The adversarial constraint is applied to transform and generate the striped noise image in an unsupervised manner. The unsupervised adversarial loss is expressed as: in, and denote the adversarial loss constraints of the fitted image and real image branches, respectively. Indicates the mathematical expectation of the results of the fitting image branch, Indicates the mathematical expectation of the result of the real image branch; D1 and D2 represent two discriminators, D1(O s ) represents the fitting image O of the discriminator D1 to the input s To judge, D2(O r ) represents the discriminator D2's response to the input real image O r Make a judgment; G r→s (O r ) and G s→r (O s ) represent the first stage generated image B r -N s and B s -N r The process of G r→s Including G3 and G4, G s→r Including G1 and G2; D1 (G r→s (O r ) represents the discriminator D1's response to the input image B r -N s To judge, D2(G s→r (O s ) represents the discriminator D2's response to the input image B s -N r Make a judgment; The supervised mean squared error loss includes: utilizing the correspondence between the true label and the fitted image, implementing a supervised constraint within the fitted image branch, and generating a clean image by introducing stripe noise based on the fitted image, thereby establishing a reliable supervision basis and enhancing the performance of the model; the supervised mean squared error loss is expressed as: L mse D||G1(O s )-B|| 2 +||G1(G s→r (YOU ARE s )-B)|| 2 D||B s -B|| 2 +||B′ s -B|| 2 (2) Among them, L mse represents the supervised loss constraint of the fitting image branch, G1(O s ) represents the fitting image O of the generator G1 to the input s Generate, B represents the real label, G s→r (O s ) represents the first stage generated image B s -N r The process of G1(G s→r (O s ) represents the generator G1 for image B s -N r The regenerated fitting image background B is generated s ′,B s represents the generated fitting image background, || || 2 It means calculating the average of the sum of squares of the differences between the true value and the predicted value; The self-consistency loss includes: applying consistency constraints to ensure the consistency of information between background images, maintaining consistency between the input image and the final generated image, maintaining a clean background image, and enhancing the semi-supervised decoupled transformer network model's attention to stripe noise generation and its conversion. The self-consistency loss is expressed as: For the input and output of the model, both the fitted image and the real image must remain unchanged, expressed as: in, and denote the consistency loss constraints on background information in the fitting and real image branches, respectively. and Represent the consistency loss constraints on image information in the fitting and real image branches respectively; Indicates the mathematical expectation of the results of the fitting image branch, Indicates the mathematical expectation of the result of the real image branch; s represents the input fitting image, O r represents the real input image; G r→s (O r ) and G s→r (O s ) represent the first stage generated image B r -N s and B s -N r The process of F r→s and F s→r Denote the second generation stage of fitting image and real image branches respectively; s ' and O r 'represent the fitted degraded image and the real degraded image generated after the two stages respectively; || ||1 represents the L1 norm, that is, the sum of the absolute values of each element.

7. The method for removing stripe noise from hyperspectral remote sensing images according to claim 6, characterized in that: The semi-supervised decoupled transformation network model also includes a comprehensive objective function, wherein the comprehensive objective function includes unsupervised adversarial loss, self-consistency loss and supervised mean square error loss, and the comprehensive objective function is expressed as: Where L represents the total loss constraint, and denote the adversarial loss constraints of the fitted image and real image branches, L mse represents the supervised loss constraint for fitting the image branch, and denote the consistency loss constraints on background information in the fitting and real image branches, respectively. and denotes the consistency loss constraints on image information in the fitting and real image branches respectively; λ adv ,λ mse ,λ cyc and denote the balance parameters of adversarial loss, supervised mean squared error loss and self-consistency loss, respectively.

8. The method for removing stripe noise from hyperspectral remote sensing images according to claim 1, wherein: The method comprises inputting the hyperspectral remote sensing image to be processed into the trained semi-supervised decoupling transform network model to obtain the hyperspectral remote sensing image with stripe noise removed, including: removing the stripe noise in the fitting image by the generator G1 in the trained semi-supervised decoupling transform network model, and removing the stripe noise in the real image by the generator G3 in the trained semi-supervised decoupling transform network model.

9. A hyperspectral remote sensing image stripe noise removal system, characterized in that: include: The original training data preprocessing module is used to preprocess the original training data set to obtain the training data set; The model training module is used to train the semi-supervised decoupling transformation network model based on the training data set to obtain a trained semi-supervised decoupling transformation network model; the semi-supervised decoupling transformation network model learns the real stripe noise distribution through decoupling and noise transformation, which is specifically manifested as follows: based on the generator G1 and the generator G3 in the semi-supervised decoupling transformation network model, the clean background component is decoupled respectively, and the fitting image stripe noise component and the real image stripe noise component are further extracted from the fitting image and the real image; based on the generator G2, stripe noise is generated from the fitting domain to the real domain, and based on the generator G4, stripe noise is generated from the real domain to the fitting domain; the discriminator D1 is trained to distinguish the fitting image O s and the generated degraded image B r -N s True and false instances, the generator G2 reduces the fitting image O s and the generated degraded image B r -N s The difference in noise distribution between them; the discriminator D2 is used to enhance the ability of the generator G4 to transfer the stripe noise from the real domain to the fitting domain; the intermediate stage of the semi-supervised decoupled transformation network model adopts a combination of unsupervised adversarial loss, self-consistency loss and supervised mean square error loss, that is: for the input fitting image O s and the real image O r , respectively for the degraded image B generated in the first stage s -N r and B r -N s Perform adversarial constraints; apply consistency constraints to ensure the consistency of information between background images, and maintain consistency between the input image and the final generated image, maintain a clean background image, and enhance the semi-supervised decoupled transform network model's attention to stripe noise generation and its conversion; through supervised mean square error loss, use the correspondence between the true label and the fitted image, implement supervised constraints within the fitted image branch, and generate a clean image by introducing stripe noise based on the fitted image; The stripe noise removal module is used to input the hyperspectral remote sensing image to be processed into the trained semi-supervised decoupling transformation network model to obtain a hyperspectral remote sensing image with stripe noise removed.

Images