Spectral graph blind universal denoising method based on dual consistency and reciprocal adversarial strategy

Abstract

The application discloses a spectrum blind general denoising method based on a dual consistency and reciprocal countermeasure strategy, comprising the following steps: constructing radar spectrum data, dividing a training set and a test set; building a training model; training a generative adversarial network model based on a dual consistency learning method and a reciprocal countermeasure strategy, and reserving a generator in the generative adversarial network as a denoiser; inputting the test set into the denoiser for denoising processing to obtain a denoised spectrum. The application introduces a guide branch on the generator in the generative adversarial network to help generate better converted input spectrum; the application helps the network model to be better trained through the dual consistency learning method and the reciprocal countermeasure strategy, effectively reduces the generalization error of the model on noise samples different from the training set, and improves the denoising performance; the application does not introduce additional enhanced data in the training process, reduces the demand for training data, saves the cost, and solves the blind general denoising problem of human body micro-Doppler signals in practice.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, specifically to a general denoising method for blind spectral images based on duality consistency and reciprocal adversarial strategies. Background Technology

[0002] With increasing demands for intelligence, human detection and identification have attracted widespread attention in recent years. Compared to optical sensors, radar can capture human movement information over long distances and has many advantages, such as strong adaptability to harsh environments and protection of human privacy.

[0003] In the real world, the presence of noise reduces the signal-to-noise ratio (SNR) of human echo signals acquired by radar sensors, significantly impacting the performance of models for subsequent problems such as human motion classification and identity recognition. To address this issue, some studies have investigated the feasibility of denoising human micro-Doppler signals. Early denoising methods relied on prior knowledge of the noise distribution, making them unsuitable for blind denoising problems, as prior knowledge is unavailable. With the rapid development of deep learning, blind denoising has been gradually solved because deep learning methods can extract prior knowledge from training set samples.

[0004] Deep learning methods leverage the powerful learning capabilities of neural networks to extract prior knowledge from training set samples. However, this capability also introduces another challenge: the model's tendency to overfit to the training samples. Overfitting in deep learning forces the model to perform well in denoising only on samples with the same prior knowledge as the training set. When faced with samples whose noise levels differ from the training set, the model's denoising performance deteriorates significantly. Therefore, an overfitted model cannot perform well in denoising under new conditions.

[0005] This task is defined as a blind universal denoising problem, which requires the denoiser to be applicable to both blind and general scenarios. "Blind" means that no prior knowledge of the sample to be denoised is required, while "universal" means that the denoiser can be used in all scenarios with different noise levels. Without prior knowledge, in the blind universal denoising problem, the denoiser faces the problem of insufficient or excessive denoising, resulting in human motion information in the radar signal being interfered with or lost by noise.

[0006] Previous blind denoisers have failed in the blind general denoising problem due to overfitting. In this problem, samples obtained from new scenes are likely to have different data distributions than those in predetermined scenes, causing the performance of the overfitting blind denoiser to degrade as the distribution discrepancy increases. When this performance degradation occurs, we consider the denoiser's generalization ability to be poor, meaning it has a high generalization error for samples outside the training set. High-quality human micro-Doppler signals are a crucial foundation for subsequent scientific research. Therefore, minimizing the noise generalization error of the denoiser for new samples outside the training set, thereby improving the performance of the denoiser for blind general denoising of human micro-Doppler signals, has research value and practical significance. Summary of the Invention

[0007] To address the problems existing in the background technology, the purpose of this invention is to provide a blind general denoising method for spectral maps based on duality consistency and reciprocal adversarial strategies. This method can effectively address the blind general denoising preprocessing problem in radar spectral maps and solves the problem that existing denoisers have high generalization errors for samples outside the training set.

[0008] To achieve the above objectives, the present invention provides the following technical solution:

[0009] A general spectral blind denoising method based on duality consistency and reciprocal adversarial strategies includes the following steps:

[0010] S1. Construct radar spectral data and divide the dataset into training and testing datasets;

[0011] S2. Build and train the model, including a denoising generative adversarial network and a noisy generative adversarial network;

[0012] S3. Input the noise spectrogram, the corresponding target spectrogram, and the guiding spectrogram from the training dataset into the constructed generative adversarial network model, train it based on the duality consistency learning method and the reciprocal adversarial strategy, update and optimize the parameters, and after training, retain the generator in the denoising generative adversarial network as the denoiser.

[0013] S4. Input the test dataset into the denoiser for denoising processing to obtain the denoised spectrum, and calculate the PSNR and SSIM index values ​​between the denoised spectrum and the target spectrum.

[0014] Preferably, in step S2, the denoising generative adversarial network performs denoising transformation on the noisy spectrum and makes a judgment; the noisy generative adversarial network performs noisy transformation on the clean spectrum and makes a judgment.

[0015] Preferably, the generator of the generative adversarial network (GAN) incorporates an additional guiding branch to assist the generator in the denoising GAN in converting the radar spectrum.

[0016] Preferably, in step S3, during the constrained training of the model using the dual consistency learning method, the radar spectrum alternately switches between the denoising generative adversarial network and the noisy generative adversarial network for discriminative optimization training, forming a loop, and introducing loop regularization constraints; simultaneously, consistency regularization constraints are introduced on both sets of generative adversarial networks. This can reduce the noise generalization error of the model to the noisy radar spectrum.

[0017] Preferably, in step S3, a reciprocal adversarial strategy is applied separately to the denoised generative adversarial network. Specifically, the intermediate features extracted by the discriminator are used for loss calculation, encouraging the generator to converge to a relatively stable Nash equilibrium. This strategy ensures semantic consistency between the denoised spectrogram and the target spectrogram at both the coarse and fine levels, meaning that the semantic information of human motion is not affected by excessive or insufficient denoising.

[0018] Preferably, in step S3, within the PyTorch deep learning framework, the training set spectrogram is input into the constructed generative adversarial network (GAN) for training. The parameter updates of the GAN model are accomplished by backpropagation of the gradient of the designed loss function. The overall loss function L used by the GAN model is... totel The specific formula is as follows:

[0019] L total =α(L advgd +L advfd )+β·L cycle +γ(L Gidt +L Fidt )+μ(L Grb +L Grp )+σ(L Drb +L Drp (1)

[0020] In the formula, α, β, γ, μ, and σ represent weighting coefficients, with values ​​of 5, 30, 15, 5 / 2, and 5 / 2, respectively; L advgd With L advfd Let L represent the adversarial loss functions of the denoised generative adversarial network and the noisy generative adversarial network, respectively. advgd With L advfd Used to identify the generated denoised spectrum and noise-added spectrum Whether it matches the target clean spectrum y and the target noise spectrum x is expressed by the following formula:

[0021]

[0022]

[0023] In the formula, Dgb and D gp It is a dual discriminator in a denoising generative adversarial network, D fb and D fp It is a dual discriminator in a noisy generative adversarial network; L cycle It is the loss function generated by loop regularization, and the specific formula is as follows:

[0024] L cycle =||F(G(x,y) s ), x s )-x||1+||F(G(y,x s ), y s )-y||1 (4)

[0025] In the formula, G is the denoising generator, F is the noise-adding generator, and y s As the guiding spectrum during the denoising process, x s This is the guiding spectrum during the noise addition process; L Gidt and L Fidt The loss function generated by consistency regularization is expressed by the following formula:

[0026]

[0027]

[0028] In the formula, V represents VGG19 loaded with pre-trained weights, which are public weights;

[0029] L Grb L Grp L Drb and L Drp Let L represent the loss function generated by the reciprocal adversarial strategy; during the generator training phase, the discriminator parameters are frozen, and the intermediate features between the denoised spectrum and the target spectrum are extracted using a denoised dual discriminator to calculate the norm difference, thus obtaining L. Grb and L Grp This makes the generated spectrogram semantically closer to the target spectrogram. During the discriminator training phase, the intermediate features between the denoised spectrogram and the target spectrogram are still extracted using a denoised dual discriminator, the norm difference is calculated, and the reciprocal is obtained, thus yielding L. Drb and L Drp This prompts the discriminator parameters to be updated to obtain a new projection space, thereby increasing the generator's learning space. The specific formula is as follows:

[0030]

[0031]

[0032]

[0033]

[0034] Where, λ i These are weighting coefficients, all of which are 1.

[0035] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0036] This invention provides a general blind denoising method for spectrograms based on duality consistency and reciprocal adversarial strategies.

[0037] This invention utilizes neural convolutional networks to construct a generative adversarial network (GAN) model, which consists of a denoising GAN and a noisy GAN. During training, this invention employs a dual consistency learning method and a reciprocal adversarial strategy to improve the model's denoising performance on unknown noise samples. In the dual consistency learning method, the two GANs form a loop and are subject to loop regularization constraints, while consistency regularization constraints are also introduced separately, thereby reducing the noise generalization error of the model on noisy radar spectra. By employing the reciprocal adversarial strategy, the intermediate features extracted by the discriminator are used to calculate the adversarial loss, encouraging the generator to converge to a relatively stable Nash equilibrium. This strategy ensures semantic consistency between the denoised spectrum and the target spectrum at both the coarse and fine levels, meaning that the semantic information of human motion is not affected by excessive or insufficient denoising. This invention does not introduce additional augmentation data during training, reducing training data requirements and saving corresponding costs, while simultaneously solving the problem of blind general denoising of human micro-Doppler signals in practice. This invention uses the dual learning concept and reciprocal adversarial strategy in deep learning to perform blind general denoising preprocessing on radar spectra, recovering human micro-Doppler signals in noisy radar spectra, and providing a reliable data source for subsequent scientific research such as human gait recognition using human micro-Doppler signals. Attached Figure Description

[0038] Figure 1 A flowchart illustrating the universal denoising method for radar spectrum blind based on dual consistency and reciprocal adversarial strategies provided by this invention.

[0039] Figure 2 A schematic diagram of a generator structure with a guiding branch;

[0040] Figure 3 This is a schematic diagram of the overall training framework based on duality consistency and reciprocal adversarial strategy;

[0041] Figure 4 This is a schematic diagram of the reciprocal adversarial strategy training method of the present invention;

[0042] Figure 5 This is a subjective schematic diagram of the denoising results of the present invention under the setting conditions of a training set signal-to-noise ratio of 20dB and a test set signal-to-noise ratio of 10dB.

[0043] Figure 6 This is a subjective schematic diagram of the denoising results of the present invention under the setting conditions of a training set signal-to-noise ratio of 20dB and a test set signal-to-noise ratio of 0dB.

[0044] Figure 7 This is a subjective schematic diagram of the denoising results of the present invention under the setting conditions of a training set signal-to-noise ratio of 20dB and a test set signal-to-noise ratio of -10dB. Detailed Implementation

[0045] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0046] To address the issue of high generalization error in existing denoisers for samples outside the training set, this invention proposes a blind general denoising method for spectral images based on duality consistency and reciprocal adversarial strategies. This method requires deploying a robust general denoising model to handle unpredictable noise distributions in measurements. However, the available samples for training the model are insufficient and lack diversity. To address this problem, this invention proposes a novel learning method based on duality consistency and utilizes a generative adversarial network (GAN) to build a blind general denoising model. This denoising model incorporates two regularization strategies during training to minimize the generalization error of the denoising model for new samples. Furthermore, this invention proposes a reciprocal adversarial training strategy. This invention is implemented using the Python programming language within the PyTorch deep learning framework. First, a GAN used for model training is built; then, parameters such as learning rate, batch size, and loss weights are set during training; next, radar spectral data from the training set is input into the network and trained on a GPU, with the network weights saved after training; finally, the resulting denoiser is used for blind general denoising testing of radar spectra. The method of this invention can effectively address the problem of blind general denoising preprocessing in radar spectra.

[0047] Example 1

[0048] Reference Figure 1 A general spectral blind denoising method based on duality consistency and reciprocal adversarial strategy includes the following steps:

[0049] S1. Construct radar spectral data and divide the dataset into training and testing datasets;

[0050] Specifically, in this embodiment, the noise distribution of the noise radar spectra in the constructed dataset is Gaussian distribution. The training dataset consists of 20 noise spectra with a signal-to-noise ratio of 20dB and corresponding clean spectra, and the test dataset consists of 100 noise spectra each with signal-to-noise ratios of 10dB, 0dB, and -10dB.

[0051] S2. Build and train the model, including a denoising generative adversarial network and a noisy generative adversarial network;

[0052] Specifically, in this embodiment, the training model includes a denoising generative adversarial network responsible for denoising and a denoising generative adversarial network responsible for adding noise.

[0053] In this system, the generators in the two sets of generative adversarial networks transform the input radar spectra; the discriminator in the generative adversarial network is used to discriminate the generated radar spectra.

[0054] Both of the aforementioned generative adversarial networks introduce a bootstrap branch in their generators, as shown in the reference. Figure 2 The generator structure of the denoising generative adversarial network is such that the guiding spectrum input to the guiding branch of the generator has the same signal-to-noise ratio as the target spectrum. The features extracted from the guiding spectrum are fused with the spectrum to be converted, thereby assisting the generator in the generative adversarial network to convert the radar spectrum.

[0055] S3. Input the noise spectrogram, the corresponding target spectrogram, and the guiding spectrogram from the training dataset into the constructed generative adversarial network model, train it based on the duality consistency learning method and the reciprocal adversarial strategy, update and optimize the parameters, and after training, retain the generator in the denoising generative adversarial network as the denoiser.

[0056] Reference Figure 3 Specifically, the hardware configuration used in this invention is an Nvidia GeForce RTX 3080Ti. Under the PyTorch deep learning framework, the training data is input into the generative adversarial network (GAN) built in step S2 for training. The specific training process is as follows: the noisy spectrum is converted into a clean spectrum by the denoising GAN, and the generated clean spectrum is then passed through the noisy GAN again to obtain a noisy spectrum, forming a loop and constructing a loop loss with the original noisy spectrum; the clean spectrum is similarly formed into a loop and a loop loss is constructed, and the clean spectrum is passed through the denoising GAN to construct a consistency loss with itself; the noisy spectrum is similarly processed by applying a reciprocal adversarial strategy separately to the denoising GAN to further improve the denoising performance of the denoising network.

[0057] In the training process of the generative adversarial network (GAN) model, this invention proposes dual consistency learning. Based on two sets of GAN structures, one GAN (i.e., a denoising GAN) is responsible for denoising, and the other GAN (i.e., a noisy GAN) is responsible for adding noise. The radar spectrum is alternately transformed and discriminated between the denoising GAN and the noisy GAN for training, forming a loop, and a loop regularization constraint is introduced. At the same time, consistency regularization constraints are introduced on both sets of GANs, thereby reducing the noise generalization error of the model to the noisy radar spectrum.

[0058] This invention also incorporates a reciprocal adversarial strategy during the training of the generative adversarial network model. This strategy utilizes intermediate features extracted by the discriminator for loss calculation, encouraging the generator to converge to a relatively stable Nash equilibrium. This strategy ensures semantic consistency between the denoised spectrogram and the target spectrogram at both the coarse and fine levels, meaning that the semantic information of human motion is not affected by excessive or insufficient denoising. Therefore, the quality of the denoised spectrogram is further improved.

[0059] In this embodiment, the parameter optimization and update of the constructed generative adversarial network model is accomplished by gradient backpropagation of the designed loss function. The overall loss function L used by the generative adversarial network model is... totel The specific formula is as follows:

[0060] L total =α(L advgd +L advfd )+β·L cycle +γ(L Gidt +L Fidt )+μ(L Grb +L Grp )+σ(L Drb +L Drp (1)

[0061] In the formula, α, β, γ, μ, and σ represent weighting coefficients, with values ​​of 5, 30, 15, 5 / 2, and 5 / 2, respectively; L advgd With L advfd Let L represent the adversarial loss functions of the denoised generative adversarial network and the noisy generative adversarial network, respectively. advgd With L advfd Used to identify the generated denoised spectrum and noise-added spectrum Whether it matches the target clean spectrum y and the target noise spectrum x is expressed by the following formula:

[0062]

[0063]

[0064] In the formula, D gb and D gp D represents the dual discriminator in a denoising generative adversarial network. fb and D fp L represents a dual discriminator in a noisy generative adversarial network; cycle This represents the loss function generated by loop regularization. The loop process utilizes the generated denoised and noisy spectra, which has a data augmentation effect. The specific formula is as follows:

[0065] L cycle =||F(G(x,y) s ), x s )-x||1+||F(G(y,x s ), y s )-y||1 (4)

[0066] In the formula, G is the denoising generator; F is the noise-adding generator; y s As the guiding spectrum during the denoising process, x s This is the guiding spectrum during the noise addition process; L Gidt and L Fidt This represents the loss function generated by consistency regularization. Consistency regularization ensures that the generator's transformation of the target spectral image remains unchanged from both pixel and semantic feature levels, i.e., it prevents the generator from altering the target domain information. The specific formula is as follows:

[0067]

[0068]

[0069] In the formula, V represents VGG19 loaded with pre-trained weights, which are public weights;

[0070] L Grb L Grp L Drb and L Drp This represents the loss function generated by the reciprocal adversarial strategy; applying the reciprocal adversarial strategy separately to the denoising generative adversarial network can further improve the denoising performance of the denoising network.

[0071] Specifically, refer to Figure 4 During the generator training phase, the discriminator parameters are frozen. A denoised dual discriminator is used to extract intermediate features from the denoised spectrum and the target spectrum, and the norm difference is calculated to obtain L. Grb and L Grp This makes the generated spectrogram semantically closer to the target spectrogram. During the discriminator training phase, the intermediate features between the denoised spectrogram and the target spectrogram are still extracted using a denoised dual discriminator, the norm difference is calculated, and the reciprocal is obtained, thus yielding L. Drb and L DrpThis prompts the discriminator parameters to be updated to obtain a new projection space, thereby increasing the generator's learning space. The specific formula is as follows:

[0072]

[0073]

[0074]

[0075]

[0076] Where, λ i This represents the weighting coefficient, and all values ​​are 1.

[0077] This invention employs multiple techniques, including guided conversion, adversarial training, dual consistency learning, and reciprocal adversarial strategy, to ensure that the generative adversarial network model improves its generalization ability to noise in new samples that differs from the training set during training, and to guarantee the preservation of human micro-Doppler target information and semantic consistency during the conversion process.

[0078] S4. Input the test dataset (the noise spectrum in it) into the denoiser for denoising processing to obtain the denoised spectrum, and calculate the PSNR and SSIM index values ​​between the denoised spectrum and the target spectrum.

[0079] Experimental Results: In this embodiment, the training dataset used 20 noise spectrograms with a signal-to-noise ratio (SNR) of 20 dB. The test set was divided into three cases: SNR of 10 dB, 0 dB, and -10 dB. That is, 20:10 in Tables 1 and 2 both represent a training set SNR of 20 dB and a test set SNR of 10 dB. The same applies to 20:0 and 20:-10. In this embodiment, three image denoising algorithms, five image translation algorithms, and one micro-Doppler denoising algorithm were selected for comparison, as shown in Tables 1 and 2. These tables show the PSNR and SSIM objective index values ​​obtained by each algorithm under different conditions on the test set.

[0080] Table 1. PSNR values ​​of denoising results at different noise levels

[0081] 20:10 13.66 17.31 19.49 17.54 12.22 18.88 12.18 21.48 15.69 22.00 20:0 6.95 14.00 13.77 14.66 5.82 13.47 5.61 16.72 14.02 17.38 20:-10 3.63 11.83 11.15 11.75 3.57 11.57 3.49 12.22 5.20 13.91 ave 8.08 14.38 14.80 14.65 7.20 14.64 7.09 16.81 11.64 17.76

[0082] Table 2. SSIM values ​​of denoising results at different noise levels

[0083] 20:10 0.344 0.553 0.603 0.483 0.347 0.619 0.269 0.736 0.219 0.771 20:0 0.128 0.392 0.355 0.408 0.126 0.373 0.116 0.528 0.122 0.582 20:-10 0.043 0.246 0.299 0.278 0.045 0.256 0.039 0.299 0.042 0.398 ave 0.172 0.397 0.419 0.390 0.173 0.416 0.141 0.521 0.128 0.584

[0084] As can be seen from the results in Tables 1 and 2 above, the PSNR and SSIM indices obtained by the method of the present invention are the highest in all cases.

[0085] Depend on Figure 5-7The subjective results presented show that the denoising effect of the method of the present invention is the best. The above results indicate that the method of the present invention has superiority in solving the problem of blind general denoising of human micro-Doppler signals.

[0086] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A general denoising method for blind spectral data based on duality consistency and reciprocal adversarial strategies, characterized in that: Includes the following steps: S1. Construct radar spectral data and divide the dataset into training and testing datasets; S2. Build and train the model, including a denoising generative adversarial network and a noisy generative adversarial network; S3. Input the noise spectrogram, the corresponding target spectrogram, and the guiding spectrogram from the training dataset into the constructed generative adversarial network model, train it based on the duality consistency learning method and the reciprocal adversarial strategy, update and optimize the parameters, and after training, retain the generator in the denoising generative adversarial network as the denoiser. S4. Input the test dataset into the denoiser for denoising processing to obtain the denoised spectrum, and calculate the PSNR and SSIM index values ​​between the denoised spectrum and the target spectrum. In S3, the overall loss function L used by the generative adversarial network model totel The specific formula is as follows: （1） In the formula, , , , as well as These represent weighting coefficients, with values ​​of 5, 30, 15, 5 / 2, and 5 / 2, respectively. and Let represent the adversarial loss functions of the denoised generative adversarial network and the noisy generative adversarial network, respectively. and Used to identify the generated denoised spectrum and noise-added spectrum Is it consistent with the target clean spectrum? and target noise spectrum The formula is consistent and is expressed as follows: （2） （3） In the formula, and It is a dual discriminator in a denoising generative adversarial network. and It is a dual discriminator in a noisy generative adversarial network; It is the loss function generated by loop regularization, and the specific formula is as follows: （4） In the formula, For noise reduction generator, For noise generator, This is a guide spectrum used in the denoising process. This is a guide spectrum during the noise addition process; and The loss function generated by consistency regularization is expressed by the following formula: （5） （6） In the formula, This indicates that a VGG19 has been loaded with pre-trained weights, which are public weights. , , and The loss function generated by the reciprocal adversarial strategy is denoted as follows: During the generator training phase, the discriminator parameters are frozen, and the intermediate features between the denoised spectrum and the target spectrum are extracted using a denoised dual discriminator to calculate the norm difference, thus obtaining the result. and This process encourages the generator to produce a spectrogram that is semantically closer to the target spectrogram. During the discriminator training phase, the intermediate features between the denoised spectrogram and the target spectrogram are still extracted using a denoised dual discriminator. The norm difference is then calculated and its reciprocal is taken, resulting in the... and This prompts the discriminator parameters to be updated to obtain a new projection space, thereby increasing the generator's learning space. The specific formula is as follows: （7） （8） （9） （10） in, These are weighting coefficients, all of which are 1.

2. The spectral blind general denoising method based on duality consistency and reciprocal adversarial strategy according to claim 1, characterized in that: In S2, the denoising generative adversarial network performs denoising transformation on the noisy spectrum and makes a judgment; the noisy generative adversarial network performs noisy transformation on the clean spectrum and makes a judgment.

3. The spectral blind general denoising method based on duality consistency and reciprocal adversarial strategy according to claim 1, characterized in that: The generator in the generative adversarial network (GAN) incorporates an additional guiding branch to assist the generator in transforming radar spectra.

4. The spectral blind general denoising method based on duality consistency and reciprocal adversarial strategy according to claim 1, characterized in that: In S3, during the process of constraining the model using the dual consistency learning method, the radar spectrogram alternately switches and performs discriminative optimization training between the denoising generative adversarial network and the noisy generative adversarial network to form a loop, and a loop regularization constraint is introduced; at the same time, consistency regularization constraints are introduced on the two sets of generative adversarial networks respectively.

5. The general spectral blind denoising method based on duality consistency and reciprocal adversarial strategy according to claim 1, characterized in that: In S3, a reciprocal adversarial strategy is applied separately to the denoising generative adversarial network. Specifically, the intermediate features extracted by the discriminator are used to calculate the adversarial loss, which encourages the generator to converge to a stable Nash equilibrium.

Images