Random noise removal method based on low-rank tensor estimation and deep learning

Abstract

The present invention provides a random noise removal method based on low-rank tensor estimation and deep learning, which belongs to the field of remote sensing image signal processing. The method includes three steps: constructing a mathematical model, solving it using the alternating direction multiplier method, constructing a deep learning network based on the alternating direction multiplier method, and adjusting parameters on a certain amount of labeled data using the deep learning method. Compared with the method based on low-rank tensor estimation, the important parameters of the present invention are automatically adjusted on a certain amount of labeled data using a deep learning method, avoiding complicated manual parameter adjustment; compared with the method based on deep learning, the method of the present invention has high interpretability and is more reliable in security-sensitive operations.

Description

Technical Field

[0001] The present invention belongs to the field of remote sensing image signal processing, and specifically relates to a random noise removal method based on low-rank tensor estimation and deep learning. Background Art

[0002] The collected remote sensing image signals contain random noise due to factors such as collection conditions, which will have a negative impact on subsequent signal analysis tasks. Existing denoising methods include: methods based on low-rank tensor estimation (LRA), methods based on deep learning (DL), etc. Although these methods have good denoising performance, they still have certain limitations: the performance of methods based on low-rank tensor estimation is sensitive to parameters, so parameters need to be manually adjusted to achieve the best performance; the end-to-end network trained by deep learning methods is not interpretable enough and lacks credibility in security-sensitive operations. Summary of the invention

[0003] The purpose of this invention is to provide a random noise removal method that combines low-rank tensor estimation and deep learning. This method constructs a highly interpretable deep learning network (named LRTNet) based on the low-rank tensor estimation method, and uses deep learning technology to automatically adjust parameters on a certain amount of labeled data.

[0004] To achieve the above object, the present invention adopts the following technical solutions:

[0005] A random noise removal method based on low-rank tensor estimation and deep learning includes the following steps:

[0006] Step S1: construct a mathematical model; select an additive noise model, introduce a weighted tensor nuclear norm and total variation regularization, and obtain an objective function; then introduce auxiliary variables, and use the augmented Lagrangian multiplier method to construct an augmented Lagrangian function;

[0007] Step S2: using the alternating direction multiplier method to solve the augmented Lagrangian function, obtaining the optimization function of each variable and the Lagrangian multiplier, and combining the optimization function into a network, the number of network layers is consistent with the number of iterations of the alternating direction multiplier method;

[0008] Step S3: Use deep learning methods to adjust parameters on a certain amount of labeled data;

[0009] Step S3.1: construct a data set, which includes noisy data and corresponding denoised data, and the data size is consistent with the size required for network propagation;

[0010] Step S3.2: Based on the network constructed in step S2, train and test on the data set and adjust the trainable parameter values;

[0011] Step S3.3: Deploy the trained network in a real-world scenario, with the input being the collected noisy data and the output being the denoising result.

[0012] The beneficial effects of the present invention are:

[0013] Compared with the method based on low-rank tensor estimation, the important parameters of the present invention are automatically adjusted on a certain amount of labeled data using a deep learning method, thus avoiding complicated manual parameter adjustment; compared with the method based on deep learning, the method of the present invention has high interpretability and is more reliable in security-sensitive operations; the present invention combines the two methods of low-rank tensor estimation and deep learning to improve the denoising performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 The figure is a flow chart of the network in the method of the present invention. DETAILED DESCRIPTION

[0015] In order to better understand the purpose, structure and function of the present invention, the following is a further detailed description of a random noise removal method based on low-rank tensor estimation and deep learning of the present invention in conjunction with the accompanying drawings.

[0016] The present invention provides a remote sensing image random noise removal method that combines low-rank tensor estimation and deep learning methods, aiming to effectively remove random noise in the collected remote sensing image signals. Compared with the denoising method based on low-rank tensor estimation, this method uses deep learning methods to adjust parameters on a certain amount of labeled data, avoiding complicated manual parameter adjustment; compared with the denoising method based on deep learning, this method has high interpretability and is more reliable in security-sensitive operations.

[0017] This method includes three steps: constructing a mathematical model, solving it using the alternating direction multiplier method, constructing a deep learning network based on the alternating direction multiplier method, and adjusting parameters on a certain amount of labeled data using the deep learning method. The details are as follows:

[0018] Step S1: construct a mathematical model; select an additive noise model, introduce a weighted tensor nuclear norm and total variation regularization, and obtain an objective function; then introduce auxiliary variables, and use the augmented Lagrangian multiplier method to construct an augmented Lagrangian function;

[0019] Step S2: using the alternating direction multiplier method to solve the augmented Lagrangian function, obtaining the optimization function of each variable and the Lagrangian multiplier, and combining the optimization function into a network, the number of network layers is consistent with the number of iterations of the alternating direction multiplier method;

[0020] Step S3: Use deep learning methods to adjust parameters on a certain amount of labeled data;

[0021] Step S3.1: construct a data set, which includes noisy data and corresponding denoised data, and the data size is consistent with the size required for network propagation;

[0022] Step S3.2: Based on the network constructed in step S2, train and test on the data set and adjust the trainable parameter values;

[0023] Step S3.3: Deploy the trained network in a real-world scenario, with the input being the collected noisy data and the output being the denoising result.

[0024] When constructing the mathematical model, the additive noise model is selected, and the weighted tensor nuclear norm and total variation regularization are introduced to obtain the objective function. By introducing auxiliary variables and using the augmented Lagrangian multiplier method, the augmented Lagrangian equation is constructed.

[0025] When solving the problem using the alternating direction multiplier method and constructing a deep learning network using the alternating direction multiplier method, the augmented Lagrange multiplier method obtained by solving the alternating direction multiplier method is first used to obtain the optimization function of each variable and the Lagrange multiplier. Then the framework of the step-by-step iteration of the augmented Lagrange multiplier method is expanded into a deep learning network, and each iteration step is used as a layer in the deep learning network. Each layer consists of 4 modules, and each module updates the variable noisy input, the auxiliary variable smoothed noisy input, the variable random noise to be removed, and the Lagrange multiplier term in turn according to the update function of the augmented Lagrangian function. The trainable parameters of the network are the weights and thresholds in the model.

[0026] The following is an embodiment for describing in detail the random noise removal method based on low-rank tensor estimation and deep learning of the present invention.

[0027] Step S1: Construct a data set; the data can be constructed based on a physical model or from real remote sensing image data. This embodiment takes the construction of a data set based on a physical model as an example: first construct a larger noise-free data, and cut the data into multiple small slice data. This embodiment selects a slice size of 80*90*100, with a total of 20 slice samples. Then add random Gaussian noise to each noise-free slice data to form noisy input data, and retain the corresponding noise-free slice data as a label.

[0028] Step S2: construct a denoising model; select an additive noise model, introduce a weighted tensor nuclear norm and total variation regularization, obtain an objective function, introduce auxiliary variables, and construct an augmented Lagrangian function using the augmented Lagrangian multiplier method;

[0029] The constructed objective function is:

[0030] In the formula: , and are the weights of each item, is a noise-free signal, is the random noise to be removed, Indicates signal The weighted tensor nuclear norm of Indicates signal The total variance norm of represents the square of the noise after Fourier transform to the frequency domain, It is the abbreviation of "subject to", which means constraints. is an additive noise model. This objective function is only used as a broad theoretical basis to describe the goal of the method; the Lagrangian function further constructed from this objective function is the specific theoretical details of the method, which will be elaborated in detail when the module is constructed.

[0031] Then, the network is constructed based on the obtained augmented Lagrangian function solution formula. The number of network layers is consistent with the expected number of iterations of the augmented Lagrangian multiplier method, which is set to 6 by default, that is, a 6-layer network by default.

[0032] Specifically, Figure 1 As shown in the figure, the noisy signal is input into a cascaded 6-layer network with parameters, and finally the updated noise-free signal is output; the single-layer network includes 4 modules, a low-rank tensor estimation module (LRA) module, a total variation regularization (TVR) module, a random noise removal (RNR) module, and a Lagrange multiplier update (LMU) module. The input data passes through the above 4 modules in turn to obtain the output; the following will explain the 4 modules respectively.

[0033] Low-rank tensor estimation module updates noise-free signals in augmented Lagrangian functions , the specific low-rank tensor estimation module implementation formula is as follows:

[0034]

[0035] In the formula, is the number of update iterations, is a noise-free signal, that is, the denoised signal finally output by the module. That is to say, The denoised signal after round of updates, if is equal to the maximum number of update rounds and satisfies the algorithm exit condition, then is the final result, output it, otherwise It is only an intermediate result, which is used for subsequent updates of the module; the soft threshold operator in the specific implementation of this embodiment use , can be replaced by other soft threshold operators; , and For Tensor singular value decomposition ( ) results, namely:

[0036]

[0037] Among them, the sword represents a tensor transpose operation, is an intermediate variable derived from the calculation, and its value is:

[0038]

[0039] in is the random noise to be removed, which is the same as the random noise removal module For the same variable, represents the auxiliary variable smoothing noisy signal, is the input noisy signal, are the parameters that the algorithm can learn: Used to represent learnable parameters, Indicates the number of iterations, Used to distinguish the learnable parameters in the algorithm, and is the Lagrange multiplier in the Lagrange multiplier method; is the weight, calculated as follows:

[0040]

[0041] in Express Element-wise squaring, is a very small positive real number to avoid the denominator being 0, and is empirically taken as 1e-5. , and Similarly, for modules, and The learnable parameters of .

[0042] Total variation regularization module updates the auxiliary variables in the augmented Lagrangian function to smooth noisy signals And other related variables, the specific total variation regularization module implementation formula is as follows:

[0043]

[0044] In the formula, and are two auxiliary variables introduced, defined by the total variance norm, satisfying ,Right now The one-norm of is equal to The total variation norm of . In the two auxiliary variable formula, is the number of iterations, is the differential reconcatenation operator, , and is along , , Differential operators in three directions, , and is along , , The weights of the three directions, Then expand the above weighted differential result into a vector, and the two auxiliary variables are only used for mathematical derivation; and Similar to the previous formula: Represented as algorithm learnable parameters, Indicates the number of iterations, and Used to refer to parameters; and They are Fourier transform and its inverse transform, differential reconstructor in frequency domain Recorded as is the unit tensor; It is an intermediate variable derived from the formula, and its value is , , and The specific implementation uses Matlab Function generation, which is a fixed value; It is an intermediate variable derived from the formula, and its value is ; The soft threshold operator in the specific implementation of this embodiment use Function, which can be replaced by other soft threshold operators; , and the Lagrange multiplier update module is the Lagrange multiplier in the Lagrange multiplier method.

[0045] The random noise removal module updates the variable random noise of the augmented Lagrangian function. The specific implementation formula of the random noise removal module is as follows:

[0046]

[0047] In the formula, It is The random noise to be removed during rounds of update iterations; is the input noisy signal; and It is round of update iterations of learnable parameters, and Used to refer to parameters.

[0048] The Lagrange multiplier update module updates the remaining Lagrange multipliers to continue the iteration of the alternating direction multiplier method. The specific implementation formula of the LMR module is as follows:

[0049]

[0050] In the formula, , and the updated total variation regularization module are all Lagrange multipliers in the Lagrange multiplier method, It is round of update iterations of learnable parameters, Used to refer to parameters.

[0051] The trainable parameters of the network are the weights and thresholds of each module in each layer, which are initialized based on experience, which can improve training efficiency and reduce the number of rounds required to complete training.

[0052] Step S3: Use the deep learning algorithm to adjust the trainable parameters of the network; initialize the variables to be updated and the Lagrange multiplier to be a tensor with a value of 0, that is:

[0053]

[0054]

[0055]

[0056]

[0057]

[0058]

[0059]

[0060] in, Indicates the generation of a A tensor of all zeros of the same size, Indicates that the input tensor is expanded into a vector. The initialization variables and noisy input data are passed into the network, and then passed through each layer of the network in turn, each variable and Lagrange multiplier are updated, and the denoising result output by the last layer of the network is taken, and the corresponding noise-free slice data is passed into the selected loss function, and the back-propagation algorithm of back-propagation deep learning is used to return the gradient, adjust the trainable parameters, and finally obtain the random noise removal model.

[0061] Train on the data set, with noisy data as input and denoising results as output. The loss function evaluates the denoising results of the network output and the annotated noise-free data, and adjusts the trainable parameter values ​​of the network. Finally, verify the trained network, that is, deploy the trained network in a real scene, input the collected noisy data, and output the required denoising results. The loss function selected in this embodiment is the L1 loss function. Practice shows that the L2 loss function and the smoothed L1 loss function also have good results.

[0062] The trained network can be deployed in actual operations. After deployment, the collected noisy data needs to be sliced ​​to meet the unique size required for network transmission, and the network should not be adjusted again to avoid performance loss. The noisy input data is passed into the network and passes through each layer of the network in turn. The denoising result output by the last layer of the network is the required denoising result.

[0063] It is to be understood that the present invention is described by some embodiments, and it is known to those skilled in the art that various changes or equivalent substitutions may be made to these features and embodiments without departing from the spirit and scope of the present invention. In addition, under the teachings of the present invention, these features and embodiments may be modified to adapt to specific circumstances and materials without departing from the spirit and scope of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are within the scope of protection of the present invention.

Claims

1. A random noise removal method based on low-rank tensor estimation and deep learning, characterized in that: The following steps are involved: Step S1: Construct a mathematical model; select an additive noise model, introduce the weighted tensor nuclear norm and total variation regularization, and obtain the objective function; then introduce auxiliary variables, and use the augmented Lagrangian multiplier method to construct the augmented Lagrangian function; the constructed objective function is: In the formula: α, μ and γ are weights of each item, is a noise-free signal, is the random noise to be removed, Indicates signal The weighted tensor nuclear norm of , Indicates signal The total variation norm of It represents the square of the noise after Fourier transform to the frequency domain. St is the abbreviation of restricted by, which means the constraint condition. is the additive noise model; Step S2: Solve the augmented Lagrangian function using the alternating direction multiplier method to obtain the optimization function of each variable and the Lagrangian multiplier, and combine the optimization function into a network. The number of network layers is consistent with the number of iterations of the alternating direction multiplier method. Each layer of the network contains 4 modules, namely, a low-rank tensor estimation module, a total variation regularization module, a random noise removal module, and a Lagrangian multiplier update module. Each module in each layer updates each variable and the Lagrangian multiplier in turn, and the input data passes through the above 4 modules in turn to obtain output. The specific implementation method of each module in the network is as follows: Low-rank tensor estimation module updates noise-free signals in augmented Lagrangian functions The specific implementation formula of the low-rank tensor estimation module is as follows: In the formula, l is the number of update iterations, That is, it represents the denoised signal after the lth round of update, is a soft threshold operator, Represents a tensor transpose operation, w l is the weight; Σ and For The result of tensor singular value decomposition t-SVD is: is an intermediate variable derived from the calculation, and its value is: in, represents the auxiliary variable smoothing noisy signal, is the noisy input signal, is the random noise to be removed, is a learnable parameter: θ is used to represent the learnable parameter, l represents the number of iterations, ρ1 is used to distinguish the learnable parameters in the formula, Γ1 and Γ2 are the Lagrange multipliers in the Lagrange multiplier method; Total variation regularization module updates the auxiliary variables in the augmented Lagrangian function to smooth noisy signals And other related variables, the specific total variation regularization module implementation formula is as follows: In the formula, is the auxiliary variable introduced, satisfying Right now The one-norm of is equal to The total variation norm of ; is the differential reconcatenation operator, is a learnable parameter; and They are Fourier transform and its inverse transform, differential reconstructor in frequency domain Recorded as is the unit tensor; It is an intermediate variable derived from the formula, and the specific calculation is as follows: in, and is a constant; Λ and Γ2 are Lagrange multipliers in the Lagrange multiplier method; The random noise removal module updates the variable random noise of the augmented Lagrangian function. The specific implementation formula of the random noise removal module is as follows: In the formula, is the random noise to be removed in the lth update iteration; and is the learnable parameter of the lth update iteration, γ and ρ3 are used to refer to the parameters; The Lagrange multiplier update module updates the remaining Lagrange multipliers to continue the iteration of the alternating direction multiplier method. The specific implementation formula of the Lagrange multiplier update module is as follows: In the formula, Γ1 and Γ2 are both Lagrange multipliers in the Lagrange multiplier method. is the learnable parameter of the lth update iteration, and ρ4 is used to refer to the parameter; Step S3: Use deep learning methods to adjust parameters on the labeled data; Step S3.1: construct a data set, which includes noisy data and corresponding denoised data, and the data size is consistent with the size required for network transmission; the data is constructed from real remote sensing image data; Step S3.2: Based on the network constructed in step S2, train and test on the data set and adjust the trainable parameter values; Step S3.3: Deploy the trained network in a real-world scenario, with the input being the collected noisy data and the output being the denoising result.

2. The random noise removal method based on low-rank tensor estimation and deep learning according to claim 1, characterized in that: During the training process of the network, the trainable parameters are the weights and thresholds of each module in each layer. The loss function is used to evaluate the denoising results output by the network and the annotated noise-free data, and the trainable parameter values ​​of the network are adjusted.

3. The random noise removal method based on low-rank tensor estimation and deep learning according to claim 2, characterized in that: The soft threshold operator Use ReLU function.

4. The random noise removal method based on low-rank tensor estimation and deep learning according to claim 3, characterized in that: The network is trained by first initializing the variables and Lagrange multipliers to be updated into tensors with a value of 0; passing the initialized variables and noisy input data into the network, passing through each layer of the network in turn, updating each variable and Lagrange multiplier, taking the denoising result output by the last layer of the network, and passing it and the corresponding noise-free slice data into the selected loss function, and using the back-propagation algorithm of back-propagation deep learning to return the gradient, adjust the trainable parameters, and finally obtain a trained random noise removal model.

5. The random noise removal method based on low-rank tensor estimation and deep learning according to claim 4, characterized in that: The loss function is an L1 loss function, an L2 loss function, or a smoothed L1 loss function.

6. The random noise removal method based on low-rank tensor estimation and deep learning according to claim 5, characterized in that: The specific number of network layers is 6.

Images