Training and Denoising Methods for Two-Layer Optimization Models Based on Regularized Outer Gradients

Abstract

This application relates to the field of computer technology, and in particular to a two-layer optimization model training method and denoising method based on regularized external gradients. The method includes acquiring a training sample set and performing a first preset number of iterations of training on the model to be trained based on the training sample set. Each iteration includes multiple iteration update processes. The optimization structure, which considers both global and local objectives through iterative training and iterative update, avoids the non-negligible computational error introduced by random sampling gradient calculation by introducing variance reduction techniques, thereby improving the model training speed and reducing the training cost.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a training method and denoising method for a two-layer optimization model based on regularized external gradients. Background Technology

[0002] Image denoising is an indispensable step in technologies such as object detection and tracking, and is widely used in fields such as biology, medicine, and autonomous driving. Currently, mainstream image denoising methods are based on machine learning techniques. By adjusting the parameters of a machine learning model on a given dataset, the accuracy of the model's predictions or decisions in future tasks can be improved, thereby enhancing its performance in real-world applications.

[0003] However, existing machine learning-based image processing methods generally rely on increasing the number of training samples to improve denoising performance. The model training process requires calculating the gradients of all samples in the high-dimensional data, which not only involves complex and time-consuming calculations that increase training costs but also ultimately raises the overall cost of image denoising.

[0004] Therefore, the existing technology still needs to be improved and enhanced. Summary of the Invention

[0005] The technical problem to be solved by this application is to provide a training method and a denoising method for a two-layer optimization model based on regularized external gradient, which addresses the shortcomings of existing technologies.

[0006] To address the aforementioned technical problems, the first aspect of this application provides a method for training a two-layer optimization model based on regularized outer gradients, wherein the method specifically includes:

[0007] Obtain a training sample set, which includes several training images;

[0008] Based on the training sample set, the model to be trained is subjected to a first preset number of iterations to obtain an image denoising model. The training process for each iteration specifically includes:

[0009] Several training samples selected in this iteration are input into the model to be trained. The model to be trained outputs the prediction result corresponding to each training sample selected in this iteration. The average gradient of the upper objective function is determined based on the prediction result of each training sample selected in this iteration and the preset upper objective function. The average gradient of the lower objective function is determined based on the prediction result of each training sample selected in this iteration and the preset lower objective function.

[0010] The model parameters of the model to be trained are updated iteratively a second preset number of times based on the mean gradient of the upper-layer objective function and the mean gradient of the lower-layer objective function, wherein each iteration update process specifically includes:

[0011] The first regularization mapping is determined based on the mean gradient of the upper objective function and the mean gradient of the lower objective function, and the model parameters of the model to be trained are projected based on the first regularization mapping to obtain intermediate model parameters.

[0012] The mean gradient of the intermediate upper-layer objective function and the mean gradient of the intermediate lower-layer objective function are determined based on the intermediate model parameters.

[0013] The second regularization mapping is determined based on the mean gradient of the intermediate upper-layer objective function and the mean gradient of the intermediate lower-layer objective function, and the second regularization mapping is projected to obtain the iteratively updated model parameters.

[0014] The two-layer optimization model training method based on regularized external gradients, wherein the lower-layer objective function is determined based on the training objective loss function corresponding to the model to be trained, and the upper-layer objective function is determined based on the validation objective loss function corresponding to the model to be trained.

[0015] The bi-layer optimization model training method based on regularized external gradients, wherein determining the mean gradient of the upper-layer objective function based on the prediction results of each training sample selected in this iteration and a preset upper-layer objective function, and determining the mean gradient of the lower-layer objective function based on the prediction results of each training sample selected in this iteration and a preset lower-layer objective function specifically includes:

[0016] Based on the prediction results of each training sample selected in this iteration of training, the gradient of the upper objective function corresponding to each training sample selected in this iteration of training is calculated through the upper objective function, and the average gradient of the upper objective function is calculated based on all the gradients of the upper objective function.

[0017] Based on the prediction results of each training sample selected in this iteration of training, the gradient of the lower-level objective function corresponding to each training sample selected in this iteration of training is calculated through the lower-level objective function, and the mean gradient of the lower-level objective function is calculated based on all the gradients of the lower-level objective function.

[0018] The bi-layer optimization model training method based on regularized external gradients, wherein determining the first regularization mapping based on the mean gradient of the upper-layer objective function and the mean gradient of the lower-layer objective function specifically includes:

[0019] Obtain the regularization parameter corresponding to the model to be trained, and determine the product of the mean gradient of the upper objective function and the regularization parameter;

[0020] The sum of the gradient mean of the lower-level objective function and the product is used as the first regularization mapping.

[0021] The training method for a two-layer optimization model based on regularized outer gradients, wherein the expression for the intermediate model parameters is:

[0022] ,

[0023] in, Indicates intermediate model parameters. Indicates the proximity operator, Indicates the iteration step size. Indicates the first The model parameters are updated in the next iteration. This represents the mean gradient of the lower-level objective function. This represents the mean gradient of the upper-level objective function. Represents the regularization parameter. This represents an auxiliary variable.

[0024] The aforementioned two-layer optimization model training method based on regularized outer gradient, wherein the mean gradient of the intermediate lower-layer objective function and the mean gradient of the intermediate upper-layer objective function are respectively expressed as:

[0025] ,

[0026] ,

[0027] in, This represents the mean gradient of the intermediate lower-level objective function. This represents the mean gradient of the lower-level objective function. This represents the gradient of the lower-level objective function under the model parameters. This represents the gradient of the lower-level objective function under intermediate model parameters. This represents the mean gradient of the intermediate-level objective function. This represents the mean gradient of the upper-level objective function. This represents the gradient of the upper-level objective function under the model parameters. This represents the gradient of the upper-level objective function under the intermediate model parameters. Describes the first randomly selected sample from a set of training samples. Training samples.

[0028] A second aspect of this application provides a denoising method, which uses an image denoising model trained using the bi-layer optimization model training method based on regularized outer gradient as described above. The denoising method specifically includes:

[0029] Obtain the image to be denoised;

[0030] The image to be denoised is input into the image denoising model, and the denoised image corresponding to the image to be denoised is output by the image denoising model.

[0031] A third aspect of this application provides a training device for a two-layer optimization model based on regularized outer gradients, wherein the training device specifically includes:

[0032] The acquisition module is used to acquire the training sample set;

[0033] The training module is used to perform a first preset number of iterations of training on the model to be trained based on the training sample set, wherein the training process of each iteration specifically includes:

[0034] Several training samples selected in this iteration are input into the model to be trained. The model to be trained outputs the prediction result corresponding to each training sample selected in this iteration. The average gradient of the upper objective function is determined based on the prediction result of each training sample selected in this iteration and the preset upper objective function. The average gradient of the lower objective function is determined based on the prediction result of each training sample selected in this iteration and the preset lower objective function.

[0035] The model parameters of the model to be trained are updated iteratively a second preset number of times based on the mean gradient of the upper-layer objective function and the mean gradient of the lower-layer objective function, wherein each iteration update process specifically includes:

[0036] The first regularization mapping is determined based on the mean gradient of the upper objective function and the mean gradient of the lower objective function, and the model parameters of the model to be trained are projected based on the first regularization mapping to obtain intermediate model parameters.

[0037] The mean gradient of the intermediate upper-layer objective function and the mean gradient of the intermediate lower-layer objective function are determined based on the intermediate model parameters.

[0038] The second regularization mapping is determined based on the mean gradient of the intermediate upper-layer objective function and the mean gradient of the intermediate lower-layer objective function, and the second regularization mapping is projected to obtain the iteratively updated model parameters.

[0039] A fourth aspect of this application provides a computer-readable storage medium storing one or more programs that can be executed by one or more processors to implement the steps in the bilayer optimization model training method based on regularized external gradients as described above.

[0040] A fifth aspect of this application provides a terminal comprising: a processor and a memory;

[0041] The memory stores a computer-readable program that can be executed by the processor;

[0042] When the processor executes the computer-readable program, it implements the steps in any of the above-described methods for training a two-layer optimization model based on regularized external gradients.

[0043] Beneficial effects: Compared with the prior art, this application provides a training method and denoising method for a two-layer optimization model based on regularized external gradient. The training method for the two-layer optimization model based on regularized external gradient includes obtaining a training sample set and performing a first preset number of iterations of training on the model to be trained based on the training sample set. Each iteration includes multiple iteration update processes. The optimization structure of iterative training and iterative update considers both global and local objectives. By introducing variance reduction technology, the non-negligible calculation error introduced by random sampling gradient calculation is avoided, thereby improving the training speed of the model and reducing the training cost of model training. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0045] Figure 1 A flowchart illustrating the training method for a two-layer optimization model based on regularized external gradients provided in this application embodiment.

[0046] Figure 2 A comparison diagram of the lower-level target error between the method provided in the embodiments of this application and existing methods.

[0047] Figure 3 A comparison diagram of the upper-level target error between the method provided in the embodiments of this application and existing methods.

[0048] Figure 4 This is a schematic diagram of the principle of a two-layer optimization model training device based on regularized external gradients provided in an embodiment of this application.

[0049] Figure 5 A schematic diagram of the terminal provided in the embodiments of this application. Detailed Implementation

[0050] This application provides a training method and denoising method for a two-layer optimization model based on regularized external gradients. To make the objectives, technical solutions, and effects of this application clearer and more explicit, the following detailed description is provided with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only for explaining this application and are not intended to limit this application.

[0051] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this application means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or wireless coupling. The term “and / or” as used herein includes all or any units and all combinations of one or more associated listed items.

[0052] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0053] It should be understood that the sequence number and size of each step in this embodiment do not imply the order of execution. The execution order of each process is determined by its function and internal logic, and should not constitute any limitation on the implementation process of this application embodiment.

[0054] Research has shown that model training refers to the process of adjusting model parameters on a given dataset to enable the model to make accurate predictions or decisions in future tasks, thereby improving its performance in real-world applications. However, current model training methods for machine learning often involve high-dimensional optimization problems. Calculating the gradients of all samples in the model requires significant time and computational resources, while random sampling for gradient calculation introduces non-negligible computational errors, slowing down convergence and consequently affecting the model's training speed, thus increasing the training cost.

[0055] To address the aforementioned issues, this embodiment transforms the model parameter optimization process during model training into a two-layer optimization process. By introducing an upper-layer objective function and a lower-layer objective function, global and local objectives are considered. In the two-layer optimization process, the mean gradient values ​​of the upper-layer and lower-layer objective functions are determined in each training iteration. Then, based on these mean gradient values, the model parameters of the model to be trained are updated for a second preset number of iterations. Furthermore, variance reduction techniques are introduced during the iteration process to avoid the non-negligible computational errors introduced by random sampling gradient calculation, thereby improving the model training speed and reducing the training cost.

[0056] The application content will be further explained below with reference to the accompanying drawings and the description of the embodiments.

[0057] This embodiment provides a training method for a two-layer optimization model based on regularized external gradients, such as... Figure 1 As shown, the method includes:

[0058] S10. Obtain the training sample set;

[0059] S20. Based on the training sample set, perform a first preset number of iterations to train the model to be trained.

[0060] Specifically, the training sample set includes several training images, each of which is a training sample. In the first preset number of training iterations, a fixed number of training samples are selected from the training sample set as the training samples used in that iteration. The training samples used in each iteration can be randomly selected from the training sample set, or the training sample set can be pre-divided into several training batches, each containing a fixed number of batches, with one batch selected as the training sample for each iteration. Furthermore, before training the two-layer optimization model based on regularized external gradients, the parameters required for model training are acquired and set. These parameters may include the number of outer loop iterations (i.e., the first preset number), the number of inner loop iterations (i.e., the second preset number), the step size, the regularization parameter, and the initialization optimization variables.

[0061] Each training iteration includes a second preset number of iterations for updating. That is, when training the model to be trained based on the training sample set, the training process is divided into an outer loop and an inner loop. The outer loop consists of a first preset number of training iterations, and the inner loop consists of a second preset number of iterations for updating. The inner loop is embedded within each iteration of the outer loop. Both the outer and inner loops introduce upper-level and lower-level objective functions. Using the mean gradients of the upper and lower-level objective functions, the model parameters are optimized through variance-reduced stochastic gradient updates. This ensures convergence of gradient estimation under uncertain conditions while maintaining computational efficiency, thereby improving the training speed and reducing the training cost.

[0062] Furthermore, the model training process is transformed into a stochastic bilevel variational inequality problem. This problem is solved using a combination of outer and inner loops to obtain the trained model parameters. The stochastic bilevel variational inequality problem can be categorized as follows:

[0063] ,

[0064] ,

[0065] in, Let represent the solution set of a stochastic two-level variational inequality problem. , Is The optimal solution (i.e., the parameters of the model to be optimized) is found in the 3D real vector space. , Is Any vector in a 3D real vector space is used to find an optimal solution. This makes it applicable to all possible All satisfy the inequality conditions. This represents a random sample. This represents the lower-level objective function. This represents the gradient of the lower-level objective function, which depends on... and random sampling samples , This represents the expected value. This represents the gradient of the upper-level objective function, which depends on the randomly sampled samples. , This represents the solution set of the lower-level stochastic variational inequality problem.

[0066] For example, let's illustrate the training method for the above-mentioned two-layer optimization model based on regularized external gradient by using the model to be trained as the image denoising model. The goal of the image denoising model is to find a set of model parameters. This makes it possible to use the training dataset Losses To minimize, we use the most common least squares loss function, which is:

[0067] ,

[0068] in, Represents a noisy image. This represents the clear image corresponding to the noisy image. This represents the model parameters of the model to be trained. During the optimization process of the model, regularization helps maintain its sparsity, meaning it only uses the most important features. Through such regularization, the model can not only remove noise but also select the most important image features.

[0069] However, not all optimal solutions perform the same way on other datasets. Therefore, this application introduces a higher-level objective function, namely, a validation dataset. loss function on The upper-level objective function helps select the minimizer of the training loss, resulting in good performance on both the training and validation sets. Therefore, the training process of the image denoising model is transformed into a two-layer problem, which can be represented as:

[0070] ,

[0071] ,

[0072] ,

[0073] in, Represents the model parameter space. This represents a verification image. This indicates the clear image corresponding to the verification image.

[0074] This application's embodiments transform the model training process into a two-layer problem, which not only improves the model's training speed but also ensures that the denoising model performs well not only on the training set but also has good generalization ability on new data by optimizing parameters through the validation set.

[0075] The iterative training (i.e., the outer loop process) and iterative update (i.e., the inner loop process) are described below to illustrate the specific process by which the embodiments of this application achieve model training by solving a two-layer problem.

[0076] During iterative training, several training samples are randomly selected from the training sample set for training. Then, the model to be trained is iterated once based on these training samples. The iterative training process specifically includes:

[0077] Several training samples selected in this iteration are input into the model to be trained. The model to be trained outputs the prediction result corresponding to each training sample selected in this iteration. The average gradient of the upper objective function is determined based on the prediction result of each training sample selected in this iteration and the preset upper objective function. The average gradient of the lower objective function is determined based on the prediction result of each training sample selected in this iteration and the preset lower objective function.

[0078] The model parameters of the model to be trained are updated iteratively a second preset number of times based on the mean gradient of the upper objective function and the mean gradient of the lower objective function.

[0079] Specifically, the mean gradient of the upper-layer objective function is determined based on the upper-layer objective function, and the mean gradient of the lower-layer objective function is determined based on the lower-layer objective function. The lower-layer objective function is determined based on the training objective loss function corresponding to the model to be trained, and the upper-layer objective function is determined based on the validation objective loss function corresponding to the model to be trained. In other words, this application treats the training process as a local objective and the validation process as a global objective. Then, it determines the lower-layer objective function based on the training set loss function used in the training process and the upper-layer objective function based on the validation set loss function corresponding to the validation process. For example, minimizing the training objective loss function can be used as the lower-layer objective function, and minimizing the validation objective loss function can be used as the upper-layer objective function. The training loss function and validation loss function can be determined according to the training objective of the model to be trained. For example, when the training objective of the model is classification, the cross-loss function can be used as both the training loss function and the validation loss function.

[0080] For example, determining the mean gradient of the upper-level objective function based on the prediction result of each training sample selected in this iteration of training and the preset upper-level objective function, and determining the mean gradient of the lower-level objective function based on the prediction result of each training sample selected in this iteration of training and the preset lower-level objective function specifically includes:

[0081] Based on the prediction results of each training sample selected in this iteration of training, the gradient of the upper objective function corresponding to each training sample selected in this iteration of training is calculated through the upper objective function, and the average gradient of the upper objective function is calculated based on all the gradients of the upper objective function.

[0082] Based on the prediction results of each training sample selected in this iteration of training, the gradient of the lower-level objective function corresponding to each training sample selected in this iteration of training is calculated through the lower-level objective function, and the mean gradient of the lower-level objective function is calculated based on all the gradients of the lower-level objective function.

[0083] Specifically, the prediction result is the prediction content formed by the model to be trained based on the training samples. After obtaining the prediction result, the gradients of the upper-layer objective function and the lower-layer objective function can be determined based on the prediction result. Specifically, when determining the gradients of the upper-layer and lower-layer objective functions based solely on the prediction result, or based on the prediction result and the corresponding annotation results of the training samples, the gradients can be determined according to the actual needs of the model to be trained. The mean gradients of the upper-layer and lower-layer objective functions can be expressed as follows:

[0084] ,

[0085] ,

[0086] in, This represents the mean gradient of the lower-level objective function. This represents the mean gradient of the upper-level objective function. Represents an auxiliary variable, the first In the next iteration of training The value is the first Model parameters after iterative training Indicates the first The number of training samples corresponding to each iteration of training. Indicates the first The first iteration of training The gradient of the lower-level objective function corresponding to each training sample Indicates the first The first iteration of training Random variables in training samples Indicates the first The first iteration of training The average gradient of the upper-level objective function corresponding to each training sample.

[0087] Furthermore, to balance computational efficiency and accuracy, the sampling rate of each training iteration can be adjusted online to ensure gradient estimation convergence, thereby replacing complex probability distributions and gradient calculations. Based on this, in one implementation, the iterative update process specifically may include:

[0088] The first regularization mapping is determined based on the mean gradient of the upper objective function and the mean gradient of the lower objective function, and the model parameters of the model to be trained are projected based on the first regularization mapping to obtain intermediate model parameters.

[0089] The mean gradient of the intermediate upper-layer objective function and the mean gradient of the intermediate lower-layer objective function are determined based on the intermediate model parameters.

[0090] The second regularization mapping is determined based on the mean gradient of the intermediate upper-layer objective function and the mean gradient of the intermediate lower-layer objective function, and the second regularization mapping is projected to obtain the iteratively updated model parameters.

[0091] Specifically, the first regularization map combines the mean gradients of the upper and lower objective functions, while the second regularization map combines the mean gradients of the intermediate upper and lower objective functions. During projection, both the first and second regularization maps are projected separately to introduce the external gradients into the iterative update process, reducing the dependence on the monotonicity of the upper objective function and thus allowing for better acquisition of the iteratively updated model parameters.

[0092] The first regularization mapping can be a regularization mapping, which can be represented as:

[0093] ,

[0094] in, This represents the output of the first regularization mapping. This represents the regularization parameter.

[0095] After obtaining the first regularization map, intermediate model parameters are obtained by projecting the first regularization map. The expression for the intermediate model parameters is as follows:

[0096] ,

[0097] in, Indicates intermediate model parameters. Indicates the proximity operator, Indicates the iteration step size. Indicates the first The model parameters are updated in the next iteration. This represents the mean gradient of the upper-level objective function. This represents the mean gradient of the lower-level objective function. This represents the regularization parameter.

[0098] After obtaining the intermediate model parameters, the mean gradients of the intermediate upper-layer objective function and the intermediate lower-layer objective function are determined based on these parameters. The process for determining these mean gradients can be as follows: Randomly select several target training samples from all training samples used in the iterative training process. Obtain the gradients of the upper-layer and lower-layer objective functions for each target training sample at the given model parameters, as well as the gradients of the intermediate upper-layer and intermediate lower-layer objective functions for each training sample selected in this iteration when the model parameters are the intermediate model parameters. Then, determine the mean gradient of the intermediate upper-layer objective function based on the mean gradient of the upper-layer objective function, the gradient of the upper-layer objective function for each target training sample, and the gradient of the intermediate upper-layer objective function for each target training sample. Similarly, determine the mean gradient of the intermediate lower-layer objective function based on the mean gradient of the lower-layer objective function, the gradient of the lower-layer objective function for each target training sample, and the gradient of the intermediate lower-layer objective function for each target training sample.

[0099] In one specific implementation, several target training samples are combined into one target training sample, denoted as training sample. Then, the mean gradient of the intermediate upper-level objective function and the mean gradient of the intermediate upper-level objective function are respectively expressed as:

[0100] ,

[0101] ,

[0102] in, This represents the mean gradient of the intermediate lower-level objective function. This represents the mean gradient of the lower-level objective function. This represents the gradient of the lower-level objective function under the model parameters. This represents the gradient of the lower-level objective function under intermediate model parameters. This represents the mean gradient of the intermediate-level objective function. This represents the mean gradient of the upper-level objective function. This represents the gradient of the upper-level objective function under the model parameters. This represents the gradient of the upper-level objective function under the intermediate model parameters. Describes the first randomly selected sample from a set of training samples. Training samples.

[0103] Furthermore, the second regularization mapping is constructed based on the mean gradient of the intermediate upper-layer objective function and the mean gradient of the intermediate lower-layer objective function, and the construction method of the second regularization mapping is the same as that of the first regularization mapping. That is, the second regularization mapping can be expressed as:

[0104] ,

[0105] in, This represents the output of the second regularization mapping. This represents the regularization parameter.

[0106] After constructing the second regularization map, projecting the second regularization map yields the updated model parameters for this iteration, which are represented as follows:

[0107] ,

[0108] in, Indicates the first The model parameters updated in the previous iteration can be used to determine the model parameters of the next iteration. The model parameters after the nth iteration of training, where the nth The model parameters after the next iteration of training can be expressed as:

[0109] ,

[0110] in, Indicates the first Model parameters after one iteration of training Indicates the second preset quantity. This represents intermediate model parameters.

[0111] Of course, in practical applications, other methods can also be used to determine the first... The model parameters after the last iteration of training, for example, can be directly used as the model parameters updated in the last iteration. Model parameters after each iteration of training, etc. Furthermore, in practical applications, other projection methods can be used, such as directly projecting the model parameters to the iteratively updated model parameters through the first mapping, without performing external gradient calculations, etc.

[0112] It should be noted that after completing all iterations (i.e., completing the inner loop), the model parameters obtained from the last iteration (completing the inner loop) are stored, and the model parameters obtained from this inner loop are used as auxiliary variables for the next outer loop. , express Auxiliary variables for each round of iterative training (i.e., the outer loop). express Once the model parameters are determined in the inner loop of the round of iterative training (i.e., the outer loop), the current outer loop (i.e., the current iterative training) ends, and the next iterative training begins.

[0113] In summary, this embodiment provides a two-layer optimization model training method based on regularized external gradients. The method includes acquiring a training sample set and performing a first preset number of iterations of training on the model to be trained based on the training sample set. Each iteration includes multiple iteration update processes. The optimization structure of iterative training and iterative update considers both global and local objectives. Furthermore, by introducing variance reduction techniques, the non-negligible computational error introduced by random sampling gradient calculation is avoided, thereby improving the training speed of the model and reducing the training cost of the model.

[0114] To illustrate the effectiveness of the bi-layer optimization model training method based on regularized external gradients provided in this application, a specific example is given here.

[0115] Example: Using a mathematical foundation dataset from a website, which consists of a data matrix Composition, which includes =1068 samples, representing 1068 noisy images, and =731 feature vectors, representing image feature points. One column is randomly selected as the output vector. The rest are input sample matrices. Use 60% of the dataset as the training set. 20% as validation set The remaining 20% ​​was used as a test set. At this point, since the number of feature vectors d is greater than the number of samples in the training set, the regression problem is over-parameterized.

[0116] The constraint set is Using test error As a performance indicator, the CVX function is used to solve the lower-level problem, and the two-level problem is reconstructed to find the optimal value of the lower-level problem. and the optimal value of the higher-level problem .

[0117] In this example, constraint parameters are set. The first and second preset quantities are both 15,000, and the running step size is... Learning rate The initial point is a randomly generated sparse vector. The performance of the SVR-R-EG method provided in this application embodiment was compared with the SBCGI algorithm, SBCGF algorithm, aR-IP-SeG algorithm, and the DBGD-Sto algorithm in the random case. The comparison results are as follows: Figure 2 , Figure 3 As shown.

[0118] exist Figure 2 and Figure 3In the results, it can be observed that the SVR-R-EG algorithm achieves smaller upper-level and lower-level target errors than other algorithms within the same number of iterations, and its convergence speed is faster. The SCGBF and SCGBI algorithms perform worse in terms of lower-level and upper-level target errors. The DBGD-Sto and aR-IP-SeG algorithms perform poorly in terms of both lower-level and upper-level target errors.

[0119] Based on the above-described training method for a two-layer optimization model based on regularized outer gradient, this embodiment provides a denoising method. The image denoising model trained using the above-described training method for a two-layer optimization model based on regularized outer gradient specifically includes:

[0120] Obtain the image to be denoised;

[0121] The image to be denoised is input into the image denoising model, and the denoised image corresponding to the image to be denoised is output by the image denoising model.

[0122] Based on the above-described training method for a two-layer optimization model based on regularized outer gradients, this embodiment provides a training device for a two-layer optimization model based on regularized outer gradients, such as... Figure 4 As shown, the two-layer optimization model training device based on regularized outer gradient specifically includes:

[0123] The acquisition module 100 is used to acquire a training sample set, which includes several training images.

[0124] Training module 200 is used to perform a first preset number of iterations of training on the model to be trained based on the training sample set to obtain an image denoising model, wherein the training process of each iteration specifically includes:

[0125] Several training samples selected in this iteration are input into the model to be trained. The model to be trained outputs the prediction result corresponding to each training sample selected in this iteration. The average gradient of the upper objective function is determined based on the prediction result of each training sample selected in this iteration and the preset upper objective function. The average gradient of the lower objective function is determined based on the prediction result of each training sample selected in this iteration and the preset lower objective function.

[0126] The model parameters of the model to be trained are updated iteratively a second preset number of times based on the mean gradient of the upper-layer objective function and the mean gradient of the lower-layer objective function, wherein each iteration update process specifically includes:

[0127] The first regularization mapping is determined based on the mean gradient of the upper objective function and the mean gradient of the lower objective function, and the model parameters of the model to be trained are projected based on the first regularization mapping to obtain intermediate model parameters.

[0128] The mean gradient of the intermediate upper-layer objective function and the mean gradient of the intermediate lower-layer objective function are determined based on the intermediate model parameters.

[0129] The second regularization mapping is determined based on the mean gradient of the intermediate upper-layer objective function and the mean gradient of the intermediate lower-layer objective function, and the second regularization mapping is projected to obtain the iteratively updated model parameters.

[0130] Based on the above-described training method for a two-layer optimization model based on regularized external gradients, this embodiment provides a computer-readable storage medium storing one or more programs that can be executed by one or more processors to implement the steps in the training method for a two-layer optimization model based on regularized external gradients as described in the above embodiment.

[0131] Based on the above-mentioned training method for a two-layer optimization model based on regularized external gradients, this application also provides a terminal, such as... Figure 5 As shown, it includes at least one processor 20; a display screen 21; and a memory 22, and may also include a communications interface 23 and a bus 24. The processor 20, display screen 21, memory 22, and communications interface 23 can communicate with each other via the bus 24. The display screen 21 is configured to display a preset user guide interface in the initial setup mode. The communications interface 23 can transmit information. The processor 20 can invoke logical instructions in the memory 22 to execute the methods described in the above embodiments.

[0132] Furthermore, the logical instructions in the aforementioned memory 22 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium.

[0133] The memory 22, as a computer-readable storage medium, can be configured to store software programs, computer-executable programs, such as program instructions or modules corresponding to the methods in the embodiments of this disclosure. The processor 20 executes functional applications and data processing by running the software programs, instructions, or modules stored in the memory 22, thereby implementing the methods in the above embodiments.

[0134] The memory 22 may include a program storage area and a data storage area. The program storage area may store the operating system and application programs required for at least one function; the data storage area may store data created based on the use of the terminal. Furthermore, the memory 22 may include high-speed random access memory (RAM) and non-volatile memory. Examples include various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks; these can also be transient storage media.

[0135] Furthermore, the specific process of loading and executing multiple instruction processors in the aforementioned storage medium and terminal has been described in detail in the above method, and will not be repeated here.

[0136] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A training method for a two-layer optimization model based on regularized outer gradient, characterized in that, The aforementioned two-layer optimization model training method based on regularized outer gradient specifically includes: Obtain a training sample set, which includes several training images; Based on the training sample set, the model to be trained is subjected to a first preset number of iterations to obtain an image denoising model. The training process for each iteration specifically includes: Several training samples selected in this iteration are input into the model to be trained. The model to be trained outputs the prediction result corresponding to each training sample selected in this iteration. The average gradient of the upper-level objective function is determined based on the prediction result of each training sample selected in this iteration and the preset upper-level objective function. The average gradient of the lower-level objective function is determined based on the prediction result of each training sample selected in this iteration and the preset lower-level objective function. The lower-level objective function is determined based on the training objective loss function corresponding to the model to be trained, and the upper-level objective function is determined based on the validation objective loss function corresponding to the model to be trained. The model parameters of the model to be trained are updated iteratively a second preset number of times based on the mean gradient of the upper-layer objective function and the mean gradient of the lower-layer objective function, wherein each iteration update process specifically includes: The first regularization mapping is determined based on the mean gradient of the upper objective function and the mean gradient of the lower objective function, and the model parameters of the model to be trained are projected based on the first regularization mapping to obtain intermediate model parameters. The mean gradient of the intermediate upper-layer objective function and the mean gradient of the intermediate lower-layer objective function are determined based on the intermediate model parameters. The second regularization mapping is determined based on the mean gradient of the intermediate upper-layer objective function and the mean gradient of the intermediate lower-layer objective function, and the second regularization mapping is projected to obtain the iteratively updated model parameters.

2. The training method for a two-layer optimization model based on regularized external gradients according to claim 1, characterized in that, The determination of the mean gradient of the upper-level objective function based on the prediction results of each training sample selected in this iteration of training and the preset upper-level objective function, and the determination of the mean gradient of the lower-level objective function based on the prediction results of each training sample selected in this iteration of training and the preset lower-level objective function specifically include: Based on the prediction results of each training sample selected in this iteration of training, the gradient of the upper objective function corresponding to each training sample selected in this iteration of training is calculated through the upper objective function, and the average gradient of the upper objective function is calculated based on all the gradients of the upper objective function. Based on the prediction results of each training sample selected in this iteration of training, the gradient of the lower-level objective function corresponding to each training sample selected in this iteration of training is calculated through the lower-level objective function, and the mean gradient of the lower-level objective function is calculated based on all the gradients of the lower-level objective function.

3. The training method for a two-layer optimization model based on regularized external gradients according to claim 1, characterized in that, Determining the first regularization mapping based on the mean gradient of the upper-level objective function and the mean gradient of the lower-level objective function specifically includes: Obtain the regularization parameter corresponding to the model to be trained, and determine the product of the mean gradient of the upper objective function and the regularization parameter; The sum of the gradient mean of the lower-level objective function and the product is used as the first regularization mapping.

4. The training method for a two-layer optimization model based on regularized external gradient according to claim 1 or 3, characterized in that, The expression for the intermediate model parameters is: ， in, Indicates intermediate model parameters. Indicates the proximity operator, Indicates the iteration step size. Indicates the first The model parameters are updated in the next iteration. This represents the mean gradient of the lower-level objective function. This represents the mean gradient of the upper-level objective function. Represents the regularization parameter. This represents an auxiliary variable.

5. The training method for a two-layer optimization model based on regularized external gradient according to claim 1, characterized in that, The mean gradient of the intermediate lower-layer objective function and the mean gradient of the intermediate upper-layer objective function are respectively expressed as: ， ， in, This represents the mean gradient of the intermediate lower-level objective function. This represents the mean gradient of the lower-level objective function. This represents the gradient of the lower-level objective function under the model parameters. This represents the gradient of the lower-level objective function under intermediate model parameters. This represents the mean gradient of the intermediate-level objective function. This represents the mean gradient of the upper-level objective function. This represents the gradient of the upper-level objective function under the model parameters. This represents the gradient of the upper-level objective function under the intermediate model parameters. Describes the first randomly selected sample from a set of training samples. Training samples.

6. A noise reduction method, characterized in that, The image denoising model trained using the bi-layer optimization model training method based on regularized outer gradient as described in any one of claims 1-5, wherein the denoising method specifically includes: Obtain the image to be denoised; The image to be denoised is input into the image denoising model, and the denoised image corresponding to the image to be denoised is output by the image denoising model.

7. A training device for a two-layer optimization model based on regularized external gradients, characterized in that, The aforementioned two-layer optimization model training device based on regularized outer gradient specifically includes: The acquisition module is used to acquire the training sample set; The training module is used to perform a first preset number of iterations of training on the model to be trained based on the training sample set, wherein the training process of each iteration specifically includes: Several training samples selected in this iteration are input into the model to be trained. The model to be trained outputs the prediction result corresponding to each training sample selected in this iteration. The mean gradient of the upper-level objective function is determined based on the prediction result of each training sample selected in this iteration and the preset upper-level objective function. The mean gradient of the lower-level objective function is determined based on the prediction result of each training sample selected in this iteration and the preset lower-level objective function. The lower-level objective function is determined based on the training objective loss function corresponding to the model to be trained, and the upper-level objective function is determined based on the validation objective loss function corresponding to the model to be trained. The model parameters of the model to be trained are updated iteratively a second preset number of times based on the mean gradient of the upper-layer objective function and the mean gradient of the lower-layer objective function, wherein each iteration update process specifically includes: The first regularization mapping is determined based on the mean gradient of the upper objective function and the mean gradient of the lower objective function, and the model parameters of the model to be trained are projected based on the first regularization mapping to obtain intermediate model parameters. The mean gradient of the intermediate upper-layer objective function and the mean gradient of the intermediate lower-layer objective function are determined based on the intermediate model parameters. The second regularization mapping is determined based on the mean gradient of the intermediate upper-layer objective function and the mean gradient of the intermediate lower-layer objective function, and the second regularization mapping is projected to obtain the iteratively updated model parameters.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores one or more programs, which can be executed by one or more processors to implement the steps in the bilayer optimization model training method based on regularized external gradients as described in any one of claims 1-5.

9. A terminal, characterized in that, include: Processor and memory; The memory stores a computer-readable program that can be executed by the processor; When the processor executes the computer-readable program, it implements the steps in the bilayer optimization model training method based on regularized external gradients as described in any one of claims 1-5.

Images