Feature noise generalization ability enhancement method based on meta-learning

Abstract

A feature noise generalization ability enhancing method based on meta-learning comprises the steps of searching data features through a torch function, then distributing weights to random noise to form feature noise, and dividing the feature noise into two environments of label replacement and pixel replacement and three different training stages of an early training stage, a middle training stage and a later training stage. Characteristic noise is respectively introduced to carry out loop training, an output generalization enhancement model enables generalization precision to be obviously improved, the accuracy of a meta-model is obviously improved by adding the characteristic noise in the three environments, a traditional loss function is replaced by a bounded function limited to [0, 1], accurate estimation of a target result is achieved, and the accuracy of the meta-model is improved. And when the accuracy does not reach the standard, taking the meta-training set after the feature noise is input into the current cycle training as a new training set to carry out the next cycle training, and finally taking the six feature noise generalization ability enhancement model with the accuracy reaching the standard as an output model, so that the loss of the meta-model is reduced, and the generalization ability of meta learning is effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of generalization capability enhancement methods, and in particular to a feature noise generalization capability enhancement method based on meta-learning. Background Art

[0002] Meta-learning is often described as learning to learn. Meta-learning represents a general methodology: it is expected that the model will learn some meta-knowledge through cross-task training. This meta-knowledge can bring effective information gain to the model, enabling it to quickly converge and generalize on new tasks. The introduction of meta-learning methods provides new solutions to many artificial intelligence problems, such as the small sample problem. When solving the small sample problem, meta-learning transforms the data space into the task space from a training perspective, and learns on multiple tasks in order to obtain meta-knowledge that is beneficial to new tasks.

[0003] However, existing meta-learning algorithms have the following drawbacks: 1. The dataset relies on manual work, requires a lot of manpower and resources, and is prone to incorrect label data.

[0004] 2. It still performs poorly on tasks with small quantity, poor quality, and lack of labeled data.

[0005] 3. It is easy to overfit the wrong labeled data and over-process the unnecessary data, resulting in unsatisfactory generalization performance of the metamodel. Summary of the invention

[0006] The purpose of the present invention is to provide a method for enhancing the generalization ability of feature noise based on meta-learning, which uses feature noise to improve the meta-model, uses a bounded loss function as a criterion to evaluate the generalization ability of the meta-model, reduces the loss of the meta-model, and improves the generalization performance.

[0007] The technical solution adopted by the present invention to solve the above technical problems is: a method for enhancing the generalization ability of feature noise based on meta-learning, comprising the following steps: Step 1: Preprocess the MNIST data set to obtain the original data set; Step 2: Generate training tasks in two environments, label replacement and pixel replacement, using the original data set to obtain two different meta-training sets. The network framework used to permute labels is a convolutional neural network with two convolutional layers, a linear hidden layer, and a linear output layer; The network framework used to permute pixels is a fully connected network with three hidden layers and a linear output layer; Step 3: Select the MLAPM (Meta Learning Adjusting Priors Master) model as the base model, and input the two meta-training sets of replacement labels and replacement pixels into the base model for cyclic training: S31, extracting features from the meta-training set through the torch function, and then combining the extracted data features with random noise according to weight distribution to obtain feature noise; S32, dividing the training process into an early training period, a middle training period and a late training period according to the time sequence, wherein the period before the training cycle starts is the early training period, the period in the training cycle is the middle training period, and the period after the training cycle ends is the late training period; Each meta-training set is repeatedly input into the basic model three times, and feature noise is input into the meta-training set in the early, middle and late stages of training, for three training runs; S33, using a bounded loss function limited to [0,1] to calculate the loss of the training result data, judging the training accuracy according to the calculation result, and when the accuracy reaches the standard, the cycle stops, and the model after the cycle training is used as the feature noise generalization ability enhancement model; When the accuracy does not meet the standard, the meta-training set after the characteristic noise is input into the current cycle training is used as the new training set, and the next cycle training is performed. First, new characteristic noise is obtained through S31, and then the new characteristic noise is input into the new training set through S32 at the same training period as the previous cycle, and finally the loss is recalculated until the accuracy meets the standard; S34. When the cyclic training of the two meta-training sets of replacement labels and replacement pixels in the early, middle and late stages of training are all stopped, the six feature noise generalization capability enhancement models that meet the accuracy standards are used as output models, and the feature noise generalization capability enhancement is completed.

[0008] Preferably, each meta-training set contains 60,000 training sample data.

[0009] Preferably, the torch function downsamples the input feature map by using multiple pooling functions to reduce the size of the output feature data.

[0010] According to the above technical solution, the beneficial effects of the present invention are: The present invention uses torch function to find data features, and then assigns weights to random noise in combination with feature information to form feature noise. Feature noise is introduced in two different environments and in three different training stages to improve and evaluate the generalization ability. The improvement of the generalization enhancement model significantly improves the generalization accuracy. Adding feature noise in the three environments significantly improves the accuracy of the meta-model. A bounded function limited to [0,1] is used to replace the traditional loss function, which is more in line with the limitation that the loss function is almost always less than 1 in practice, so as to achieve accurate estimation of the target result and reduce the loss of the meta-model. Experimental results show that in the permuted label environment, the three training stages are improved by 1.71%, 1.77% and 1.79% respectively, all reaching an accuracy of more than 99%. In the permuted pixel environment, the three training stages are improved by 1.19% accuracy, which effectively improves the generalization ability of meta-learning and provides technical support for information processing. DETAILED DESCRIPTION

[0011] The embodiment provides a method for enhancing the generalization ability of feature noise based on meta-learning, comprising the following steps: Step 1: Preprocess the MNIST data set, remove samples with poor data quality, select data with good quality and enter the task pool for processing to obtain the original data set.

[0012] Step 2: Generate training tasks in two environments, label replacement and pixel replacement, using the original dataset to obtain two different meta-training sets, each containing 60,000 training sample data.

[0013] In the permuted label environment, each task is created by randomly permuting the labels, and the network framework used for permuted labels is a convolutional neural network with two convolutional layers, a linear hidden layer, and a linear output layer.

[0014] In the pixel replacement environment, a limited number of position swaps are performed to ensure that the tasks have reasonable relevance. The network framework used to replace pixels is a fully connected network with three hidden layers and one linear output layer.

[0015] Step 3: Select the MLAPM (Meta Learning Adjusting Priors Master) model as the base model, and input the two meta-training sets of replacement labels and replacement pixels into the base model for cyclic training.

[0016] The following experimental environment and configuration are used to train the model: the operating system is Windows 10, the CPU is Intel (R) Xeon (R) Gold 6326 CPU @ 2.90GHz, 256GB memory, and the GPU model is NVIDIA GeForce RTX 4090, 24GB video memory. The entire experiment uses PyCharm as the development platform, Python 3.8 as the code interpreter, Pytorch 2.0 as the deep learning framework, and CUDA 11.7 to accelerate the calculation speed.

[0017] S31. Extract features from the meta-training set through the torch function. The torch function downsamples the input feature map by using multiple pooling functions to reduce the size of the output feature data. Then combine the extracted data features with random noise according to the weight distribution to obtain feature noise.

[0018] S32, dividing the training process into an early training period, a middle training period and a late training period according to the time sequence, wherein the period before the training cycle starts is the early training period, the period in the training cycle is the middle training period, and the period after the training cycle ends is the late training period.

[0019] Each meta-training set is repeatedly input into the basic model three times, and feature noise is input into the meta-training set in the early, middle and late stages of training, and training is performed three times.

[0020] S33. Use a bounded loss function limited to [0,1] to calculate the loss of the training result data, and judge the training accuracy based on the calculation results. Bounded loss function is a type of loss function used to limit the range of loss values, which can reduce the adverse effects of outliers on the model training process, thereby improving the robustness of the model.

[0021] When the accuracy reaches the target, the cycle stops, and the model after this cycle training is used as the model with enhanced feature noise generalization ability.

[0022] When the accuracy does not meet the standard, the meta-training set after the characteristic noise is input into this cycle training is used as the new training set for the next cycle training. First, the new characteristic noise is obtained through S31, and then the new characteristic noise is input into the new training set through S32 in the same training period as the previous cycle. Finally, the loss is recalculated until the accuracy meets the standard.

[0023] S34. When the cyclic training of the two meta-training sets of replacement labels and replacement pixels in the early, middle and late stages of training are all stopped, the six feature noise generalization capability enhancement models that meet the accuracy standards are used as output models, and the feature noise generalization capability enhancement is completed.

[0024] The detection and counting results of the training experiment show that in the permuted label environment, the three training stages improved by 1.71%, 1.77% and 1.79% respectively, all reaching an accuracy of more than 99%. In the permuted pixel environment, the accuracy of the three training stages was improved by 1.19%. The results show that by using the method of this embodiment, a model that effectively improves the generalization ability of meta-learning can be obtained, which significantly improves the generalization accuracy, has the characteristics of high accuracy, low complexity, and economic efficiency.

Claims

1. A method for enhancing the generalization ability of feature noise based on meta-learning, characterized in that: The following steps are involved: Step 1: Preprocess the MNIST data set to obtain the original data set; Step 2: Generate training tasks in two environments, label replacement and pixel replacement, using the original data set to obtain two different meta-training sets. The network framework used to permute labels is a convolutional neural network with two convolutional layers, a linear hidden layer, and a linear output layer; The network framework used to permute pixels is a fully connected network with three hidden layers and a linear output layer; Step 3: Select the MLAPM (Meta Learning Adjusting Priors Master) model as the base model, and input the two meta-training sets of replacement labels and replacement pixels into the base model for cyclic training: S31, extracting features from the meta-training set through the torch function, and then combining the extracted data features with random noise according to weight distribution to obtain feature noise; S32, dividing the training process into an early training period, a middle training period and a late training period according to the time sequence, wherein the period before the training cycle starts is the early training period, the period in the training cycle is the middle training period, and the period after the training cycle ends is the late training period; Each meta-training set is repeatedly input into the basic model three times, and feature noise is input into the meta-training set in the early, middle and late stages of training, for three training runs; S33, using a bounded loss function limited to [0,1] to calculate the loss of the training result data, judging the training accuracy according to the calculation result, and when the accuracy reaches the standard, the cycle stops, and the model after the cycle training is used as the feature noise generalization ability enhancement model; When the accuracy does not meet the standard, the meta-training set after the characteristic noise is input into the current cycle training is used as the new training set, and the next cycle training is performed. First, new characteristic noise is obtained through S31, and then the new characteristic noise is input into the new training set through S32 at the same training period as the previous cycle, and finally the loss is recalculated until the accuracy meets the standard; S34. When the cyclic training of the two meta-training sets of replacement labels and replacement pixels in the early, middle and late stages of training are all stopped, the six feature noise generalization capability enhancement models that meet the accuracy standards are used as output models, and the feature noise generalization capability enhancement is completed.

2. The method for enhancing the generalization ability of feature noise based on meta-learning according to claim 1, characterized in that: Each meta-training set contains 60,000 training sample data.

3. The method for enhancing the generalization ability of feature noise based on meta-learning according to claim 1, characterized in that: The torch function downsamples the input feature map using a variety of pooling functions to reduce the size of the output feature data.