A multi-stage adjustment data augmentation method for model training

Abstract

The present invention discloses a multi-stage adjustment data augmentation method for model training, which includes five modules: an information preprocessing module, a discriminator module, a training preference calculation module, a class information amplification module, and a training set update module. The steps of this method include the establishment of an online class information library and the pre-training of the model, model training and discrimination, calculation of phased training preferences, amplification of class information, and training set update and model continuous training. The present invention optimizes the data augmentation method based on the model training preferences, optimizing the problems of most current data augmentation with manually set parameters and artificial matching, making the matching between data augmentation and the model more objective. At the same time, a discriminator is introduced in the training of the model, and the training is divided into multiple stages according to the training situation of the model. The class information amount of the training set is adjusted multiple times with phased training preferences, forcing the model to pay more attention to the learning of certain classes in the next training, making the matching between the data augmentation algorithm and the model more perfect.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and particularly to a data augmentation method for model training. Background Art

[0002] Object detection is one of the most active fields in the field of deep learning, and the dataset plays an important role in this process. The model learns from the dataset to enable itself to have the ability to distinguish external things. However, due to the influence of objective environmental factors and subjective factors of researchers, dataset production is a complex and resource-consuming task. During the dataset sampling process, it is easy to generate the problem of class imbalance, and even the long-tailed distribution of classes appears.

[0003] The applicant submitted a patent application to the Chinese Patent Office on June 8, 2022, with the publication number CN114898162A. This method adds an adaptive algorithm to the original copy-paste augmentation method, making the training of the model orderly and improving the recognition performance of the model. In subsequent practice, the applicant found that this method is usually optimized based on the deficiencies of the dataset itself, considering the needs of the model for the augmentation algorithm, and the matching between the data augmentation algorithm and the model has not reached the optimal. This application is a further improvement of the above application. Summary of the Invention

[0004] In order to overcome the deficiencies of the prior art, the present invention provides a multi-stage adjustment data augmentation method for model training.

[0005] The technical solution adopted by the present invention to solve its technical problems is:

[0006] A multi-stage adjustment data augmentation method for model training includes five modules: an information preprocessing module, a discriminator module, a training preference calculation module, a category information amplification module, and a training set update module.

[0007] The steps of the multi-stage adjustment data augmentation method for model training are as follows:

[0008] Step 1: Establishment of an online category information library and pre-training of the model

[0009] In the information preprocessing module, an online category information library is established according to the label file, and the model is pre-trained.

[0010] The online category information library contains the semantic information of all categories in the training set, and the semantic information of these categories is collected and classified through the corresponding label files; pre-training means that before formal training, the model uses the original dataset for training, and through pre-training, the evaluation results of the model on the original dataset are obtained, and then the training preference of the model in pre-training is calculated, and the original training set is preliminarily optimized according to the training preference.

[0011] Step 2. Model Training and Discrimination

[0012] In the discriminator module, the model starts formal training using the training set generated in the information preprocessing module. During the training process, the model training may be unstable, and there may be different training preferences at different training times. According to the training situation of the model, the model is divided into multiple stages. The discriminator module will monitor the training situation of the model in real time, monitor and discriminate each stage. When the slope of the AP drops significantly, the discriminator will pause the training, check the categories with unstable phenomena, and then use this as the first stage. Calculate the training preference of this stage using the evaluation results, extract the corresponding evaluation results, and prepare for calculating the training preference of the current stage later.

[0013] Step 3. Calculation of Stage Training Preferences

[0014] During formal training, before the next training stage starts, the training preference calculation module extracts the evaluation results of the model in the current training stage and quantifies the training preference of the current stage. The calculation of the training preference requires the mAP of this stage and the AP of the categories. The training preference is obtained based on the evaluation results of this stage, and its detailed expression is as follows:

[0015] ;

[0016] In the above formula, is the training situation of the category in the th training stage, is the evaluation result of each category, is the serial number corresponding to each category, represents the evaluation result of the first epoch in this stage, represents the evaluation result of the last epoch in this stage, is the adjustment coefficient, and by adjusting to change the number of categories to be amplified, is the enhancement coefficient, and the expression of the enhancement coefficient is as follows:

[0017] .

[0018] Step 4. Amplification of Category Information

[0019] In the category information amplification module, the quantified training preference is weighted and combined with the sample number of the category to generate a set of amplified numbers for the category.

[0020] The amplification method is to use data augmentation strategies to calculate the model's requirements during training using the training preference of the model, that is, the optimal value of the number of samples to be amplified in the current stage. Its expression is as follows:

[0021] ;

[0022] In the above formula, is the optimal value of the number of samples to be amplified in the current stage, is the number of samples of the category in the original training set, is to determine whether to amplify the number of samples for each category, The expression is as follows:

[0023] .

[0024] Finally, the final output result of the category is calculated using the normalization method, and the formula is as follows:

[0025] ;

[0026] In the above formula, is the normalization range, is the final output result of the category, is set of.

[0027] Step Five: Training Set Update and Model Continued Training

[0028] In the training set update module, the update object is the label information in the training set. The newly amplified category information data is mixed with the original training set to form a new training set, and the model starts the training of the next stage with the new training set.

[0029] The beneficial effects of the present invention are as follows: The present invention optimizes the data augmentation method based on the training preference of the model, optimizes the problems of most current data augmentation with manual parameter setting and manual matching, makes the matching between data augmentation and the model more objective. At the same time, a discriminator is introduced in the training of the model, and the training is divided into multiple stages according to the training situation of the model. The category information volume of the training set is adjusted multiple times with the training preference of each stage, forcing the model to pay more attention to the learning of certain categories in the next training, solving the deficiencies in the patent application document with the publication number CN 114898162A submitted by the applicant, and making the matching between the data augmentation algorithm and the model more perfect. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The present invention will be further described below in conjunction with the drawings and embodiments.

[0031] Figure 1 is the step flowchart of the present invention;

[0032] Figure 2 is the flowchart of the embodiment. Embodiment

[0033] Refer toFigure 1 , a multi-stage adjustment data augmentation method for model training, which improves the class imbalance problem in the dataset through an adaptive copy-paste augmentation method, makes the feature distribution of the dataset more suitable for model training, and thus achieves the purpose of improving model performance; the multi-stage adjustment data augmentation method includes five modules: an information preprocessing module, a discriminator module, a training preference calculation module, a class information amplification module, and a training set update module.

[0034] The steps of the multi-stage adjustment data augmentation method for model training are as follows:

[0035] Step 1: Establishment of an online class information library and pre-training of the model

[0036] In the information preprocessing module, an online class information library is established according to the label file, and the model is pre-trained.

[0037] The purpose of establishing the online class information library is to reduce the time spent in each training set update and simplify the process of updating the training set. The online class information library contains the semantic information of all classes in the training set. The semantic information of these classes is collected and classified through the corresponding label files. Some of the class information in the online class information library can also be obtained through online collection. Pre-training means that before formal training, the model is trained using the original dataset (for the pre-training method, please refer to the patent document CN114898162A). The training volume of pre-training is not fixed. Through pre-training, the evaluation results of the model on the original dataset are obtained, and then the training preference of the model in pre-training is calculated. According to the training preference, the original training set is preliminarily optimized to relieve the long-tail effect of classes in the training set and reduce the impact of class imbalance in training. When updating the training set each time, the model only needs to extract the corresponding amount of information from the online class information library.

[0038] The training preference of the model reflects the training needs of the model. For model training, the larger the number of samples of a class in the dataset does not mean that the AP (average precision) of this class is higher. For some classes with relatively few samples, they have a relatively high AP, while for classes with a large sample content in the dataset, the model may not be able to accurately detect this class. Different structures of models are good at detecting different classes. This phenomenon is called the training preference of the model. The difference in training preferences between models is related to the structure of the models themselves.

[0039] Step 2: Model training and discrimination

[0040] In the discriminator module, the model starts formal training using the training set generated in the information preprocessing module. During the training process, the training of the model may be unstable, and there may be different training preferences at different training times. According to the training situation of the model, the model is divided into multiple stages. The discriminator module will monitor the training situation of the model in real time, monitor and discriminate each stage. When the slope of the AP drops significantly, the discriminator will pause the training, check the categories with unstable phenomena, and then use this as the first stage, calculate the training preference of this stage using the evaluation results, extract the corresponding evaluation results, and prepare for calculating the training preference of the current stage in the follow-up.

[0041] Step 3: Calculation of stage-based training preference

[0042] During formal training, before the start of the next training stage, the training preference calculation module extracts the evaluation results of the model in the current training stage and quantifies the training preference of the current stage. The calculation of the training preference mainly uses the mAP of this stage and the AP of each category (AP and mAP are evaluation methods in the image classification task and belong to the prior art). The training preference is obtained based on the evaluation results of this stage, and its detailed expression is as follows:

[0043] ;

[0044] In the above formula, is the training situation of the category in the th training stage, is the evaluation result of each category, is the serial number corresponding to each category, represents the evaluation result of the first epoch (chart library) in this stage, represents the evaluation result of the last epoch (chart library) in this stage, is the adjustment coefficient, and by adjusting the number of categories to be amplified can be changed (the meaning and value of the adjustment coefficient in the following formulas are the same), is the enhancement coefficient, and the expression of the enhancement coefficient is as follows:

[0045] ;

[0046] The enhancement coefficient can enhance the training preference of each category. When is greater than or equal to 1, it means that in this training stage, the detection effect of the model on this category is decreasing, and the training preference is enhanced. When is less than 1, it means that in this stage, the detection effect of the model on this category remains stable or has been improved to a certain extent, and the training preference remains unchanged.

[0047] Step 4: Amplification of category information

[0048] In the category information amplification module, the quantified training preference is weighted and combined with the sample quantity of the category to generate a set of amplified quantities for the category.

[0049] The main method of amplification is to use data augmentation strategies. Here, one or more strategies can be used to amplify the sample quantity. The objects of amplification are the categories with insufficient information in the training set and low AP in this stage. Calculate the requirements of the model during training using the training preference of the model, that is, the optimal value of the sample quantity to be amplified in the current stage. Its expression is as follows:

[0050] ;

[0051] In the above formula, is the optimal value of the sample quantity to be amplified in the current stage, is the sample quantity of the category in the original training set, is to judge whether each category needs to amplify the sample quantity. This discriminant is a binary classifier, and the judgment basis is to compare the evaluation values of the category between different epochs (chart library), The expression is as follows:

[0052] ;

[0053] Finally, the final output result of the category still continues to use the previous normalization method. The formula is as follows:

[0054] ;

[0055] In the above formula, is the normalization range, is the final output result of the category, is the set of.

[0056] Step 5: Update of the training set and continued training of the model

[0057] In the training set update module, the object to be updated is the label information in the training set. During formal training, every time the category information in the training set is updated, the label information of the corresponding data needs to be updated synchronously, that is, after each amplification of the category information is completed, the label information corresponding to the training set needs to be updated synchronously. The updated information is mainly the newly amplified category information in the training set. Mix the newly amplified category information data with the original training set to form a new training set, and the model starts the next stage of training with the new training set.

[0058] Figure 2This is an embodiment of traffic dataset enhancement. First, the model is pre-trained using the original training set to extract corresponding evaluation results and calculate preliminary training preferences. At the same time, the category information in the training set is extracted according to the label file to establish an online category information library for subsequent use in the process of updating the training set. The extracted training preferences are weighted and combined with the category information of the training set to generate the amplification quantity of sample information. Then, based on this set of results, the data augmentation method is used to amplify the sample quantity of the categories with insufficient information, optimize the original training set, and finally generate a new training set. The model starts training with the new training set. During the training process, the training situation of the model is monitored in real time. When the gradient of AP drops significantly, the model will be paused for training. Then, taking this as the first stage, the training preferences of this stage are calculated using the evaluation results, and then weighted and combined with the category imbalance problem to generate a set of amplification quantities for categories. According to the results, the corresponding category information is extracted from the online category information library, and finally, the data augmentation method is used to amplify this information into the original training set to generate a new training set for the next stage of training. The above is the operation process of stage n = 1. After re-importing the newly generated training set into the model, the training of stage n = 2 starts, and the operations of stage n = 1 are repeated in each stage. It should be noted that the parameter n here is uncertain, and the value of this parameter is determined by the training situation of the model. The data augmentation strategy includes various methods such as color change, noise, and occlusion. Therefore, the data augmentation strategy used for category information amplification can be different.

[0059] The present invention optimizes the data augmentation method based on the model training preferences, optimizing the problem of most current data augmentation with manually set parameters and manual matching, making the matching between data augmentation and the model more objective. At the same time, a discriminator is introduced in the training of the model. According to the training situation of the model, the training is divided into multiple stages, and the category information of the training set is adjusted multiple times with the stage training preferences, forcing the model to pay more attention to the learning of certain categories in the next training, solving the deficiencies in the patent application document with the publication number CN114898162A submitted by the applicant, and making the matching between the data augmentation algorithm and the model more perfect.

[0060] The above embodiments cannot limit the protection scope of the present invention. Equivalent modifications and variations made by those skilled in the professional technical field without departing from the overall concept of the present invention still fall within the scope covered by the present invention.

Claims

1. A multi-stage adjustment data augmentation method for model training, characterized in that Including: An information preprocessing module, a discriminator module, a training preference calculation module, a category information amplification module, and a training set update module; The method steps are as follows: Step 1. Establishment of an online category information library and pre-training of the model In the information preprocessing module, an online category information library is established according to the label file, and the model is pre-trained; the online category information library contains the semantic information of all categories in the training set, and the semantic information of these categories is collected and classified through the corresponding label files; pre-training means that before formal training, the model is trained using the original data set, and the evaluation results of the model on the original data set are obtained through pre-training. Then, the training preference of the model in pre-training is calculated, and the original training set is preliminarily optimized according to the training preference; the training set is traffic-related images. Step 2. Model training and discrimination In the discriminator module, the model starts formal training using the training set generated in the information preprocessing module. During the training process, the model is divided into multiple stages according to the training situation of the model. The discriminator module will monitor the training situation of the model in real time, monitor and discriminate each stage. When the slope of the AP drops significantly, the discriminator will pause the training. Then, taking this as the first stage, the training preference of this stage is calculated using the evaluation results, and the corresponding evaluation results are extracted to prepare for calculating the training preference of the current stage; Step 3. Calculation of stage-based training preference Before the start of the next training stage, the training preference calculation module extracts the evaluation results of the model in the current training stage and quantifies the training preference of the current stage. The calculation of the training preference needs to use the mAP of this stage and the AP of the category. The training preference is obtained based on the evaluation results of this stage, and the formula is as follows: ； In the above formula, is the training situation of the category in the th training stage, is the evaluation result of each category, is the serial number corresponding to each category, represents the evaluation result of the first epoch in this stage, represents the evaluation result of the last epoch in this stage, is the adjustment coefficient, is the enhancement coefficient, and the expression of the enhancement coefficient is as follows: ； Step 4. Amplification of category information In the category information amplification module, the quantified training preference is weighted and combined with the sample quantity of the category to generate a set of amplified quantities of the category; The amplification method is to use a data augmentation strategy to calculate the requirements of the model in training using the training preference of the model, that is, the optimal value of the sample quantity to be amplified in the current stage. The expression is as follows: ； In the above formula, is the optimal value of the number of samples to be amplified in the current stage, is the number of samples of the category in the original training set, is to determine whether to amplify the number of samples for each category, The expression is as follows: ； Finally, the final output result of the category is calculated using a normalization method, and the formula is as follows: ； In the above formula, is the normalized range, is the final output result of the category, is a set of; Step 5. Training set update and model continuous training In the training set update module, the update object is the label information in the training set. The newly amplified category information data is mixed with the original training set to form a new training set, and the model starts the next stage of training using the new training set; the new training set is traffic-related images.

Images