Fine adjustment method of industrial defect detection pre-training model
Through the combination of incremental learning and logarithmic class weight balance function, the problem of insufficient recognition ability and catastrophic forgetting of industrial defect detection models on category imbalanced data sets is solved, and faster convergence and better generalization are achieved Performance reduces data labeling costs.
Patent Information
- Application Number
- CN202411190240.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-28
- Publication Date
- 2024-12-06
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Existing industrial defect detection pretrained models are prone to bias when facing data sets with unbalanced categories, resulting in insufficient recognition of rare defect types, and may forget pretrained knowledge during the fine-tuning process, resulting in a decrease in generalization ability.
Using incremental learning method, using logarithmic class weight balance function, fine-tuning the model on specific task data sets through Adam optimization algorithm and cosine annealing algorithm, retaining the weight of the pre-trained model, combining large-scale data sets and new data sets Training to ensure that the model can converge quickly on specific tasks and maintain generalization capabilities.
It effectively improves the generalization performance and convergence speed of the model, reduces the demand for defect labeling data, reduces data labeling costs, and avoids catastrophic forgetting, ensuring that the model can accurately identify defects on specific tasks.
Smart Images

Figure CN119091250A_ABST
Abstract
Description
Technical Field
[0001] The present invention is applied to the technical field of industrial visual inspection, and in particular relates to a fine-tuning method for an industrial defect detection pre-training model. Background Art
[0002] With the rapid development of deep learning technology, pre-trained models have become an important part of the field of industrial defect detection. Industrial pre-trained models are models that are fine-tuned for specific industries or application scenarios, aiming to improve the performance of the model on specific tasks. Fine-tuning refers to the process of adapting a large general pre-trained model to specific tasks or industry needs by performing additional training on data in a specific field. Pre-training the training model on a large-scale defect dataset can learn extensive knowledge and strong generalization capabilities. In addition, since the pre-trained model has learned a large number of common defect features, fewer parameters need to be adjusted during the fine-tuning process, and the model can converge to a better solution faster. Pre-trained models can also save time and resources and reduce data requirements. In practical applications, only less defect annotation data is needed to achieve better results, reducing the cost of data annotation. There may be some challenges in making pre-trained models on large-scale defect datasets, which affect the overall performance and generalization ability of the model. First, some defect types may occupy a large proportion in the dataset, while other types of samples are relatively few. This class imbalance phenomenon will cause the model to be biased during the training process, making it perform better on common defect types, but perform poorly when dealing with rare defect types. Specifically, during the training process, the model will tend to learn the ability to recognize categories with a large number of categories, because these categories occupy a major position in the calculation of the loss function, resulting in the model's lack of sufficient recognition and generalization capabilities when facing rare categories. Because different defect types often have different degrees of harm, the missed detection of certain key defects may bring huge safety hazards and economic losses. Secondly, when fine-tuning the pre-trained model, the adjustment of weights may cause the model to forget some of the knowledge learned in the pre-training stage, especially when the new task is very different from the source domain task. This phenomenon is called catastrophic forgetting. The essence of the fine-tuning process is to train the model with data on the new task to adapt it to the new task requirements. However, in this process, the weights of the model will be adjusted, and these adjustments may cover or replace the original pre-training weights, causing the model to forget the knowledge accumulated during pre-training. This problem is particularly prominent in the field of deep learning, because deep neural networks have highly complex structures and a large number of parameters, which makes them prone to over-adjusting weights during the optimization process when facing new tasks, thereby losing the ability to generalize to the source domain tasks. If a fine-tuning method for the industrial defect detection pre-trained model can be designed that requires less defect annotation data for practical applications, uses incremental learning methods, samples and learns old data, uses a logarithmic class weight balancing function, and improves the generalization performance and convergence speed of the model, the above problems can be solved. Summary of the invention
[0003] The technical problem to be solved by the present invention is to overcome the shortcomings of the prior art and provide a fine-tuning method for an industrial defect detection pre-trained model that requires less defect annotation data for practical applications, uses an incremental learning method, samples and learns old data, and uses a logarithmic class weight balancing function to improve the model's generalization performance and convergence speed.
[0004] The technical solution adopted by the present invention is: the present invention comprises the following specific steps: ① Determine and train the pre-training model dataset; ② Initialize the parameters in the pre-trained model and fine-tune it on a dataset for a specific task; ③Use Adam optimization algorithm to update model parameters; The fine-tuning in step ② minimizes the loss function in a specific task or field: ,in Represents the fine-tuned model parameters and the initialization parameters in the pre-trained model , and fine-tuned on a dataset for a specific task, the calculation formula is ,in, represents the number of samples of the fine-tuning dataset, represents the input sample, Indicates the corresponding label, Indicates the use of parameters Model, represents the loss function for the fine-tuning task.
[0005] Furthermore, step ① performs weight migration on the industrial training model, retains part or all of the parameters of the pre-training model, and generates the weight of the pre-training model by training on a large-scale data set. The calculation formula is: ,in, represents the number of samples in the pre-training dataset, represents the input sample, Indicates the corresponding label, Indicates the use of parameters Model, The pre-trained model learns the model weights of common defect features from a large-scale defect pre-training dataset. Represents the loss function of the pre-training task.
[0006] Furthermore, in step ①, the class weight cross entropy loss function is used to train defect samples, where the weights of different classes are inversely proportional to their frequencies of occurrence in the training set. The calculation formula is: ,in It is The weight of the samples, Represents the corresponding label, which is calculated based on the frequency of its category in the training set.
[0007] Furthermore, in step ①, the weight of each defect category is calculated by the logarithmic weight calculation method. The calculation formula is: ,in It is The weight of each category, is the frequency of the category in the training set, It is a smoothing parameter that maps the category weights to a continuous range through logarithmic weight calculation.
[0008] Furthermore, in step ①, the cosine annealing algorithm is used to dynamically adjust the learning rate, eliminating the ambiguous labeled data in the pre-training defect data set. The learning rate is gradually reduced during the training process so that the model can accurately converge to the local optimal solution. The calculation formula is: ,in It is in The learning rate for the iteration steps, is the initial learning rate, is the total number of training iterations.
[0009] Further, after the pre-training model is completed in step ①, the pre-training dataset is , which includes samples, the dataset of the target task is , which includes samples, each sample is represented by ,in is the image data, is the corresponding defect category label, and the target category of the specific defect detection task is , filter out samples with the same target category from the pre-training dataset to form an auxiliary dataset , which contains samples, with a new dataset for the target task Primary and secondary data sets The old data is supplemented by the sampling frequency Old data Divide equally into each epoch and the new data Train together. The number of training samples in each epoch during the fine-tuning phase is .
[0010] Furthermore, in step ③, when fine-tuning, the Adam optimization algorithm is used to update the model parameters, and the formula for calculating the model weight update rule is: ; ; ; ; ,in, represents the fine-tuned model parameters, Indicates the current parameters. represents the learning rate, Represents the loss function Relative to the parameters The gradient of and denote the first-order moment estimate and the second-order moment estimate, respectively. and denote the decay rate of momentum and the decay rate of the second-order moment, respectively. Indicates the number of current iterations. and They represent the bias-corrected first-order moment estimate and second-order moment estimate, respectively. is a small constant that maintains numerical stability. BRIEF DESCRIPTION OF THE DRAWINGS
[0011] Figure 1 It is a work flow chart of the present invention. DETAILED DESCRIPTION
[0012] like Figure 1 As shown, in this embodiment, the present invention includes the following specific steps: ① Determine and train the pre-training model dataset; ② Initialize the parameters in the pre-trained model and fine-tune it on a dataset for a specific task; ③Use Adam optimization algorithm to update model parameters; The fine-tuning in step ② minimizes the loss function in a specific task or field: ,in Represents the fine-tuned model parameters and the initialization parameters in the pre-trained model , and fine-tuned on a dataset for a specific task, the calculation formula is ,in, represents the number of samples of the fine-tuning dataset, represents the input sample, Indicates the corresponding label, Indicates the use of parameters Model, Represents the loss function of the fine-tuning task. It can be seen that the knowledge of general defect features learned by the pre-training model is transferred to the specific defect detection task, thereby improving the generalization performance and convergence speed of the model. The use of pre-training model-fine-tuning in practical applications only requires less defect annotation data to achieve better results, reducing the data annotation cost. The use of the logarithmic category weight balance function can reduce the impact of extreme values, making the loss function smoother, making it easier to optimize the model, and using the incremental learning method to sample and learn old data. Avoid learning new target data in the fine-tuning stage and forgetting the knowledge learned in the pre-training data set, and fine-tuning on the data set of a specific task can ensure better fine-tuning results.
[0013] like Figure 1 As shown, in this embodiment, step ① performs weight migration on the industrial training model, retains part or all of the parameters of the pre-training model, and generates the weight of the pre-training model by training on a large-scale data set. The calculation formula is ,in, represents the number of samples in the pre-training dataset, represents the input sample, Indicates the corresponding label, Indicates the use of parameters Model, The pre-trained model learns the model weights of common defect features from a large-scale defect pre-training dataset. Represents the loss function of the pre-training task. It can be seen that the weight transfer of industrial pre-training models is based on the idea of transfer learning, which uses the common features and knowledge learned by the pre-training model on industrial data to help the new task model converge faster and improve generalization ability. During the migration process, part or all of the parameters of the pre-training model are usually retained, and then fine-tuned on the data set in a specific industrial scenario to adapt the model to the specific features and requirements of a specific industrial task.
[0014] like Figure 1 As shown, in this embodiment, in step ①, the class weight cross entropy loss function is used to train the defect samples, where the weights of different classes are inversely proportional to their occurrence frequencies in the training set, and the calculation formula is: ,in It is The weight of the samples, Represents the corresponding label, which is calculated according to the frequency of its category in the training set. It can be seen that in order to improve the efficiency of pre-training model production, only defect samples are used for training instead of non-defect samples. The model will be able to focus more on learning defect-specific features without being disturbed by non-defect samples. When the pre-training model is applied to a specific detection task, it will be fine-tuned in combination with defect and non-defect samples. At the same time, in order to maintain the weight balance of each defect category, the training data is appropriately adjusted for sample weights. The purpose of this strategy is to prevent the model from being biased towards more common defect categories during the learning process and ignoring other relatively rare defect categories. The loss function is adjusted by class weights so that samples of different categories contribute differently to the loss. Specifically, a class-weighted cross-entropy loss function is used, in which the weights of different categories are inversely proportional to their frequency of occurrence in the training set.
[0015] like Figure 1 As shown, in this embodiment, the weight of each defect category is calculated by the logarithmic weight calculation method in step ①, and the calculation formula is: ,in It is The weight of each category, is the frequency of the category in the training set, is a smoothing parameter that maps the category weights to a continuous range through logarithmic weight calculation. is a smoothing parameter used to avoid the problem of taking logarithms when the frequency is 0. Through this logarithmic form of weight calculation, the weight of the category can be mapped to a continuous range, and it can better handle categories with lower frequencies. The advantage of this weight calculation method is that it can better adapt to different data sets and show better stability when dealing with category imbalance problems. Compared with simply using the inverse of the frequency as the weight, the logarithmic form of weight calculation method can better balance the influence between different categories, thereby improving the performance and generalization ability of the model in defect detection tasks. In addition, the logarithmic form of weight calculation method also has certain mathematical significance. Using logarithmic category weights can smooth the curve of the loss function to a certain extent. Due to the characteristics of the logarithmic function, it can reduce the influence of extreme values, making the loss function smoother, making it easier to optimize the model.
[0016] like Figure 1 As shown, in this embodiment, the cosine annealing algorithm is used in step ① to dynamically adjust the learning rate, excluding the ambiguous labeled data of the pre-training defect data set, and gradually reducing the learning rate during the training process so that the model can accurately converge to the local optimal solution. The calculation formula is ,in It is in The learning rate for the iteration steps, is the initial learning rate, is the total number of training iterations. It can be seen that in the process of making the pre-trained model, the ambiguous labeled data of the pre-trained defect data set are excluded to ensure that the model focuses on learning clear and reliable defect features. The cosine annealing algorithm is used to dynamically adjust the learning rate, optimize the training process, and improve the performance and convergence speed of the model. The cosine annealing algorithm is an effective learning rate adjustment strategy that can gradually reduce the learning rate during the training process, so that the model can converge to the local optimal solution more accurately and avoid falling into the local minimum point.
[0017] like Figure 1 As shown, in this embodiment, after the pre-training model is completed in step ①, the pre-training data set is , which includes samples, the dataset of the target task is , which includes samples, each sample is represented by ,in is the image data, is the corresponding defect category label, and the target category of the specific defect detection task is , filter out samples with the same target category from the pre-training dataset to form an auxiliary dataset , which contains samples, with a new dataset for the target task Primary and secondary data sets The old data is supplemented by the sampling frequency Old data Divide equally into each epoch and the new data Train together. The number of training samples in each epoch during the fine-tuning phase is It can be seen that after completing the production of the pre-trained model, fine-tuning is performed for a specific detection task, and the new dataset of the target task is used. Primary and secondary data sets The old data is supplemented by the sampling frequency Old data Divide equally into each epoch and the new data Train together.
[0018] like Figure 1 As shown, in this embodiment, in step ③, when fine-tuning, the Adam optimization algorithm is used to update the model parameters, and the formula for calculating the model weight update rule is: ; ; ; ; ,in, represents the fine-tuned model parameters, Indicates the current parameters. represents the learning rate, Represents the loss function Relative to the parameters The gradient of and denote the first-order moment estimate and the second-order moment estimate, respectively. and denote the decay rate of momentum and the decay rate of the second-order moment, respectively. Indicates the number of current iterations. and They represent the bias-corrected first-order moment estimate and second-order moment estimate, respectively. It is a small constant that maintains numerical stability. It can be seen that through this process, the weight parameters of the pre-trained model can be used as an initial point, and then the model can be fine-tuned to better adapt to specific tasks.
[0019] The working principle of the present invention is as follows: weight migration is performed in the industrial training model, defect samples are trained through the category weight cross entropy loss function, the weight of each defect category is calculated, the learning rate is dynamically adjusted using the cosine annealing algorithm, the training data set is determined, the parameters are initialized in the pre-training model and fine-tuned on the data set of the specific task, and the model parameters are updated using the Adam optimization algorithm.
[0020] Although the embodiments of the present invention are described with practical solutions, they do not constitute limitations on the meaning of the present invention. For those skilled in the art, it is obvious to modify the implementation scheme and combine it with other solutions based on this description.
Claims
1. A fine-tuning method for a pre-trained model for industrial defect detection, characterized in that , which includes the following specific steps: ① Determine and train the pre-training model dataset; ② Initialize the parameters in the pre-trained model and fine-tune it on a dataset for a specific task; ③Use Adam optimization algorithm to update model parameters; The fine-tuning in step ② minimizes the loss function in a specific task or field: ,in Represents the fine-tuned model parameters and the initialization parameters in the pre-trained model , and fine-tuned on a dataset for a specific task, the calculation formula is ,in, represents the number of samples of the fine-tuning dataset, represents the input sample, Indicates the corresponding label, Indicates the use of parameters Model, represents the loss function for the fine-tuning task.
2. The fine-tuning method of a pre-trained model for industrial defect detection according to claim 1, characterized in that: Step ① Perform weight migration on the industrial training model, retain part or all of the parameters of the pre-training model, and generate the weights of the pre-training model by training on a large-scale data set. The calculation formula is: ,in, represents the number of samples in the pre-training dataset, represents the input sample, Indicates the corresponding label, Indicates the use of parameters Model, The pre-trained model learns the model weights of common defect features from a large-scale defect pre-training dataset. Represents the loss function of the pre-training task.
3. The fine-tuning method of a pre-trained model for industrial defect detection according to claim 2, characterized in that: In step ①, the class weight cross entropy loss function is used to train defect samples, where the weights of different classes are inversely proportional to their frequency of occurrence in the training set. The calculation formula is: ,in It is The weight of the samples, Represents the corresponding label, which is calculated based on the frequency of its category in the training set.
4. The fine-tuning method of a pre-trained model for industrial defect detection according to claim 3 is characterized in that: In step ①, the weight of each defect category is calculated using the logarithmic weight calculation method. The calculation formula is: ,in It is The weight of each category, is the frequency of the category in the training set, It is a smoothing parameter that maps the category weights to a continuous range through logarithmic weight calculation.
5. The fine-tuning method of a pre-trained model for industrial defect detection according to claim 4, characterized in that: In step ①, the cosine annealing algorithm is used to dynamically adjust the learning rate, eliminating the ambiguous labeled data in the pre-training defect data set. The learning rate is gradually reduced during the training process so that the model can accurately converge to the local optimal solution. The calculation formula is: ,in It is in The learning rate for the iteration steps, is the initial learning rate, is the total number of training iterations.
6. The fine-tuning method of a pre-trained model for industrial defect detection according to claim 5, characterized in that: After the pre-training model is completed in step ①, the pre-training dataset is , which includes samples, the dataset of the target task is , which includes samples, each sample is represented by ,in is the image data, is the corresponding defect category label, and the target category of the specific defect detection task is , filter out samples with the same target category from the pre-training dataset to form an auxiliary dataset , which contains samples, with a new dataset for the target task Primary and secondary data sets The old data is supplemented by the sampling frequency Old data Divide equally into each epoch and the new data Train together. The number of training samples in each epoch during the fine-tuning phase is .
7. The fine-tuning method of a pre-trained model for industrial defect detection according to claim 6, characterized in that: Step ③ When fine-tuning, use the Adam optimization algorithm to update the model parameters. The formula for calculating the model weight update rule is: ; ; ; ; ,in, represents the fine-tuned model parameters, Indicates the current parameters. represents the learning rate, Represents the loss function Relative to the parameters The gradient of and denote the first-order moment estimate and the second-order moment estimate, respectively. and denote the decay rate of momentum and the decay rate of the second-order moment, respectively. Indicates the number of current iterations. and They represent the bias-corrected first-order moment estimate and second-order moment estimate, respectively. is a small constant that maintains numerical stability.
Citation Information
Patent Citations
Welding image defect identification method and device, storage medium and equipment
CN112116560A
Photovoltaic image defect classification method based on transfer learning and unsupervised learning methods
CN115761359A
Surface defect segmentation method based on two-stage incremental learning
CN115797309A
Artificial intelligence segmentation and classification algorithm for industrial appearance defect detection
CN116206104A
Feature decoupling small-amount sample pre-training model robustness fine tuning method and device
CN116994099A
Cited By
Training method of migration target detection model from GPU to NPU and target detection method
CN120911529A
