Defect detection loss function weight optimization method and system based on a proxy model

Abstract

The application belongs to the technical field of image processing, and particularly discloses a defect detection loss function weight optimization method and system based on a proxy model, which comprises the following steps: S1, collecting image data and marking defects to form a training set; S2, determining a neural network structure for defect detection and determining a proxy model according to the neural network structure; S3, determining a candidate loss function and combining the candidate loss functions into a fusion loss function by weighting; S4, training the proxy model based on the current fusion loss function through the training set, and evaluating the performance of the fusion loss function according to the accuracy of the proxy model; S5, optimizing the weight parameters according to the performance evaluation result; and S6, repeating S4 and S5 until a preset iteration stopping condition is reached, and selecting the fusion loss function with the best performance as the final loss function. The application can select appropriate weights for the fusion loss function used by the neural network, thereby improving the performance of the model on the industrial data set.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, and more specifically, relates to a method and system for optimizing the weights of a defect detection loss function based on a surrogate model. Background Technology

[0002] With the advancement of big data and computing power, deep learning has achieved significant success in various visual tasks, providing new solutions for visual inspection. The loss function plays a crucial role in training neural networks.

[0003] However, it remains unclear whether a manually predefined single loss function is the optimal choice for the current dataset. In industrial settings, industrial datasets differ significantly from natural image datasets due to factors such as the manufacturing environment. First, because industrial products have extremely high pass rates, the total number of normal samples and pixels in the dataset is usually far greater than that of defective samples. Different defect types contribute differently to the loss value variation. Learning features from a large number of negative samples may reduce the network's training efficiency and accuracy, resulting in poor prediction performance for defect types with fewer data samples that require more attention. Second, due to regular changes such as tool wear and irregular changes such as environmental interference during processing, defects in industrial datasets are highly diverse. Some defects may be very similar to the background, and their areas vary in size. Since image semantic segmentation is pixel-level, defect classes with fewer pixels contribute less to the loss, which may also lead to poor actual detection performance of the model. Therefore, a manually predefined single loss function is likely to introduce some error into the final result.

[0004] In recent years, research has begun to explore incorporating more types of loss functions into the model training process, combining different loss functions to form new error metrics. There are two main approaches to implementing fusion loss: one is based on prior human knowledge, using fixed weights to weight and fuse different loss functions, such as direct proportional allocation or pre-comparison experiments. However, this subjective judgment-based method may lead to incorrect choices and is not suitable for different datasets. The other approach is to use grid search to validate all possible fusion methods through batch learning. However, validating each candidate loss function or fusion ratio requires training a complete neural network model, which is very costly. Therefore, optimizing the fusion loss function reasonably and at low cost is crucial for improving the accuracy of industrial inspection and advancing the process of automated inspection. Summary of the Invention

[0005] To address the aforementioned deficiencies or improvement needs of existing technologies, this invention provides a method and system for optimizing the weights of a defect detection loss function based on a surrogate model. The purpose is to rapidly optimize the loss function when performing defect detection through a neural network, thereby improving the accuracy of defect segmentation.

[0006] To achieve the above objectives, according to a first aspect of the present invention, a method for optimizing the weights of a defect detection loss function based on a surrogate model is proposed, comprising the following steps:

[0007] S1. Collect image data from the industrial site and mark the defective areas in the images to form a training set;

[0008] S2. Determine the neural network structure for defect detection, and determine the surrogate model based on the neural network structure;

[0009] S3. Determine the candidate loss functions and combine them with weights to form a fusion loss function. Use the weight parameters during weighting as the variables to be optimized and set their initial values ​​randomly.

[0010] S4. Based on the current fusion loss function, train the proxy model using the training set, and evaluate the performance of the current fusion loss function based on the accuracy of the trained proxy model.

[0011] S5. Optimize the weight parameters based on the performance evaluation results;

[0012] S6. Repeat steps S4 and S5 until the preset iteration stop condition is met; select the fusion loss function with the best performance during the iteration process as the final fusion loss function.

[0013] As a further preferred embodiment, step S2, determining the surrogate model based on the neural network structure, includes:

[0014] A surrogate model is obtained by reducing the network structure and designing early stopping based on the neural network structure.

[0015] As a further preferred step, in step S5, when optimizing the weight parameters based on the performance evaluation results, each weight parameter is optimized sequentially. For a given weight parameter:

[0016] During the iteration process, the weight parameter is updated by forward / backward optimization while other weight parameters are fixed; then, the optimization direction of the weight parameter is determined based on the changing trend of the fusion loss function performance before and after the update.

[0017] As a further optimization, in step S5, the weight parameters are updated and optimized according to the following formula:

[0018] α i+1 =α i +(acc i-acc i-1 )*G

[0019] Where, α i Let α be the weight parameter at the current i-th iteration. i+1 Let acc be the weight parameters at the (i+1)th iteration after the update. i For the performance evaluation result of the i-th iteration, acc i-1 This represents the performance evaluation result of the (i-1)th iteration, where G is a preset constant.

[0020] As a further preferred option, the performance evaluation result is the IOU accuracy of the trained proxy model.

[0021] As a further preferred option, the candidate loss function includes the cross-entropy loss function and the Dice loss function.

[0022] As a further preferred embodiment, the neural network structure adopts a Unet structure, and the encoding part of the Unet is replaced with a VGG Block.

[0023] According to a second aspect of the present invention, a defect detection loss function weight optimization system based on a surrogate model is provided, comprising a processor for executing the aforementioned defect detection loss function weight optimization method based on a surrogate model.

[0024] According to a third aspect of the present invention, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the above-described method for optimizing the weights of a defect detection loss function based on a proxy model.

[0025] In summary, compared with the prior art, the above-described technical solutions conceived by this invention mainly possess the following technical advantages:

[0026] This invention utilizes a small surrogate model to optimize the loss function, thereby improving the accuracy of defect detection and segmentation and significantly increasing computational efficiency, while avoiding the need for multiple manual trials and corrections.

[0027] Specifically, the training process of small agent models provides reliable prior knowledge for determining the loss function, which strongly supports the determination of the optimal fusion loss function. It can quickly determine a loss function with good performance, greatly saving manpower, time and computing costs. Based on the optimized loss function, the performance and training speed of neural network models can be significantly improved, providing a more efficient and accurate solution for industrial applications. Attached Figure Description

[0028] Figure 1 This is a schematic diagram illustrating the weight optimization of the loss function based on the surrogate model / human prior knowledge in an embodiment of the present invention;

[0029] Figure 2 This is a schematic diagram of the defect segmentation network downsampling structure constructed according to an embodiment of the present invention;

[0030] Figure 3 This is a schematic diagram of the proxy model construction method according to an embodiment of the present invention;

[0031] Figure 4 Figures (a) and (b) in the figure are a comparison of the accuracy of the proxy model and the original model in the embodiment of the present invention;

[0032] Figure 5 This is a schematic diagram of the loss function weight optimization method according to an embodiment of the present invention;

[0033] Figure 6 This is a schematic diagram illustrating the process of optimizing the fusion loss function using the proxy model in an embodiment of the present invention.

[0034] Figure 7 In the first embodiment of the present invention, (a) to (f) show the defect image, defect label, and detection results using the fusion loss function, BCE loss, Dice loss, and Focal loss of the present invention, respectively.

[0035] Figure 8 Images (a) through (f) in the second embodiment of the present invention show the defect image, defect label, and detection results using the fusion loss function, BCE loss, Dice loss, and Focal loss of the present invention, respectively.

[0036] Figure 9 Images (a) through (f) show the defect image, defect label, and detection results using the fusion loss function, BCE loss, Dice loss, and Focal loss of the third embodiment of the present invention, respectively. Detailed Implementation

[0037] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0038] This invention provides a method for optimizing the weights of a defect detection loss function based on a surrogate model, such as... Figure 1 As shown, it includes the following steps:

[0039] S1. Collect image data from the industrial site and mark the defective areas in the images to form labels, thereby obtaining the training set.

[0040] S2. Determine the neural network structure for defect detection (segmenting the defect region of the image to be detected), and obtain a surrogate model based on the neural network structure by reducing the network structure and designing early stopping, so that the surrogate model and the neural network model maintain similarity.

[0041] Specifically, such as Figure 3 As shown, reducing the network structure involves decreasing the number of filters used for feature extraction, resulting in a model with a proportionally reduced number of channels; early stopping is achieved by reducing the number of iterations in the training process. Figure 4 As shown, the surrogate model exhibits the same performance change trend as the original model in terms of accuracy, and only has a certain performance difference with the original model in the final training result. This can provide reliable prior knowledge for evaluating the performance of candidate loss functions.

[0042] S3. Determine candidate loss functions based on the data characteristics of the task to be processed, and combine them with weights to form a fusion loss function. Use the weight parameters during weighting as the variables to be optimized, and randomly set their initial values.

[0043] Specifically, the fusion loss function Final The expression is as follows:

[0044]

[0045] Where α, β, ..., λ are different candidate loss functions. j The corresponding weight parameters together constitute the weight combination ω, and the initial value is randomly obtained.

[0046] S4. Based on the current fusion loss function, train the proxy model using the training set, and evaluate the performance of the current fusion loss function based on the accuracy of the trained proxy model.

[0047] S5. Optimize the weight parameters based on the performance evaluation results.

[0048] Specifically, the next optimization direction for the weight parameters is determined based on the performance change trend of the fusion loss function during the iteration process, such as... Figure 5 As shown, we can first set a weight parameter to be updated in either a positive or negative direction (i.e., increased or decreased), and then evaluate how to update the parameter based on the training results. If increasing a certain weight α leads to better training results, then we continue to increase the weight α; otherwise, we decrease the value of the weight α. Preferably, we use the IOU accuracy of the trained surrogate model as the performance evaluation result.

[0049] Furthermore, each weight parameter can be optimized sequentially. For a given weight parameter: perform forward / backward optimization and update the weight parameter while keeping other weight parameters fixed; then, based on the trend of the performance change of the fusion loss function before and after the update, determine the optimization direction for the weight parameter and evaluate whether it is necessary to continue updating the weight parameter; for example, if the performance decreases or the performance improvement is less than a preset threshold after optimizing the weight parameter, stop optimizing the weight parameter and proceed to the next weight parameter optimization.

[0050] You can also set constraints on each weight parameter to optimize the weight parameters simultaneously; for example, when there are only two weight parameters α and β, α + β = 1.

[0051] Furthermore, the weight parameters are optimized and updated according to the following formula:

[0052] α i+1 =α i +(acc i -acc i-1 )*G

[0053] Where, α i Let α be the weight parameter at the current i-th iteration. i+1 Let acc be the weight parameters at the (i+1)th iteration after the update. i For the performance evaluation result of the i-th iteration, acc i-1 This represents the performance evaluation result of the (i-1)th iteration, where G is a preset constant.

[0054] S6. Repeat steps S4 and S5 until the preset number of iterations is reached; select the fusion loss function with the best performance during the iteration process as the final fusion loss function and transfer it to the neural network.

[0055] S7. Train the neural network using the training set to ensure complete convergence of the network, and use the trained neural network as the defect segmentation model.

[0056] The following are specific examples:

[0057] The method of this invention is described in detail using commutator cylindrical defect data as an example, but the application of this invention is not limited to this. The commutator is an important component in a motor. During the production process, its outer surface is left with texture features due to regular changes in turning and grinding, as well as irregular changes due to environmental interference. Simultaneously, surface defect features vary due to changes in machining tools and molds, exhibiting different contrast characteristics. This further increases the difficulty of manually selecting the optimal loss function. This invention designs an automatic optimization method that integrates loss weight parameters to address the above situations.

[0058] S1. Collect industrial images and mark abnormal areas at the pixel level to form labels, finally obtaining a private dataset containing 2286 images, of which 2187 are normal images and only 99 are defective images, which brings a great challenge to neural network-based defect detection.

[0059] S2. Based on the task requirements, design a neural network model for defect segmentation. This model can be selected or designed according to the task and dataset requirements, such as using Unet, PSPNet, or DeepLab. In this embodiment, targeting the grayscale and edge features of the defect region in the dataset, a Unet structure is used as the segmentation network, and the encoding part of Unet is replaced with a VGG block, effectively enhancing the model's ability to extract low-level semantic information features from the dataset. The specific structure is as follows... Figure 2 As shown.

[0060] Then, a proxy model C_Model is constructed, using two methods—reducing the network structure and early stopping—to maintain similarity to the final model. For example... Figure 3 As shown, reducing the network structure involves decreasing the number of filters used for feature extraction, resulting in a model with a proportionally reduced number of channels. Early stopping can be achieved by reducing the number of training epochs. An early stopping condition can be set, such as when the model has reached a certain number of training epochs or when the IOU accuracy continuously decreases during model validation. In this embodiment, 50 epochs are selected as the early stopping condition. Figure 4 As shown, the surrogate model exhibits the same performance change trend as the original model in terms of accuracy, and only has a certain performance difference with the original model in the final training result, which can provide reliable prior knowledge for the performance evaluation of the loss function.

[0061] S3. Based on the characteristics of the dataset for the task to be processed, candidate loss functions are selected, and then appropriate fusion weights are chosen to weight and fuse these candidate loss functions. In this embodiment, Binary Cross-Entropy Loss (BCE) and Dice loss are used. This combination can better handle the problem of severe imbalance of defective samples in the dataset, while ensuring that the network model maintains good robustness in the early stages of training.

[0062] The fusion loss function is initialized as follows:

[0063]

[0064] Here, α1 and β1 constitute the weight combination ω1, which represents the weights corresponding to the two types of loss respectively. The initial values ​​are randomly obtained, and α1+β1=1. The selected BCE loss and Dice loss.

[0065] S4. Train the surrogate model using the fusion loss function, and use its final accuracy acc1 as the performance evaluation result of the current fusion loss function.

[0066] S5. Optimize the weight parameters based on the performance evaluation results.

[0067] During the first weight update, an update direction is randomly selected, for example, increasing α1 to obtain α2, thus obtaining the weight combination ω2, and returning to S4. In subsequent weight update optimizations, the next optimization direction is determined based on the performance trend of the fusion loss function, such as... Figure 5 As shown.

[0068] Specifically, a weight parameter can be updated in either the forward or reverse direction, i.e., by increasing or decreasing its value. Then, the current validation result (acc2) is used to assess whether further updates to the parameter are necessary. If acc2 is large, it indicates that the current update direction yields better performance, and it's advisable to maintain that direction and adjust further. Conversely, if acc2 is small, it suggests that updating the weight in the reverse direction would provide better performance.

[0069] Based on the adjusted weights, the fusion loss function is updated and then fed back into the surrogate model for training, such as... Figure 6 As shown.

[0070] S6. Repeat steps S4 and S5 until the preset number of iterations is reached; select the fusion loss function with the best performance during the iteration process as the final fusion loss function and apply it to the final model training process.

[0071] S7. Train the neural network model from scratch using the finalized loss function, ensuring complete convergence of the network. Then, use the trained neural network model for actual detection, such as... Figures 7 to 9 As shown, this paper presents three defect images and detection results in the dataset. It can be seen that compared with commonly used BCE, Dice, Focal loss, etc., the optimized fusion loss function of this invention can make the network pay more attention to the distribution of defect regions and obtain more accurate defect segmentation results.

[0072] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for optimizing the weights of a defect detection loss function based on a surrogate model, characterized in that, Includes the following steps: S1. Collect image data from the industrial site and mark the defective areas in the images to form a training set; S2. Determine the neural network structure for defect detection, and determine the surrogate model based on the neural network structure; S3. Determine the candidate loss functions and combine them with weights to form a fusion loss function. Use the weight parameters during weighting as the variables to be optimized and set their initial values ​​randomly. S4. Based on the current fusion loss function, train the proxy model using the training set, and evaluate the performance of the current fusion loss function based on the accuracy of the trained proxy model. S5. Optimize the weight parameters based on the performance evaluation results; specifically, optimize each weight parameter in turn. For a given weight parameter: During the iteration process, the weight parameter is optimized and updated in a forward / backward direction while other weight parameters are fixed. Then, based on the trend of the performance of the fusion loss function before and after the update, the optimization direction of the weight parameter is determined. The weight parameters are optimized and updated according to the following formula: in, These are the weight parameters for the current i-th iteration. These are the weight parameters for the (i+1)th iteration after the update. acc i This represents the performance evaluation result for the i-th iteration. acc i-1 The performance evaluation result is the IOU accuracy of the trained surrogate model, where G is a preset constant; the performance evaluation result is the IOU accuracy of the trained surrogate model. S6. Repeat steps S4 and S5 until the preset iteration stop condition is met; select the fusion loss function with the best performance during the iteration process as the final fusion loss function.

2. The defect detection loss function weight optimization method based on the surrogate model as described in claim 1, characterized in that, Step S2, determining the surrogate model based on the neural network structure, includes: A surrogate model is obtained by reducing the network structure and designing early stopping based on the neural network structure.

3. The defect detection loss function weight optimization method based on a surrogate model as described in claim 1 or 2, characterized in that, The candidate loss functions include the cross-entropy loss function and the Dice loss function.

4. The defect detection loss function weight optimization method based on the surrogate model as described in claim 1 or 2, characterized in that, The neural network structure adopts the Unet structure, and the encoding part of the Unet is replaced with VGG Block.

5. A defect detection loss function weight optimization system based on a surrogate model, characterized in that, Includes a processor for executing the defect detection loss function weight optimization method based on the surrogate model as described in any one of claims 1-4.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the defect detection loss function weight optimization method based on the surrogate model as described in any one of claims 1-4.

Images