Defect detection method, device, apparatus and storage medium
By iteratively optimizing the reinforcement learning model and the target detection model, synthetic defect images are generated to optimize the target detection model, which solves the problem of low accuracy of the target detection model and achieves efficient accuracy improvement with a small amount of data.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- GOERTEK INC
- Filing Date
- 2023-06-21
- Publication Date
- 2026-06-02
AI Technical Summary
Existing target detection models have low accuracy in the field of industrial quality inspection, which limits their application, especially when there is insufficient data, making it difficult to effectively improve model accuracy.
Synthetic defect images are generated by a reinforcement learning model and input into an object detection model along with the original images. The reinforcement learning model is then optimized based on the output of the object detection model. This process is iterated until the training termination condition is met, thus optimizing the training process of the object detection model.
Improving the accuracy and usability of target detection models under limited data conditions enhances the accuracy and efficiency of defect detection.
Smart Images
Figure CN116823758B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of defect detection technology, and in particular to a defect detection method, apparatus, equipment and storage medium. Background Technology
[0002] With the development of deep learning, artificial intelligence is increasingly being applied in the field of industrial quality inspection. Object detection models are widely used because they can clearly pinpoint the location of defects, making them suitable for industrial applications. However, training object detection models requires significantly more data than traditional classification models, resulting in lower accuracy and limiting their application. Therefore, improving the accuracy of object detection models has become a pressing technical problem. Summary of the Invention
[0003] The main objective of this invention is to provide a defect detection method, apparatus, device, and storage medium, which aims to solve the technical problem of low accuracy of target detection models in the prior art.
[0004] To achieve the above objectives, the present invention provides a defect detection method, the method comprising the following steps:
[0005] The defect image and the original image of the product are input into the reinforcement learning model respectively to obtain the synthetic defect image output by the reinforcement learning model, which is used as the effective synthetic defect image of the target detection model trained in this training.
[0006] The effective synthesized defect image and the original image are input into the target detection model. If the target detection model does not meet the preset training termination condition, the reinforcement learning model is optimized based on the output of the target detection model.
[0007] Return to the operation of inputting the defect image and the original image of the product into the reinforcement learning model respectively, and replace the reinforcement learning model with the optimized reinforcement learning model to obtain a new synthetic defect image. Based on the new synthetic defect image, determine the effective synthetic defect image to be used for the next training of the target detection model, start the next training of the target detection model, until the target detection model meets the preset training termination condition.
[0008] The target detection model after training is used to detect defects in the product.
[0009] Optionally, the object detection model used in each training session is either the initial object detection model or the object detection model obtained after the previous training.
[0010] Optionally, the reinforcement learning model includes a first feature extraction module, a second feature extraction module, a feature integration module, and a policy function;
[0011] The defect image and the original image of the product are input into the reinforcement learning model respectively to obtain a synthetic defect image output by the reinforcement learning model, including:
[0012] The first feature extraction module extracts features from the defect image of the product to obtain the first feature vector;
[0013] The second feature extraction module extracts features from the original image of the product to obtain the second feature vector;
[0014] The first and second feature vectors are concatenated and calculated by the feature integration module to obtain the concatenated feature vector.
[0015] The defect matrix and placement parameters are output based on the concatenated feature vectors using the strategy function;
[0016] The composite defect image is output based on the defect matrix and placement parameters.
[0017] Optionally, a synthesized defect image is output based on the defect matrix and placement parameters, including:
[0018] The defect image is filtered based on the defect matrix to obtain the filtered defect image;
[0019] The filtered defect image is superimposed on the original image based on the placement parameters, and the superimposed image is output as the synthetic defect image.
[0020] Optionally, the effective synthetic defect image to be used in the next training of the object detection model is determined based on the new synthetic defect image, including:
[0021] The overlap of defect regions is determined based on the predicted defect regions output by the target detection model and the defect regions in the new synthetic defect image.
[0022] If the predicted probability value of the new synthetic defect image output by the target detection model is greater than the preset probability value and the overlap of the defect regions is greater than the preset overlap, then the new synthetic defect image is determined to be a valid synthetic defect image for the next training of the target detection model.
[0023] Optionally, the reinforcement learning model also includes a value function, the output of which includes predicted probability values;
[0024] Optimize the reinforcement learning model based on the output of the object detection model, including:
[0025] The reward estimate is output by a value function based on the spliced feature vector, the defect matrix, and the placement position parameters.
[0026] The actual reward value is determined based on the predicted probability value;
[0027] The error value is determined based on the estimated reward value and the actual reward value;
[0028] The error values are propagated to the policy function and value function for gradient calculation to optimize the reinforcement learning model.
[0029] Optionally, the output may include predicted defect areas;
[0030] The actual reward value is determined based on the predicted probability value, including:
[0031] Determine the first image area of the defective image and the second image area of the defective region in the synthesized defective image;
[0032] Determine the overlap of defect regions based on the predicted defect regions and the defect regions in the synthetic defect images;
[0033] The actual reward value is determined based on the area of the first image, the area of the second image, the overlap of the defect area, and the predicted probability value.
[0034] Furthermore, to achieve the above objectives, the present invention also proposes a defect detection device, the device comprising:
[0035] The input module is used to input the defect image and the original image of the product into the reinforcement learning model respectively, and obtain the synthetic defect image output by the reinforcement learning model, which serves as the effective synthetic defect image of the target detection model trained in this session.
[0036] The optimization module is used to input the effectively synthesized defect image and the original image into the target detection model. When the target detection model does not meet the preset training termination condition, the reinforcement learning model is optimized based on the output result of the target detection model.
[0037] The training module is used to return the operation of inputting the defect image and the original image of the product into the reinforcement learning model respectively, and replace the reinforcement learning model with the optimized reinforcement learning model to obtain a new synthetic defect image. Based on the new synthetic defect image, the effective synthetic defect image to be used for the next training of the target detection model is determined, and the next training of the target detection model is started until the target detection model meets the preset training termination conditions.
[0038] The detection module is used to detect defects in the product using the target detection model after training.
[0039] In addition, to achieve the above objectives, the present invention also proposes a defect detection device, the device comprising: a memory, a processor, and a defect detection program stored in the memory and executable on the processor, the defect detection program being configured to implement the steps of the defect detection method described above.
[0040] In addition, to achieve the above objectives, the present invention also proposes a storage medium storing a defect detection program, which, when executed by a processor, implements the steps of the defect detection method described above.
[0041] This invention proposes a defect detection method. The method involves inputting a defect image and an original image of a product into a reinforcement learning model (LLM) to obtain a synthetic defect image output by the LLM, which serves as the effective synthetic defect image for training the target detection model. The effective synthetic defect image and the original image are then input into the target detection model. If the target detection model does not meet a preset training termination condition, the LLM is optimized based on its output. The method then returns to the previous process of inputting the defect image and the original image into the LLM, replacing the original LLM with the optimized LLM to obtain a new synthetic defect image. Based on this new synthetic defect image, the effective synthetic defect image for the next training iteration of the target detection model is determined, and the next training iteration begins until the target detection model meets the preset training termination condition. Finally, the trained target detection model is used to detect defects in the product. This invention, through the concept of reinforcement learning, transforms the optimization task of the target detection model into a reinforcement learning task based on the LLM model. By optimizing the reinforcement learning model simultaneously with the target detection model, the accuracy and usability of the target detection model can be improved with limited data. Attached Figure Description
[0042] Figure 1 This is a schematic diagram of the structure of a defect detection device for the hardware operating environment involved in the embodiments of the present invention;
[0043] Figure 2 This is a flowchart illustrating an embodiment of the defect detection method of the present invention;
[0044] Figure 3 This is a flowchart illustrating another embodiment of the defect detection method of the present invention;
[0045] Figure 4 This is a flowchart illustrating another embodiment of the defect detection method of the present invention;
[0046] Figure 5 This is a flowchart illustrating another embodiment of the defect detection method of the present invention;
[0047] Figure 6 This is a schematic diagram of the reinforcement learning model and the target detection model in one embodiment of the defect detection method of the present invention;
[0048] Figure 7 This is another schematic diagram of the reinforcement learning model and the target detection model to be trained in one embodiment of the defect detection method of the present invention;
[0049] Figure 8 This is a structural block diagram of an embodiment of the defect detection device of the present invention.
[0050] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation
[0051] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.
[0052] Reference Figure 1 , Figure 1 This is a schematic diagram of the defect detection device structure of the hardware operating environment involved in the embodiment of the present invention.
[0053] like Figure 1 As shown, the defect detection device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen or an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wireless-Fidelity (Wi-Fi) interface). The memory 1005 may be high-speed random access memory (RAM) or stable non-volatile memory (NVM), such as a disk storage device. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.
[0054] Those skilled in the art will understand that Figure 1 The structure shown does not constitute a limitation on the defect detection equipment and may include more or fewer components than shown, or combine certain components, or have different component arrangements.
[0055] like Figure 1 As shown, the memory 1005, which serves as a storage medium, may include an operating system, a network communication module, a user interface module, and a defect detection program.
[0056] exist Figure 1In the defect detection device shown, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 can be set in the defect detection device. The defect detection device calls the defect detection program stored in the memory 1005 through the processor 1001 and executes the defect detection method provided in the embodiment of the present invention.
[0057] This invention provides a defect detection method, referring to... Figure 2 , Figure 2 This is a flowchart illustrating an embodiment of the defect detection method of the present invention.
[0058] In this embodiment, the defect detection method includes the following steps:
[0059] Step S10: Input the defect image and the original image of the product into the reinforcement learning model respectively, and obtain the synthetic defect image output by the reinforcement learning model, which serves as the effective synthetic defect image for this training of the target detection model.
[0060] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, or an electronic device, defect detection device, etc., capable of performing the above functions. The following uses a defect detection device as an example to illustrate this embodiment and the subsequent embodiments.
[0061] It is understandable that a defective image can be an image containing a defective region segmented from an original image; the original image can be a product image with defects; a synthesized defective image can be an image containing a defective region synthesized by a reinforcement learning model based on a defective image and an original image; an effective synthesized defective image can be a synthesized defective image that meets preset conditions, which can be set according to specific scenarios.
[0062] As one implementation method, the input of the reinforcement learning model includes two types of images: original images extracted from the original image dataset and defect images extracted from the defect image dataset. The defect images and the corresponding original images are input into the reinforcement learning model respectively to obtain the synthetic defect images output by the reinforcement learning model based on the defect images and the original images.
[0063] Step S20: Input the effectively synthesized defect image and the original image into the target detection model. If the target detection model does not meet the preset training termination condition, optimize the reinforcement learning model based on the output of the target detection model.
[0064] It is understandable that the object detection model can be a model that needs to be optimized and trained to detect defects in a product; the output result can be the prediction result of the effective synthetic defect image output by the object detection model; the implementation method of optimizing the reinforcement learning model based on the output result can be: determining the error of the reinforcement learning model based on the prediction result output by the object detection model, and optimizing the reinforcement learning model based on the error.
[0065] As one implementation method, the effectively synthesized defect image and the original image are input into the target detection model. If the target detection model does not meet the preset training termination condition, the error of the reinforcement learning model is determined based on the prediction result of the effectively synthesized defect image output by the target detection model, and the reinforcement learning model is optimized based on the error.
[0066] Step S30: Return to the operation of inputting the defect image and the original image of the product into the reinforcement learning model respectively, and replace the reinforcement learning model with the optimized reinforcement learning model to obtain a new synthetic defect image. Based on the new synthetic defect image, determine the effective synthetic defect image to be used for the next training of the target detection model, and start the next training of the target detection model until the target detection model meets the preset training termination condition.
[0067] It is understandable that the method for determining the effective synthetic defect image to be used in the next training of the target detection model based on the new synthetic defect image can be as follows: input the new synthetic defect image into the target detection model, and determine the effective synthetic defect image to be used in the next training of the target detection model based on the prediction result output by the target detection model; the preset training termination condition can be a pre-set condition for ending the training of the target detection model, and the preset training conditions include, but are not limited to: the model training times reaching a preset number, the model accuracy being greater than a preset accuracy, etc.
[0068] Step S40: Use the target detection model after training to detect defects in the product.
[0069] One implementation involves extracting defect images and original images from the defect image dataset and the original image dataset, respectively. These images are then input into a reinforcement learning model to obtain a synthetic defect image generated by the reinforcement learning model based on the defect image and the corresponding original image. This synthetic defect image is used as the effective synthetic defect image for training the target detection model. The effective synthetic defect image and the original image are then input into the target detection model. If the target detection model does not meet the preset termination condition, the error of the reinforcement learning model is determined based on the prediction results of the effective synthetic defect image output by the target detection model. The reinforcement learning model is then optimized based on the error, and the process returns to continue, inputting the product's defect image and original image into the reinforcement learning model, replacing the original model with the optimized one. This yields a new synthetic defect image output by the optimized model. This new synthetic defect image is then input into the target detection model, and the effective synthetic defect image for the next training iteration is determined based on the prediction results output by the target detection model. The target detection model continues to be trained using the effective synthetic defect image and the original image until it meets the preset training termination condition. Finally, the trained target detection model is used to detect defects in the product.
[0070] Optionally, the object detection model used in each training session is either the initial object detection model or the object detection model obtained after the previous training.
[0071] Understandably, the initial object detection model can be an untrained object detection model.
[0072] Optionally, for an untrained object detection model, the same input image to be detected may result in different predictions. Therefore, if the predictions of the untrained object detection model are used directly to optimize the reinforcement learning model, the reinforcement learning model will fail to train. To solve the above problem, the method further includes: pre-training the initial object detection model with the original image dataset; if the loss value of the initial object detection model is less than the loss threshold during the pre-training process, the object detection model is obtained.
[0073] As one implementation method, the initial object detection model is pre-trained using the original image dataset. During the training process, if the loss value of the object detection model is less than the loss threshold, the pre-training is stopped, and the object detection model is obtained. This object detection model can initially detect defects in the image, thereby improving the accuracy of the prediction results and enhancing the optimization effect of the reinforcement learning model.
[0074] As another implementation method, the initial object detection model is pre-trained using the original image dataset. If the number of training iterations is greater than or equal to the preset number of training iterations, the object detection model is obtained.
[0075] This embodiment proposes a defect detection method. A defect image and an original image of a product are input into a reinforcement learning model to obtain a synthetic defect image output by the reinforcement learning model. This synthetic defect image serves as the effective synthetic defect image for training the target detection model. The effective synthetic defect image and the original image are then input into the target detection model. If the target detection model does not meet a preset training termination condition, the reinforcement learning model is optimized based on its output. The process is repeated, inputting the product defect image and the original image into the reinforcement learning model again, replacing the original model with the optimized one to obtain a new synthetic defect image. The new synthetic defect image is used to determine the effective synthetic defect image for the next training iteration of the target detection model, initiating the next training iteration until the target detection model meets the preset training termination condition. Finally, the trained target detection model is used to detect defects in the product. This embodiment transforms the optimization task of the target detection model into a reinforcement learning task based on the reinforcement learning model, optimizing the target detection model simultaneously with the reinforcement learning model. This achieves improved accuracy and usability of the target detection model with limited data.
[0076] refer to Figure 3 , Figure 3 This is a flowchart illustrating another embodiment of the defect detection method of the present invention.
[0077] In this embodiment, the reinforcement learning model includes a first feature extraction module, a second feature extraction module, a feature integration module, and a policy function. Step S10 includes:
[0078] Step S101: Extract features from the defect image of the product using the first feature extraction module to obtain the first feature vector.
[0079] Understandably, the first feature vector can be a vector that can characterize the features of a defective image.
[0080] Step S102: Extract features from the original image of the product using the second feature extraction module to obtain the second feature vector.
[0081] Understandably, the second feature vector can be a vector that can characterize the features of the original image.
[0082] Step S103: The first feature vector and the second feature vector are concatenated and calculated by the feature integration module to obtain the concatenated feature vector.
[0083] It is understandable that the spliced feature vector can be a vector obtained by splicing the vector representing the features of the defective image and the vector representing the features of the original image.
[0084] Step S104: Output the defect matrix and placement parameters based on the spliced feature vectors using the strategy function.
[0085] Understandably, the defect matrix can be a matrix storing the retention probability values of pixels. The higher the retention probability value, the higher the probability that the pixel is a defective pixel. The retention probability value can be a value greater than or equal to 0 and less than or equal to 1. The placement parameters can be the position parameters for placing the defective image into the original image. For example, the placement parameters include, but are not limited to: the position coordinates (x, y) of the center of the defective image relative to the original image, the ratio of the length and width of the defective image to the length and width of the original image, and the rotation angle of the defect, etc.
[0086] Step S105: Output the synthesized defect image based on the defect matrix and placement parameters.
[0087] It is understandable that the method to output a synthesized defect image based on the defect matrix and placement parameters can be: superimposing the defect image with the original image based on the defect matrix and placement parameters to output a synthesized defect image.
[0088] Optionally, in order to ensure the diversity and stability of defects in the synthesized defect image, the synthesized defect image is output according to the defect matrix and the placement position parameters, including: filtering the defect image according to the defect matrix to obtain a filtered defect image; superimposing the filtered defect image with the original image according to the placement position parameters, and outputting the superimposed image as the synthesized defect image.
[0089] It is understandable that the method of filtering the defect image based on the defect matrix to obtain the filtered defect image can be: performing a bitwise AND operation between the defect matrix and the defect image, retaining the effective part of the defect image, which corresponds to the part of the defect matrix whose retention probability value is greater than a preset probability value.
[0090] Optionally, if the defect matrix and the defect image have different sizes, the defect matrix is scaled so that the scaled defect matrix has the same size as the defect image.
[0091] Optionally, to prevent the defect matrix from tending to retain all defects in the defect image without filtering, a pixel limit is set so that the number of pixels in the defect matrix is below the pixel limit.
[0092] In one example: Let the original image dataset be Dataset1, and the corresponding label set be Label1. Based on Label1, images containing defective regions can be segmented from Dataset1. All segmented images containing defective regions are then combined into a defective image dataset, Defect1. Original images and defective images are extracted from Dataset1 and Defect1, respectively. These extracted original and defective images are then input into a reinforcement learning model. The first feature extraction module in the reinforcement learning model extracts features from the defective images to obtain a first feature vector. The second feature extraction module extracts features from the original images to obtain a second feature vector. The feature integration module concatenates the first and second feature vectors to form a concatenated feature vector. This concatenated feature vector is then input into the policy function. The function is responsible for outputting the defect matrix mask and the placement parameters, which include, but are not limited to, the position coordinates (x, y) of the center of the defect image relative to the original image, the ratios h and w of the length and width of the defect image relative to the length and width of the original image, and the rotation angle alpha of the defect image relative to the original image. If the size of the mask is inconsistent with the size of the defect image, the mask is scaled to match the size of the defect image. Assuming the size of the mask is a*b and the size of the defect image is h1*w1, the size of the mask is scaled to h1*w1. The scaled mask is then ANDed with the defect image, retaining the portion of the mask in the defect image where the value is greater than k. For example, k can be 0.8, 0.92, 0.86, or other values, which can be set according to the detection accuracy. This embodiment does not impose any restrictions on this.
[0093] It should be noted that if the reinforcement learning model is allowed to automatically output synthesized defect images, it may output defects that do not exist in reality. In other words, the defects output by the reinforcement learning model may be fabricated, offering no benefit to training and optimizing the target detection model. In this embodiment, the image containing the defect region is extracted from the original image. The defect image and the original image are used as input to the reinforcement learning model, allowing the model to output a synthesized defect image based on these images. This ensures that the defects in the synthesized defect image are objectively existing defects, guaranteeing the stability of the synthesized defect image and its usability for optimizing the target detection model. The defect image may contain non-defect components. Regions (non-defect areas, such as rectangular or circular regions, may be cropped during image extraction) are masked using a reinforcement learning model, retaining only the necessary parts, i.e., the defective image portions, while masking out the unnecessary parts. The reinforcement learning model learns which parts can be considered defects. Since the masks generated by the reinforcement learning model are not exactly the same each time, the diversity of defects in the synthesized defective image is ensured. By overlaying the filtered defective image with the original image and outputting the overlaid image as the synthesized defective image, it is ensured that the defective regions in the generated defective image are distributed in the same way as the defective regions in the original image, thus improving the realism of the generated defective image.
[0094] This embodiment transforms the task of outputting a synthetic defect image from a generation task into a discrete action task that can be performed by a reinforcement learning model. This discrete action task includes two independent sub-tasks: outputting a defect matrix and placing position parameters. The output defect matrix ensures the diversity and stability of the synthetic defect image, while the placement of position parameters generates the synthetic defect image, ensuring that the synthetic defect image and the original image are identically distributed.
[0095] In another embodiment of the defect detection method proposed in this invention, referring to Figure 4 The effective synthetic defect images to be used in the next training of the target detection model are determined based on the new synthetic defect images, including:
[0096] Step S01: Determine the overlap of the defect regions based on the predicted defect regions output by the target detection model and the defect regions in the new synthetic defect image.
[0097] It is understandable that the predicted defect region can be the defect region in the synthetic defect image predicted by the target detection model; the defect region overlap can also be called the Intersection over Union (IOU). IOU calculates the overlap rate between the "predicted bounding box" and the "real bounding box", that is, the ratio of their intersection to their union.
[0098] As one approach, the overlap of the defect region can be calculated based on the defect location and area information output by the detection model of the target to be detected and the defect location and area in the synthesized defect image.
[0099] Step S02: If the predicted probability value of the new synthetic defect image output by the target detection model is greater than the preset probability value and the overlap of the defect region is greater than the preset overlap, then the new synthetic defect image is determined as a valid synthetic defect image to be used in the next training of the target detection model.
[0100] In one example, the overlap between the predicted defect region and the defect region in the new synthetic defect image is m, and the predicted probability value output by the object detection model is n. Assuming the preset overlap is m0 and the preset probability value is n0, if m>m0 and n>n0, then the new synthetic defect image is determined to be the effective synthetic defect image used for the next training of the object detection model. The effective synthetic defect image dataset can also be set as Dataset2, and the effective synthetic defect images in this set can be denoted as img2, and the corresponding label set is Label2.
[0101] In this embodiment of the invention, the overlap of defect regions is determined based on the predicted defect region output by the target detection model and the defect region in the new synthetic defect image. If the predicted probability value of the new synthetic defect image output by the target detection model is greater than a preset probability value and the overlap of the defect regions is greater than a preset overlap, then the new synthetic defect image is determined to be a valid synthetic defect image for the next training of the target detection model. By using synthetic defect images with a probability defect value greater than a preset probability value and a defect region overlap greater than a preset overlap as valid synthetic defect images for the next training of the target detection model, the accuracy of model training can be improved.
[0102] In another embodiment of the defect detection method proposed in this invention, the reinforcement learning model further includes a value function, and the output includes a predicted probability value.
[0103] Reference Figure 5 Optimizing the reinforcement learning model based on the output of the object detection model includes:
[0104] Step S001: Output the reward estimate using the value function based on the spliced feature vector, defect matrix, and placement position parameters.
[0105] In this embodiment, the value function outputs a reward estimate based on the concatenated feature vector, the defect matrix, and the placement position parameter.
[0106] Step S002: Determine the actual reward value based on the predicted probability value.
[0107] Step S003: Determine the error value based on the estimated reward value and the actual reward value.
[0108] Understandably, the error between the estimated reward and the actual reward can be calculated using a preset error function.
[0109] Step S004: Propagate the error value to the policy function and value function for gradient calculation to optimize the reinforcement learning model.
[0110] Optionally, in order to transform the optimization task of the object detection model into a reinforcement learning task based on the reinforcement learning model, the output includes predicting the defect region; determining the actual reward value based on the predicted probability value, including: determining the first image area of the defect image and the second image area of the defect region in the synthesized defect image; determining the defect region overlap based on the predicted defect region and the defect region in the synthesized defect image; and determining the actual reward value based on the first image area, the second image area, the defect region overlap, and the predicted probability value.
[0111] It is understandable that the area of the first image can be the area of the defective image; the area of the second image can be the area of the synthesized defective region in the synthesized defective image.
[0112] In one example, refer to Figure 6 , Figure 6 The diagram illustrates the reinforcement learning model and the object detection model. S10': Input the defect image and the original image into the reinforcement learning model to obtain a synthetic defect image output by the reinforcement learning model, which serves as the valid synthetic defect image for this training of the object detection model. S20': Input the valid synthetic defect image and the original image into the object detection model. If the object detection model does not meet the preset training termination condition, optimize the reinforcement learning model based on the output result of the object detection model. S30': Continue executing the operation of inputting the defect image and the original image into the reinforcement learning model, and replace the reinforcement learning model with the optimized reinforcement learning model to obtain a new synthetic defect image output by the optimized reinforcement learning model. Determine the valid synthetic defect image for the next training of the object detection model based on the new synthetic defect image, and start the next training of the object detection model. S40': Use the object detection model that meets the preset training termination condition to perform defect detection on the product.
[0113] In another example, refer to Figure 7 , Figure 7The diagram illustrates the reinforcement learning model and the target detection model to be trained. Step ①: Input the defect image into the first feature extraction module of the reinforcement learning model to obtain the first feature vector; Step ②: Input the original image into the second feature extraction module of the reinforcement learning model to obtain the second feature vector; Step ③: The feature integration module in the reinforcement learning model concatenates the first and second feature vectors to obtain a concatenated feature vector; Step ④: The concatenated feature vector is transmitted to the policy function, which outputs a defect matrix and placement parameters based on the concatenated feature vector; Step ⑤: Output a synthesized defect image based on the defect matrix and placement parameters, serving as the effective synthesized defect image for this training of the target detection model; Step ⑥: Input the effective synthesized defect image and the original image into the target detection model to obtain the predicted probability value output by the target detection model; Step ⑦: ... If the target detection model does not meet the preset training termination condition, the actual reward value is determined based on the predicted probability value output by the target detection model. The reward estimate is determined based on the actual probability value and the value function, the feature concatenation vector, the defect matrix, and the placement position parameters. The error value is then propagated to the policy function and the value function for gradient calculation to optimize the reinforcement learning model. The original reinforcement learning model is replaced with the optimized reinforcement learning model. Step ① is then executed to obtain a new synthetic defect image. Based on the new synthetic defect image, the effective synthetic defect image to be used in the next training of the target detection model is determined. The next training of the target detection model is then started until the target detection model meets the preset training termination condition. The target detection model after training is then used to perform defect detection on the product. The actual reward value can be determined by formula (1):
[0114]
[0115] In the formula, `score` is the actual reward value; `origin_defect_area` is the area of the defect image. Assuming the length and width of the defect image are h1 and w1 respectively, then `origin_defect_area = h1 * w1`; `resize_defect_area` is the area of the synthesized defect region in the synthesized defect image. Assuming the length and width of the original image are h2 and w2 respectively, then the ratio of the length and width of the defect image to the length and width of the original image is h and w, then `resize_defect_area = h2 * w2 * h * w`; `possibility` is the predicted probability value. Assuming the reward estimate output by the value function is `score_P` and the actual reward value is `score`, then the error value can be calculated using the error function based on the reward estimate and the actual reward value. For example, the error function includes, but is not limited to: `Loss = (socre - socre_P)`. 2Loss = |socre - socre_P|. The error function can be set to other forms according to the specific scenario. Since the value function itself is based on the estimation of the defect matrix output by concatenating the feature vector and the policy function and putting in the position parameters, the error calculation of the value function can also be transformed into the calculation of the error generated by the policy function output, thereby optimizing the policy function.
[0116] In this embodiment, the actual reward value is determined based on the predicted probability value output by the target detection model, and the error value is determined based on the actual reward value and the reward estimate output by the value function. The error value is then propagated to the policy function and the value function for gradient calculation to optimize the reinforcement learning model. This approach optimizes the target detection model while simultaneously optimizing the reinforcement learning model, thereby achieving high accuracy for the target detection model even with limited data and improving the model's usability.
[0117] Furthermore, this embodiment of the invention also proposes a storage medium storing a defect detection program, which, when executed by a processor, implements the steps of the defect detection method described above.
[0118] Reference Figure 8 , Figure 8 This is a structural block diagram of an embodiment of the defect detection device of the present invention.
[0119] like Figure 8 As shown, the defect detection device proposed in this embodiment of the invention includes:
[0120] Input module 10 is used to input the defect image and the original image of the product into the reinforcement learning model respectively, and obtain the synthetic defect image output by the reinforcement learning model as the effective synthetic defect image of the target detection model trained in this training.
[0121] Optimization module 20 is used to input the effective synthesized defect image and the original image into the target detection model. When the target detection model does not meet the preset training termination condition, the reinforcement learning model is optimized according to the output result of the target detection model.
[0122] The training module 30 is used to return the operation of inputting the defect image and the original image of the product into the reinforcement learning model respectively, and replace the reinforcement learning model with the optimized reinforcement learning model to obtain a new synthetic defect image. Based on the new synthetic defect image, the effective synthetic defect image to be used for the next training of the target detection model is determined, and the next training of the target detection model is started until the target detection model meets the preset training termination condition.
[0123] The detection module 40 is used to detect defects in the product using the target detection model after training.
[0124] This invention proposes a defect detection device. A defect image and an original image of a product are input into a reinforcement learning model to obtain a synthetic defect image output by the reinforcement learning model. This synthetic defect image serves as the effective synthetic defect image for training the target detection model. The effective synthetic defect image and the original image are then input into the target detection model. If the target detection model does not meet a preset training termination condition, the reinforcement learning model is optimized based on its output. The process returns to inputting the product defect image and the original image into the reinforcement learning model, replacing the original model with the optimized one to obtain a new synthetic defect image. Based on this new synthetic defect image, the effective synthetic defect image for the next training of the target detection model is determined, and the next training of the target detection model begins until the target detection model meets the preset training termination condition. The trained target detection model is then used to detect defects in the product. This invention, through the concept of reinforcement learning, transforms the optimization task of the target detection model into a reinforcement learning task based on the reinforcement learning model. By optimizing the reinforcement learning model simultaneously with the target detection model, the accuracy and usability of the target detection model can be improved with limited data.
[0125] Based on one embodiment of the defect detection device of the present invention, another embodiment of the defect detection device of the present invention is proposed.
[0126] In this embodiment, the input module 10 is further configured to: extract features from the defect image of the product using the first feature extraction module to obtain a first feature vector; extract features from the original image of the product using the second feature extraction module to obtain a second feature vector; perform concatenation calculation on the first and second feature vectors using the feature integration module to obtain a concatenated feature vector; output a defect matrix and placement parameters based on the concatenated feature vector using a policy function; and output a synthesized defect image based on the defect matrix and placement parameters. The reinforcement learning model includes the first feature extraction module, the second feature extraction module, the feature integration module, and the policy function.
[0127] The input module 10 is also used to filter the defect image according to the defect matrix to obtain a filtered defect image; to superimpose the filtered defect image with the original image according to the placement position parameters, and to output the superimposed image as a synthesized defect image; and to output the synthesized defect image according to the defect matrix and the placement position parameters.
[0128] The training module 30 is further configured to determine the overlap of defect regions based on the predicted defect regions output by the target detection model and the defect regions in the new synthetic defect image; if the predicted probability value of the new synthetic defect image output by the target detection model is greater than a preset probability value and the overlap of the defect regions is greater than a preset overlap, then the new synthetic defect image is determined to be a valid synthetic defect image for the next training of the target detection model.
[0129] The input module 10 is also used to output a reward estimate based on the concatenated feature vector, defect matrix, and placement position parameters through a value function; determine the actual reward value based on the predicted probability value; determine the error value based on the reward estimate and the actual reward value; and propagate the error value to the policy function and the value function for gradient calculation to optimize the reinforcement learning model. The reinforcement learning model also includes a value function, and the output includes the predicted probability value.
[0130] The input module 10 is also used to determine the first image area of the defect image and the second image area of the defect region in the synthesized defect image; determine the overlap of the defect region based on the predicted defect region and the defect region in the synthesized defect image; determine the actual reward value based on the first image area, the second image area, the overlap of the defect region and the predicted probability value; and the output result includes the predicted defect region.
[0131] The device also includes a preheating training module 50, which is used to preheat the initial object detection model using the original image dataset; if the loss value of the initial object detection model is less than the loss threshold during the preheating training process, the object detection model is obtained.
[0132] Other embodiments or specific implementations of the defect detection device of the present invention can be referred to the above-described method embodiments, and will not be repeated here.
[0133] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.
[0134] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.
[0135] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as read-only memory / random access memory, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods of the various embodiments of the present invention.
[0136] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.
Claims
1. A defect detection method, characterized in that, The method includes: The defect image and the original image of the product are respectively input into the reinforcement learning model to obtain the synthetic defect image output by the reinforcement learning model, which is used as the effective synthetic defect image of the target detection model trained in this training. The effective synthetic defect image and the original image are input into the target detection model. If the target detection model does not meet the preset training termination condition, the reinforcement learning model is optimized based on the output result of the target detection model. Return to the operation of inputting the product defect image and the original image into the reinforcement learning model respectively, and replace the reinforcement learning model with the optimized reinforcement learning model to obtain a new synthetic defect image. Determine the overlap of the defect regions based on the predicted defect regions output by the target detection model and the defect regions in the new synthetic defect image. If the predicted probability value of the new synthetic defect image output by the target detection model is greater than the preset probability value and the overlap of the defect region is greater than the preset overlap, then the new synthetic defect image is determined to be a valid synthetic defect image for the next training of the target detection model, and the next training of the target detection model is started until the target detection model meets the preset training termination condition. The target detection model after training is used to detect defects in the product.
2. The method as described in claim 1, characterized in that, The object detection model used in each training session is either the initial object detection model or the object detection model obtained after the previous training.
3. The method as described in claim 1, characterized in that, The reinforcement learning model includes a first feature extraction module, a second feature extraction module, a feature integration module, and a policy function; The step of inputting the defect image and the original image of the product into the reinforcement learning model respectively to obtain the synthetic defect image output by the reinforcement learning model includes: The first feature extraction module extracts features from the defect image of the product to obtain a first feature vector. The second feature extraction module extracts features from the original image of the product to obtain a second feature vector. The feature integration module performs concatenation calculations on the first feature vector and the second feature vector to obtain a concatenated feature vector. The strategy function outputs the defect matrix and placement parameters based on the spliced feature vector; A synthesized defect image is output based on the defect matrix and the placement position parameters.
4. The method as described in claim 3, characterized in that, The step of outputting a synthesized defect image based on the defect matrix and the placement position parameters includes: The defect image is filtered according to the defect matrix to obtain a filtered defect image; The filtered defect image is superimposed on the original image according to the placement parameters, and the superimposed image is output as a synthetic defect image.
5. The method as described in claim 3, characterized in that, The reinforcement learning model also includes a value function, and the output includes a predicted probability value. The step of optimizing the reinforcement learning model based on the output of the target detection model includes: The value function outputs a reward estimate based on the spliced feature vector, the defect matrix, and the placement position parameter. The actual reward value is determined based on the predicted probability value; The error value is determined based on the estimated reward value and the actual reward value; The error value is propagated to the policy function and the value function for gradient calculation to optimize the reinforcement learning model.
6. The method as described in claim 5, characterized in that, The output includes predicted defect areas; The step of determining the actual reward value based on the predicted probability value includes: Determine the first image area of the defect image and the second image area of the defect region in the synthesized defect image; The overlap of the defect regions is determined based on the predicted defect regions and the defect regions in the synthesized defect image. The actual reward value is determined based on the area of the first image, the area of the second image, the overlap of the defective region, and the predicted probability value.
7. A defect detection device, characterized in that, The device includes: The input module is used to input the defect image and the original image of the product into the reinforcement learning model respectively, and obtain the synthetic defect image output by the reinforcement learning model, which serves as the effective synthetic defect image of the target detection model trained in this exercise. The optimization module is used to input the effective synthesized defect image and the original image into the target detection model. When the target detection model does not meet the preset training termination condition, the reinforcement learning model is optimized according to the output result of the target detection model. The training module is used to return the operation of inputting the defect image and the original image of the product into the reinforcement learning model respectively, and replace the reinforcement learning model with the optimized reinforcement learning model to obtain a new synthetic defect image. The overlap of the defect regions is determined based on the predicted defect region output by the target detection model and the defect region in the new synthetic defect image. If the predicted probability value of the new synthetic defect image output by the target detection model is greater than a preset probability value and the overlap of the defect regions is greater than a preset overlap, then the new synthetic defect image is determined to be a valid synthetic defect image for the next training of the target detection model, and the next training of the target detection model is started until the target detection model meets the preset training termination condition. The detection module is used to detect defects in the product using the target detection model after training.
8. A defect detection device, characterized in that, The device includes: a memory, a processor, and a defect detection program stored in the memory and executable on the processor, the defect detection program being configured to implement the steps of the defect detection method as described in any one of claims 1 to 6.
9. A storage medium, characterized in that, The storage medium stores a defect detection program, which, when executed by a processor, implements the steps of the defect detection method as described in any one of claims 1 to 6.