Defect detection method, device and equipment based on visual detection and storage medium

Abstract

The application discloses a kind of detection methods, devices and equipment based on visual detection defect and storage medium based on visual detection defect, involve image processing technical field, discloses a kind of detection methods based on visual detection defect includes: obtaining the image data to be handled and carries out data preprocessing, obtains defect image data;Defect detection model is based on defect image data and carries out defect detection processing, obtains surface defect detection data, loss function is added in defect detection model, and defect detection model is obtained by improving to YOLO model;Surface defect detection data is screened, and surface defect detection result is obtained.The application improves the accuracy of defect detection.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method, apparatus, device and storage medium for detecting defects based on visual inspection. Background Technology

[0002] In the manufacturing industry, defect detection is a crucial step. Traditional manual inspection methods are inefficient and prone to errors. With the development of computer vision technology, automated visual inspection methods are gradually becoming mainstream. However, existing automated inspection methods still face challenges when handling complex scenes and imbalanced datasets.

[0003] In recent years, the rise of deep learning technology has brought revolutionary improvements to visual inspection. Convolutional neural networks, in particular, have achieved remarkable results in image recognition and classification tasks and are widely used in defect detection. However, in existing deep learning-based defect detection methods, positive samples (defects) are typically far fewer than negative samples (no defects), leading to insufficient attention to the minority class during model training and consequently reducing the accuracy of defect detection. Existing deep learning-based methods also suffer from low localization accuracy, further impacting detection accuracy. Therefore, it is clear that current deep learning-based methods still suffer from relatively low defect detection accuracy.

[0004] Therefore, improving the accuracy of defect detection is a problem that urgently needs to be solved. Summary of the Invention

[0005] The main objective of this application is to provide a visual defect detection method, apparatus, device, and storage medium, aiming to solve the technical problem of how to improve the accuracy of defect detection.

[0006] To achieve the above objectives, this application proposes a visual defect detection method, the method comprising:

[0007] The image data to be processed is acquired and preprocessed to obtain defect image data;

[0008] The defect image data is processed based on the defect detection model to obtain surface defect detection data. The defect detection model is an improved version of the YOLO (You Only Look Once) model.

[0009] The surface defect detection data is filtered to obtain the surface defect detection results.

[0010] In one embodiment, the loss function includes the SIoU (Symmetric Intersection over Union) function, which is used to calculate the localization loss. The step of performing defect detection processing on the image data based on the defect detection model includes:

[0011] Based on the defective image data, a predicted bounding box and a preset ground truth bounding box are obtained;

[0012] The SIoU loss value is obtained by comparing the predicted bounding box with the preset ground truth bounding box using the SIoU function;

[0013] During defect detection processing, the loss weight of the SIoU loss value in the positioning loss calculation is increased.

[0014] In one embodiment, the step of increasing the loss weight of the SIoU loss value in the positioning loss calculation further includes:

[0015] The weight penalty term is calculated using a preset optimizer;

[0016] The SIoU loss value is optimized by using the weight penalty term to obtain the optimized SIoU loss value.

[0017] In one embodiment, the loss function includes a Focal Loss function, and the step of performing defect detection processing on the image data based on the defect detection model includes:

[0018] Calculate the binary cross-entropy loss value based on the defective image data;

[0019] The true class probability is calculated using the binary cross-entropy loss value.

[0020] The Focal Loss function is obtained by calculating the true class probability. The Focal Loss function includes a first parameter and a second parameter, which are adjustable. The first parameter is a coefficient parameter and the second parameter is an exponential parameter.

[0021] During defect detection processing, the defect image data is classified using the Focal Loss function.

[0022] In one embodiment, the surface defect detection data includes confidence levels, and the step of filtering the surface defect detection data to obtain surface defect detection results further includes:

[0023] If the confidence level of the surface defect detection data is higher than the first confidence level threshold, then the surface defect detection data will be output as the surface defect detection result.

[0024] If the confidence level of the surface defect detection data is higher than the second confidence threshold, then the surface defect detection data will be stored as data to be identified, where the second confidence threshold is lower than the first confidence threshold.

[0025] In one embodiment, the step of acquiring the image data to be processed and performing data preprocessing to obtain defective image data further includes:

[0026] The image data is converted to grayscale to obtain grayscale image data;

[0027] The grayscale image data is binarized to obtain the defect image data, which is image data containing both black and white pixels.

[0028] In one embodiment, the step of acquiring the image data to be processed further includes:

[0029] The product under test is illuminated by a light box, and image data of the product under test is acquired by a camera to obtain image data to be processed. The light box consists of a box body, a light source and a light-transmitting panel.

[0030] Furthermore, to achieve the above objectives, this application also proposes a defect detection device, which includes:

[0031] The data acquisition module is used to acquire the image data to be processed and perform data preprocessing to obtain defect image data;

[0032] The data processing module is used to perform defect detection processing on the defect image data based on the defect detection model to obtain surface defect detection data. The defect detection model is an improved version of the YOLO model and includes a loss function.

[0033] The data filtering module is used to filter the surface defect detection data to obtain the surface defect detection results.

[0034] In addition, to achieve the above objectives, this application also proposes a defect detection device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the visual defect detection method described above.

[0035] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the visual defect detection method described above.

[0036] This application provides a visual defect detection method. First, it preprocesses the image data to obtain defect image data, simplifying the image data while improving its recognizability and making defect areas more obvious, thus facilitating subsequent defect detection. Then, it uses an improved defect detection model to process the defect image data, obtaining surface defect detection data. This improved model, which incorporates a loss function, solves the problems of class imbalance and localization accuracy, thereby improving the accuracy of defect detection. Finally, it filters the surface defect detection data to obtain the surface defect detection results, ensuring that the output surface defect detection results have high accuracy.

[0037] In summary, this application overcomes the technical shortcomings of class imbalance and insufficient localization accuracy in the YOLO model by preprocessing image data, performing defect detection processing on defect image data through an improved defect detection model, and finally filtering the results, thereby improving the accuracy of defect detection. Attached Figure Description

[0038] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0039] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 This is a schematic flowchart of an embodiment of the visual defect detection method of this application;

[0041] Figure 2 This is a schematic flowchart of an embodiment of the visual defect detection method of this application;

[0042] Figure 3 A schematic diagram of a scenario for a vision-based defect detection method provided in an embodiment of this application;

[0043] Figure 4 This is a schematic diagram of the module structure of the defect detection device according to an embodiment of this application;

[0044] Figure 5 This is a schematic diagram of the hardware operating environment involved in the visual defect detection method in this application embodiment.

[0045] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0046] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0047] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0048] The main solution of this application embodiment is: to acquire image data to be processed and perform data preprocessing to obtain defect image data; to perform defect detection processing on the defect image data based on a defect detection model to obtain surface defect detection data, wherein a loss function is added to the defect detection model, and the defect detection model is an improvement of the YOLO model; and to filter the surface defect detection data to obtain surface defect detection results.

[0049] Currently, in existing deep learning-based defect detection methods, positive samples (defects) are typically far fewer than negative samples (no defects), leading to insufficient attention to the minority class during model training and consequently reducing defect detection accuracy. Existing deep learning-based methods also suffer from low localization accuracy, further impacting detection accuracy. Therefore, it is clear that current deep learning-based methods still suffer from relatively low defect detection accuracy.

[0050] This application overcomes the technical shortcomings of class imbalance and insufficient localization accuracy in the YOLO model by preprocessing image data, performing defect detection processing on defect image data through an improved defect detection model, and finally filtering the results, thereby improving the accuracy of defect detection.

[0051] In this embodiment, for ease of description, the following description uses a defect detection device as the execution subject.

[0052] Based on this, embodiments of this application provide a method for detecting defects based on visual inspection, referring to... Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of the visual defect detection method of this application.

[0053] In this embodiment, the visual defect detection method includes steps S10 to S30:

[0054] Step S10: Obtain the image data to be processed and perform data preprocessing to obtain defect image data;

[0055] It should be noted that the image data to be processed refers to the raw image information obtained from data collection of the product under test, i.e., the image data that has not been processed, while the defective image data refers to the image information that has undergone data preprocessing.

[0056] Understandably, the original image information contains a large amount of data, and the defects on the product under test are generally small, such as cracks or oil stains. Therefore, the computational load for defect detection in the image data to be processed is large, and the accuracy cannot be guaranteed. So, step S10 can reduce the computational load of defect detection and improve the quality of image data, thereby improving the accuracy of defect detection.

[0057] In one feasible implementation, the data preprocessing step in step S10 may include steps A11 to A12:

[0058] Step A11: Convert the image data to grayscale to obtain grayscale image data;

[0059] It should be noted that grayscale image data refers to a data format in which image data to be processed is converted. The pixel value of each pixel in grayscale image data is represented as the grayscale level, and the pixel value generally varies from 0 (black) to 255 (white).

[0060] It is understandable that grayscale can be achieved through processing methods such as linear weighting, averaging, brightness methods, and contrast-sensitive methods. Moreover, color information is basically not needed in defect detection. Therefore, using grayscale can reduce the amount of data processing and highlight defects such as scratches and wear.

[0061] Step A12: Binarize the grayscale image data to obtain defect image data, which is image data containing both black and white pixels.

[0062] It should be noted that defective image data refers to image information after binarization processing. Binarization is the process of converting grayscale image data into black and white pixels.

[0063] Understandably, in a binarized image, white pixels usually represent the foreground (defects), while black pixels represent the background (non-defect areas). Alternatively, black pixels can represent the foreground (defects), and white pixels can represent the background (non-defect areas). The key factor is the setting of the global threshold, which refers to whether to set it as the minimum threshold for white or black.

[0064] In one feasible embodiment, binarization is performed using the Otsu algorithm. First, a histogram of the grayscale image data is calculated, representing the frequency of each grayscale level. Then, the inter-class variance is initialized and calculated. Inter-class variance refers to the weighted sum of the variances of the foreground and background after the image is divided into foreground and background using a certain threshold. Then, different global thresholds are used to calculate the inter-class variance and find the optimal global threshold.

[0065] The method in this embodiment achieves dynamic control of the global threshold. Compared with a fixed global threshold, the defect detection method in this embodiment can better adapt to defect detection under various lighting conditions, improve the robustness of the defect detection method in this embodiment, and highlight the defects in white or black through grayscale and binary processing, thereby making subsequent image analysis, such as feature extraction, measurement and classification, easier.

[0066] In one feasible implementation, the step of acquiring the image data to be processed in step S10 may include step B11:

[0067] Step B11: Illuminate the product under test through the light box and acquire image data of the product under test through the camera to obtain image data to be processed. The light box consists of a box body, a light source and a light-transmitting panel.

[0068] It should be noted that a lightbox is a type of lighting device, commonly used in advertising, displays, photography, and other fields. It consists of a housing, a light source, and a translucent panel. The working principle of a lightbox is to use the light source to evenly illuminate the translucent panel, thereby producing bright and uniform light.

[0069] Understandably, the lightbox contains multiple light sources to facilitate preprocessing of image data. By adjusting the light source, defects in the preprocessed defect image data can be made more accurate and obvious.

[0070] In this embodiment, the combination of a lightbox and a camera solves the problem of unclear defect display or unclear images after preprocessing, thereby improving the robustness and intelligence of the defect prediction and detection method in this embodiment.

[0071] Step S20: Based on the defect detection model, the defect image data is processed to obtain surface defect detection data. A loss function is added to the defect detection model, which is an improvement on the YOLO model.

[0072] It should be noted that defect detection processing refers to the data processing operations performed by the defect detection model during defect detection. For example, it involves using a loss function to reduce the error in surface defect detection data and decrease the complexity of the model. Surface defect detection data refers to the predicted bounding box data containing defects obtained after defect detection processing.

[0073] Understandably, the mainstream YOLO model uses the CIoU (Complete Intersection over Union) loss function to improve bounding box regression. CIoU loss takes into account the alignment and scale of the bounding boxes, and is an improvement over the traditional IoU (Intersection over Union) loss.

[0074] This embodiment improves the YOLO model by introducing a new loss function, optimizing the YOLO model in multiple aspects, such as classification loss and localization loss, thereby improving the accuracy and speed of defect detection.

[0075] In one feasible implementation, the loss function includes the SIoU function, and step S20 may include steps C11 to C13:

[0076] Step C11: Based on the defect image data, obtain the predicted bounding box and the preset ground truth bounding box;

[0077] It should be noted that the predicted bounding box refers to the predicted location and extent of defects during defect detection, while the preset true bounding box refers to the rectangular box manually drawn in the image by the data labeler.

[0078] In one feasible embodiment, during the model training phase, annotators can manually annotate preset ground truth boxes. In another feasible embodiment, during the pipeline detection phase, the accuracy of the predicted boxes can be evaluated by using pre-collected annotated test sets or validation sets as preset ground truth boxes.

[0079] Step C12: The predicted bounding box and the preset ground truth bounding box are compared using the SIoU function to obtain the SIoU loss value;

[0080] It should be noted that the SIoU loss value refers to the loss value calculated using the SIoU function.

[0081] Understandably, the SIoU function calculates the SIoU loss value by comprehensively considering four aspects: angle, distance, shape, and IoU.

[0082] Step C13: During defect detection processing, increase the loss weight of SIoU loss value in the location loss calculation.

[0083] It should be noted that the loss weight refers to the proportion of the SIoU loss value when calculating the localization loss in the defect detection model.

[0084] It is understandable that adjusting the loss weight to 0.6 can ensure that while reducing positional errors, it prevents erroneous penalties for predicting more accurate bounding boxes. More accurate bounding boxes are those with errors but which do not affect subsequent defect calculations.

[0085] In this embodiment, by increasing the loss weight to penalize prediction errors, the problem of large differences between the predicted box and the actual defect location during defect detection is solved, which can reduce the impact of location errors on defect detection.

[0086] In one feasible implementation, step C13 is followed by steps C130 to C131:

[0087] Step C130: Calculate the weight penalty term using the preset optimizer;

[0088] Step C131: Optimize the SIoU loss value using a weight penalty term to obtain the optimized SIoU loss value.

[0089] It's important to note that the pre-configured optimizer refers to an optimization algorithm instance configured before model training begins, such as SGD (Stochastic Gradient Descent), Adam (Adaptive Moment Estimation), and RMSprop (Root Mean Square Propagation). The weight penalty term refers to parameters used to prevent overfitting, such as L2 regularization and L1 regularization.

[0090] In one feasible embodiment, the trainable parameters in the defect detection model are updated by creating an Adam optimizer to optimize the loss weights. The weight decay and learning rate are set, wherein the weight decay is set to 1e-5 and the learning rate is set to 0.001.

[0091] Understandably, the learning rate controls the step size of each update to the trainable parameters. A smaller learning rate may require more iterations to converge, while a larger learning rate may lead to an unstable training process. The weight decay term, also known as the L2 regularization term, prevents overfitting by adding a penalty term proportional to the square of the weights to the SIoU loss function.

[0092] This embodiment prevents overfitting of the defect detection model by introducing a weight penalty term, such as an L2 regularization term, into the SIoU function, and reduces the complexity of the defect detection model by adjusting the weight decay value.

[0093] In another feasible implementation, the loss function includes the Focal Loss function, and step S20 may include steps D11 to D14:

[0094] Step D11: Calculate the binary classification cross-entropy loss value based on the defect image data;

[0095] It should be noted that the binary cross-entropy loss value is an indicator used to measure the difference between the probability distribution predicted by the binary classification model and the actual label distribution.

[0096] Step D12: Calculate the true class probability using the binary cross-entropy loss value;

[0097] It should be noted that the true class probability refers to the probability value of the target class (predicted box) calculated based on the true labels (preset true boxes). In binary or multi-class classification problems, the true class probability is used to evaluate the accuracy of the model's predictions and serves as the basis for calculating the loss function.

[0098] Step D13: The Focal Loss function is obtained by calculating the true class probability. The Focal Loss function includes a first parameter and a second parameter, which are adjustable. The first parameter is a coefficient parameter and the second parameter is an exponential parameter.

[0099] It should be noted that the Focal Loss function is a loss function used to solve the class imbalance problem in defect detection.

[0100] Understandably, the functional expression of the Focal Loss function is:

[0101]

[0102] Where α is the first parameter, γ is the second parameter, p is the true class probability, and y refers to the class of the predicted box. When y=1, the predicted box contains the defect, and other y values ​​contain the predicted box that does not contain the defect.

[0103] Furthermore, the Focal Loss function is used to address class imbalance and the problem of too many easily classifiable samples in classification problems. By improving the cross-entropy loss, an adjustment factor α×(1-p)γ is introduced to reduce the weight of easily classifiable samples (defect-free image data), causing the model to focus more on difficult-to-classify samples (defective image data) during training. However, when using the YOLO model for defect detection, since defects are a rare sample relative to defect-free samples, defects are equivalent to difficult-to-classify samples, while defect-free samples are equivalent to easily classifiable samples.

[0104] Step D14: During defect detection processing, the defect image data is classified using the Focal Loss function.

[0105] It's understandable that classifying defective image data involves adjusting the weights of data containing and without defects. Defective data is considered positive samples, and undefective data is considered negative samples. Defective image data contains multiple positive and negative samples. The Focal Loss function introduces a modifier to reduce the weight of negative samples, causing the model to focus more on the difficult-to-classify positive samples during training. The penalty for positive samples can be strengthened by adjusting the parameters in Focal Loss (increasing the γ value and decreasing the α value), thereby optimizing the overall performance of the model.

[0106] This embodiment solves the class imbalance problem in defect prediction and detection by applying the Focal Loss function to the YOLO model, thereby increasing the attention paid to predicted bounding boxes containing defects by the defect detection method in this embodiment, and thus improving the accuracy of defect detection.

[0107] Step S30: Filter the surface defect detection data to obtain the surface defect detection results.

[0108] It should be noted that the surface defect detection results refer to the surface defect detection data after screening.

[0109] This embodiment provides a visual defect detection method. By preprocessing image data and performing defect detection processing on defect image data through an improved defect detection model, and finally filtering the results, it overcomes the technical defects of class imbalance and insufficient localization accuracy in the YOLO model, and improves the accuracy of defect detection.

[0110] In one feasible implementation, the surface defect detection data includes confidence levels, such as... Figure 2 Step S30 may include steps S21 to S22:

[0111] Step S21: If the confidence level of the surface defect detection data is higher than the first confidence level threshold, then the surface defect detection data is output as the surface defect detection result.

[0112] It should be noted that the first confidence threshold refers to the surface defect detection data used to screen for highly certain surface defect detection data for improved defect detection models.

[0113] In one feasible implementation, the surface defect detection data includes three detected defects. If the confidence level of one of the defects is lower than a first confidence threshold, then the surface defect detection result will only output the two defects with a confidence level higher than the first confidence threshold.

[0114] Step S22: If the confidence level of the surface defect detection data is higher than the second confidence threshold, then the surface defect detection data is stored as data to be identified. The second confidence threshold is lower than the first confidence threshold.

[0115] It should be noted that the second confidence threshold is used to screen surface defect detection data for which the improved defect detection model is not entirely certain but may still be correct. The data to be screened refers to surface defect detection data for which the improved defect detection model is not entirely certain but may still be correct.

[0116] Understandably, the data to be identified that has passed the second confidence threshold does not mean that the data to be identified is not defective, but rather that the data is uncertain. Further identification of the data to be identified can be carried out in the future, such as re-detecting defects.

[0117] In one feasible embodiment, a scene classifier can be added to the improved defect detection model. The scene classifier outputs different first confidence thresholds and second confidence thresholds for different scenes. The scene classifier determines the usage scene through preset judgment conditions. The usage scenes include scratch detection, oil stain detection, dent detection, etc. The usage scene is determined by classifying the surface defect detection data, and the first confidence threshold and second confidence threshold corresponding to the scene are adopted.

[0118] Through the mechanism of this embodiment, the improved defect detection model can be automatically and dynamically adjusted to adjust the first confidence threshold and the second confidence threshold, thereby improving the generalization ability of the improved defect detection model and reducing overfitting.

[0119] For example, to help understand the implementation flow of the visual defect detection method obtained by combining this embodiment with the above embodiments, please refer to... Figure 3 , Figure 3 A scenario illustration of a visual defect detection method is provided, specifically:

[0120] The rectangle is the prediction box, and the label of the prediction box is a combination of defect type and confidence level. Here, scratch represents scratch and oil represents oil stains.

[0121] It is understandable that the prediction boxes in the figure represent surface defect detection data. Filtering is performed by setting a first confidence threshold of 0.9 and a second confidence threshold of 0.4. The confidence level of the oil stain defect (Oil 0.27) is lower than 0.4, indicating a misjudgment of the front-facing camera in the phone, incorrectly identifying it as an oil stain defect. The surface defect detection data marked Scratch 0.87, Scratch 0.83, and Scratch 0.88 in the figure are output as surface defect detection results, while the surface defect detection data of Scratch 0.43 and Scratch 0.58 are used as data to be screened. The defect detection method in the above embodiment can be re-executed to re-examine the data to be screened, improving the accuracy of defect detection.

[0122] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the visual inspection method for defects in this application. Any simple modifications based on this technical concept are within the scope of protection of this application.

[0123] This application also provides a defect detection device, please refer to... Figure 4 The defect detection device includes:

[0124] Data acquisition module 10 is used to acquire image data to be processed and perform data preprocessing to obtain defect image data;

[0125] Data processing module 20 is used to perform defect detection processing on the defect image data based on the defect detection model to obtain surface defect detection data. The defect detection model is an improved version of the YOLO model and includes a loss function.

[0126] The data filtering module 30 is used to filter the surface defect detection data to obtain the surface defect detection results.

[0127] Optionally, the loss function includes an SIoU function, which is used to calculate the localization loss. The data processing module 20 is further used for:

[0128] Based on the defective image data, a predicted bounding box and a preset ground truth bounding box are obtained;

[0129] The SIoU loss value is obtained by comparing the predicted bounding box with the preset ground truth bounding box using the SIoU function;

[0130] During defect detection processing, the loss weight of the SIoU loss value in the positioning loss calculation is increased.

[0131] Optionally, the data processing module 20 is also used for:

[0132] The weight penalty term is calculated using a preset optimizer;

[0133] The SIoU loss value is optimized by using the weight penalty term to obtain the optimized SIoU loss value.

[0134] Optionally, the loss function includes a Focal Loss function, and the data processing module 20 is further configured to:

[0135] Calculate the binary cross-entropy loss value based on the defective image data;

[0136] The true class probability is calculated using the binary cross-entropy loss value.

[0137] The Focal Loss function is obtained by calculating the true class probability. The Focal Loss function includes a first parameter and a second parameter, which are adjustable. The first parameter is a coefficient parameter and the second parameter is an exponential parameter.

[0138] During defect detection processing, the defect image data is classified using the Focal Loss function.

[0139] Optionally, the surface defect detection data includes confidence levels, and the data filtering module 30 is further used for:

[0140] If the confidence level of the surface defect detection data is higher than the first confidence level threshold, then the surface defect detection data will be output as the surface defect detection result.

[0141] If the confidence level of the surface defect detection data is higher than the second confidence threshold, then the surface defect detection data will be stored as data to be identified, where the second confidence threshold is lower than the first confidence threshold.

[0142] Optionally, the data acquisition module 10 is also used for:

[0143] The image data is converted to grayscale to obtain grayscale image data;

[0144] The grayscale image data is binarized to obtain the defect image data, which is image data containing both black and white pixels.

[0145] Optionally, the data acquisition module 10 is also used for:

[0146] The product under test is illuminated by a light box, and image data of the product under test is acquired by a camera to obtain image data to be processed. The light box consists of a box body, a light source and a light-transmitting panel.

[0147] The defect detection device provided in this application employs the visual defect detection method described in the above embodiments, which can solve the technical problem of how to improve the accuracy of defect detection. Compared with the prior art, the beneficial effects of the defect detection device provided in this application are the same as those of the visual defect detection method provided in the above embodiments, and other technical features in the defect detection device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0148] The following is for reference. Figure 5 The diagram illustrates a structural schematic of a defect detection device suitable for implementing embodiments of this application. The defect detection device in these embodiments may include, but is not limited to, mobile terminals such as laptops and PADs (Portable Application Description: Tablet PCs), and fixed terminals such as digital TVs and desktop computers. Figure 5 The defect detection device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0149] like Figure 5 As shown, the defect detection device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the defect detection device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. Communication device 1009 allows the defect detection equipment to communicate wirelessly or wiredly with other devices to exchange data. Although the figure shows defect detection equipment with various systems, it should be understood that implementation or possession of all the systems shown is not required. More or fewer systems may be implemented alternatively.

[0150] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0151] The defect detection device provided in this application employs the visual defect detection method described in the above embodiments, which can solve the technical problem of how to improve the accuracy of defect detection. Compared with the prior art, the beneficial effects of the defect detection device provided in this application are the same as those of the visual defect detection method provided in the above embodiments, and other technical features of this defect detection device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0152] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0153] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0154] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the visual defect detection method in the above embodiments.

[0155] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0156] The aforementioned computer-readable storage medium may be included in the defect detection equipment; or it may exist independently and not be assembled into the defect detection equipment.

[0157] The aforementioned computer-readable storage medium carries one or more programs. When the aforementioned one or more programs are executed by the defect detection device, the defect detection device causes the following actions: to acquire image data to be processed and perform data preprocessing to obtain defect image data; to perform defect detection processing on the defect image data based on a defect detection model to obtain surface defect detection data, wherein a loss function is added to the defect detection model, and the defect detection model is an improvement on the YOLO model; and to filter the surface defect detection data to obtain surface defect detection results.

[0158] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0159] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0160] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0161] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described visual defect detection method, thereby solving the technical problem of how to improve the accuracy of defect detection. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the visual defect detection method provided in the above embodiments, and will not be repeated here.

[0162] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A method for detecting defects based on visual inspection, characterized in that, The method includes: The image data to be processed is acquired and preprocessed to obtain defect image data; The defect image data is processed based on the defect detection model to obtain surface defect detection data. The defect detection model is an improved version of the YOLO model and includes a loss function. The surface defect detection data is filtered to obtain the surface defect detection results; The surface defect detection data includes confidence levels, and the defect detection module also includes a scene classifier. The step of filtering the surface defect detection data to obtain the surface defect detection results further includes: The scene classifier determines the defect detection scene corresponding to the surface defect detection data, and obtains the first confidence threshold and the second confidence threshold output by the scene classifier for the defect detection scene. If the confidence level of the surface defect detection data is higher than the first confidence level threshold, then the surface defect detection data will be output as the surface defect detection result. If the confidence level of the surface defect detection data is higher than the second confidence threshold, then the surface defect detection data will be stored as data to be identified, and the second confidence threshold is lower than the first confidence threshold. The step of acquiring the image data to be processed also includes: The product under test is illuminated by a light box, and image data of the product under test is acquired by a camera to obtain image data to be processed. The light box consists of a box body, a light source and a light-transmitting panel. The light-transmitting panel is used to make the light from the light source evenly illuminate the product under test.

2. The method as described in claim 1, characterized in that, The loss function includes the SIoU function, which is used to calculate the localization loss. The steps of performing defect detection processing on the image data based on the defect detection model include: Based on the defective image data, a predicted bounding box and a preset ground truth bounding box are obtained; The SIoU loss value is obtained by comparing the predicted bounding box with the preset ground truth bounding box using the SIoU function; During defect detection processing, the loss weight of the SIoU loss value in the positioning loss calculation is increased.

3. The method as described in claim 2, characterized in that, After the step of comparing the predicted bounding box and the preset ground truth bounding box using the SIoU function to obtain the SIoU loss value, the method further includes: The weight penalty term is calculated using a preset optimizer; The SIoU loss value is optimized by using the weight penalty term to obtain the optimized SIoU loss value.

4. The method as described in claim 1, characterized in that, The loss function includes the Focal Loss function, and the steps for performing defect detection processing on image data based on the defect detection model include: Calculate the binary cross-entropy loss value based on the defective image data; The true class probability is calculated using the binary cross-entropy loss value. The Focal Loss function is obtained by calculating the true class probability. The Focal Loss function includes a first parameter and a second parameter, which are adjustable. The first parameter is a coefficient parameter and the second parameter is an exponential parameter. During defect detection processing, the defect image data is classified using the Focal Loss function.

5. The method as described in claim 1, characterized in that, The step of acquiring the image data to be processed and performing data preprocessing to obtain the defective image data further includes: The image data is converted to grayscale to obtain grayscale image data; The grayscale image data is binarized to obtain the defect image data, which is image data containing both black and white pixels.

6. A defect detection device, characterized in that, The device includes: The data acquisition module is used to acquire the image data to be processed and perform data preprocessing to obtain defect image data; The data processing module is used to perform defect detection processing on the defect image data based on the defect detection model to obtain surface defect detection data. The defect detection model is an improved version of the YOLO model and includes a loss function. The data filtering module is used to filter the surface defect detection data to obtain the surface defect detection results; The surface defect detection data includes confidence levels, the defect detection module also includes a scene classifier, and the data filtering module is further used for: The scene classifier determines the defect detection scene corresponding to the surface defect detection data, and obtains the first confidence threshold and the second confidence threshold output by the scene classifier for the defect detection scene. If the confidence level of the surface defect detection data is higher than the first confidence level threshold, then the surface defect detection data will be output as the surface defect detection result. If the confidence level of the surface defect detection data is higher than the second confidence threshold, then the surface defect detection data will be stored as data to be identified, and the second confidence threshold is lower than the first confidence threshold. The data acquisition module is also used for: The product under test is illuminated by a light box, and image data of the product under test is acquired by a camera to obtain image data to be processed. The light box consists of a box body, a light source and a light-transmitting panel. The light-transmitting panel is used to make the light from the light source evenly illuminate the product under test.

7. A defect detection device, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the visual defect detection method as described in any one of claims 1 to 5.

8. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the visual defect detection method as described in any one of claims 1 to 5.

Images