Defect detection method and device, electronic equipment and machine readable storage medium

Abstract

This application provides a defect detection method, apparatus, electronic device, and machine-readable storage medium. The method includes: during training, training a defect detection model using training sample data of a target to be detected, and using the defect detection model to learn feature information of qualified images of the target to be detected; wherein the training sample data of the target to be detected includes at least one qualified image of the target to be detected, and the qualified image of the target to be detected is an image in which the target to be detected is determined to be qualified; during defect detection, using the trained defect detection model, based on the feature information of the qualified image, determining whether the target to be detected in the image to be detected is qualified or has a defect. This method can achieve automatic defect detection and improve the efficiency of defect detection.

Description

Technical Field

[0001] This application relates to the field of robot vision perception, and in particular to a defect detection method, device, electronic device, and machine-readable storage medium. Background Technology

[0002] In the field of machine vision, quality inspection has wide applications in food, pharmaceuticals, lithium batteries, and 3C products. Product quality is crucial, and quality inspection is an important method for verifying product quality.

[0003] Traditional quality inspection relies on human eyes for testing, which is inefficient and has low accuracy. Summary of the Invention

[0004] In view of this, this application provides a defect detection method, apparatus, electronic device, and machine-readable storage medium.

[0005] According to a first aspect of the embodiments of this application, a defect detection method is provided, comprising:

[0006] During the training process, the defect detection model is trained using the training sample data of the target to be detected, and the defect detection model is used to learn the feature information of qualified images of the target to be detected; wherein, the training sample data of the target to be detected includes at least one qualified image of the target to be detected, and the qualified image of the target to be detected is the image in which the target to be detected is judged to be qualified.

[0007] During the defect detection process, the trained defect detection model is used to determine whether the target in the image to be detected is qualified or has defects based on the feature information of the qualified image.

[0008] According to a second aspect of the embodiments of this application, a defect detection apparatus is provided, comprising:

[0009] The training unit is used to train the defect detection model using training sample data of the target to be detected during the training process, and to learn the feature information of qualified images of the target to be detected using the defect detection model; wherein, the training sample data of the target to be detected includes at least one qualified image of the target to be detected, and the qualified image of the target to be detected is an image in which the target to be detected is judged to be qualified.

[0010] The detection unit is used to determine, during the defect detection process, whether the target to be detected in the image to be detected is qualified or has defects based on the feature information of the qualified image, using the trained defect detection model.

[0011] According to a third aspect of the embodiments of this application, an electronic device is provided, including a processor and a memory, the memory storing machine-executable instructions executable by the processor, the processor being configured to execute the machine-executable instructions to implement the method provided in the first aspect.

[0012] According to a fourth aspect of the embodiments of this application, a machine-readable storage medium is provided, wherein machine-executable instructions are stored therein, and when the machine-executable instructions are executed by a processor, the method provided in the first aspect is implemented.

[0013] The defect detection method of this application uses training sample data including at least one qualified image of the target to be detected to train a defect detection model. The defect detection model learns the feature information of the qualified image of the target to be detected. Then, in the defect detection process, the trained defect detection model can be used to determine whether the target to be detected in the image is qualified or has defects based on the feature information of the learned qualified image, thereby realizing automatic defect detection and improving the efficiency of defect detection. In addition, since the training sample data can be at least one qualified image, the cost of collecting and designing training data can be effectively saved, saving time and effort. Attached Figure Description

[0014] Figure 1 This is a schematic flowchart of a defect detection method provided in an embodiment of this application;

[0015] Figure 2 This is a flowchart illustrating a training process provided in an embodiment of this application;

[0016] Figure 3 This is a schematic diagram of a positioning area and a comparison area provided in an embodiment of this application;

[0017] Figure 4 This is a schematic diagram of a qualified image and a defective image provided in an embodiment of this application;

[0018] Figure 5 This is a schematic diagram of the structure of a defect detection device provided in an embodiment of this application;

[0019] Figure 6 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0020] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0021] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0022] To enable those skilled in the art to better understand the technical solutions provided in the embodiments of this application, and to make the above-mentioned objectives, features and advantages of the embodiments of this application more apparent and understandable, the technical solutions in the embodiments of this application will be further described in detail below with reference to the accompanying drawings.

[0023] It should be noted that the sequence number of each step in the embodiments of this application does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0024] Please see Figure 1 This is a flowchart illustrating a defect detection method provided in an embodiment of this application, as shown below. Figure 1 As shown, the defect detection method may include the following steps:

[0025] Step S100: During the training process, the defect detection model is trained using the training sample data of the target to be detected, and the defect detection model is used to learn the feature information of qualified images of the target to be detected; wherein, the training sample data of the target to be detected includes at least one qualified image of the target to be detected, and the qualified image of the target to be detected is the image in which the target to be detected is judged to be qualified.

[0026] In this embodiment of the application, in order to achieve automatic defect detection and improve the efficiency of defect detection, for any type of target to be detected, training sample data including at least one qualified image of the target to be detected can be used to train a machine learning model (hereinafter referred to as the defect detection model) for automatic defect detection, so that the defect detection model learns the feature information of the qualified image of the target to be detected.

[0027] For example, the qualified images mentioned above can be obtained by acquiring images of targets that have been manually determined to be qualified for detection.

[0028] It should be noted that, in the embodiments of this application, when the training sample data includes multiple qualified images, the defect detection model can learn the feature information of the multiple qualified images and perform defect detection based on the feature information of the multiple qualified images.

[0029] Furthermore, when the training sample data includes multiple qualified images, the number of features can be reduced by weight reduction to improve computational efficiency.

[0030] Step S110: During the defect detection process, the trained defect detection model is used to determine whether the target in the image to be detected is qualified or has defects based on the feature information of the qualified images learned.

[0031] In this embodiment of the application, once the defect detection model has been trained, the trained defect detection model can be used for defect detection.

[0032] For example, in the defect detection process, for any image of a target to be detected, a trained defect detection model can be used to determine whether the target in the image is qualified or has a defect based on the feature information of qualified images learned from it.

[0033] For example, the image to be detected can be determined as a qualified image or a defective image based on the similarity between the feature information of the image to be detected and the feature information of a qualified image learned by the defect detection model.

[0034] For example, if the similarity between the feature information of the image to be detected and the feature information of the qualified image exceeds a preset similarity threshold, the target to be detected in the image to be detected can be determined to be qualified; otherwise, the target to be detected in the image to be detected can be determined to be defective.

[0035] It should be noted that when the defect detection model has learned the feature information of multiple qualified images, it can determine the similarity between the feature information of the image to be detected and the feature information of each qualified image. If the similarity between the feature information of the image to be detected and the feature information of each qualified image exceeds a preset similarity threshold, the target to be detected in the image to be detected is determined to be qualified; otherwise, if the similarity between the feature information of the image to be detected and the feature information of any qualified image does not exceed the preset similarity threshold, the target to be detected in the image to be detected is determined to have a defect.

[0036] Alternatively, if the defect detection model has learned the feature information of multiple qualified images, the defect detection model can determine the similarity between the feature information of the image to be detected and the feature information of each qualified image. If the similarity between the feature information of the image to be detected and the feature information of any qualified image exceeds a preset similarity threshold, the target to be detected in the image to be detected is determined to be qualified; otherwise, if the similarity between the feature information of the image to be detected and the feature information of each qualified image does not exceed the preset similarity threshold, the target to be detected in the image to be detected is determined to have a defect.

[0037] For example, the image of the target to be detected can be obtained by acquiring images of the target that actually needs to be detected for defect detection.

[0038] For example, for any target to be detected, an image of the target can be acquired, and a trained defect detection model can be used to determine whether there is a defect in the target in the image.

[0039] It can be seen that, in Figure 1 In the method described, a defect detection model is trained using training sample data including at least one qualified image of the target to be detected. The defect detection model learns the feature information of qualified images of the target to be detected. Then, during the defect detection process, the trained defect detection model can be used to determine whether the target in the image to be detected is qualified or has defects based on the feature information of the learned qualified images, thus realizing automatic defect detection and improving the efficiency of defect detection. In addition, since the training sample data can be at least one qualified image, the cost of collecting and designing training data can be effectively saved, saving time and effort.

[0040] In some embodiments, the above-mentioned method of learning feature information of qualified images of the target to be detected using a defect detection model may include:

[0041] The defect detection model is used to learn the feature information of the contrast region in the qualified image of the target to be detected.

[0042] The above-mentioned determination of whether the image to be detected is a qualified image or a defective image based on the feature information of qualified images may include:

[0043] Based on the feature information of the comparison area in the image to be detected and the feature information of the comparison area in the qualified image, it is determined whether the target in the image to be detected is qualified or has defects.

[0044] For example, in practical applications, the focus of defect detection in an image might be limited to a small region. For instance, in the case of a beverage bottle cap, the focus might be on the area where the production date is printed. Therefore, this small region can be designated as a contrast region. In this case, feature extraction can be focused on this contrast region, resulting in more accurate detection results and less time consumption.

[0045] Accordingly, during the training process of the defect detection model, the feature information of the contrast region in the qualified image of the target to be detected can be learned by the defect detection model.

[0046] Similarly, during defect detection, feature information of the comparison area in the image to be detected can be obtained, and based on the feature information of the comparison area in the image to be detected and the feature information of the comparison area in the qualified image, it can be determined whether the target in the image to be detected is qualified or has defects.

[0047] It should be noted that, in the embodiments of this application, the comparison area is not limited to a part of the image to be detected, but can also be the entire image area. In this case (i.e., when the comparison area is the entire image area), feature extraction and feature information comparison need to be performed on the entire image area.

[0048] In one example, the contrast region in an image can be determined in the following way:

[0049] For any training image or image to be detected, identify the location of the localization region in the training image or image to be detected;

[0050] Based on the position of the localization region in the reference image of the target to be detected, and the position of the localization region in the training image or the image to be detected, the transformation matrix between the training image or the image to be detected and the reference image is determined.

[0051] Based on the location of the contrast region in the reference image, and using the transformation matrix, the contrast region in the training image or the image to be detected is determined.

[0052] For example, considering that the region to be detected (such as the comparison region mentioned above) may have positional shifts or rotations in different images, in order to keep the comparison region for feature comparison in different images consistent and to ensure the accuracy of defect detection, it is necessary to be able to accurately identify the comparison region in the image.

[0053] To achieve automatic localization of contrast regions in an image, a reference image can be set for any type of target to be detected, in which the localization region and the contrast region are set.

[0054] For example, the localization region can be a common area in the image of the target to be detected of this type, and the localization region can be located with specific textures or features.

[0055] For example, the location of the region can be achieved based on specific geometric shapes, texture shapes, color features, features extracted by neural networks, or other methods that enable the location of the region.

[0056] Accordingly, for any training image or image to be detected (the training image is used as an example below), the location region can be identified in the training image to determine the position of the location region in the training image.

[0057] For example, the location can be the coordinates of the positioning area in the image coordinate system.

[0058] Once the location of the localization region in the training image is determined, the transformation matrix between the training image and the reference image can be determined based on the location of the localization region in the reference image of the target to be detected and the location of the localization region in the training image. Furthermore, the contrast region in the training image or the image to be detected can be determined based on the transformation matrix, based on the location of the contrast region in the reference image.

[0059] It should be noted that, in the embodiments of this application, when the size and position of the comparison area are consistent in all images according to the direction, it is not necessary to locate the comparison area in the above manner. Instead, the comparison area in the image can be determined directly based on the pre-defined position of the comparison area in the image.

[0060] Furthermore, when the comparison region includes a portion of the image to be detected, the comparison region may include multiple non-connected regions in the image to be detected. In this case, it is necessary to determine whether the target in the image to be detected is qualified or defective based on the feature information of each comparison region in the image to be detected and the feature information of the corresponding comparison region in the qualified image.

[0061] For example, if the similarity between the feature information of any comparison region in the image to be detected and the feature information of the corresponding comparison region in the qualified image is lower than a preset similarity threshold, it can be determined that the target to be detected in the image to be detected has a defect.

[0062] In one example, determining whether a target in the image to be detected is qualified or defective based on feature information of the comparison region in the image to be detected and feature information of the comparison region in the qualified image can include:

[0063] Based on the similarity between the feature information of the comparison region in the image to be detected and the feature information of the comparison region in the qualified image, as well as the similarity threshold in the current scene, the image to be detected is determined to be a qualified image or a defective image.

[0064] The similarity thresholds are not exactly the same in different scenarios.

[0065] For example, the similarity between the feature information of the comparison region in the image to be detected and the feature information of the comparison region in the qualified image can be used to determine whether the target to be detected in the image to be detected is qualified or has defects.

[0066] For example, considering that the requirements for defects may differ in different scenarios. For instance, in some scenarios, minor defects may still allow the device to function normally, while in others, even small defects are unacceptable. Therefore, when using a defect detection model for defect detection, the similarity threshold set for different scenarios may not be entirely the same.

[0067] For example, different similarity thresholds can be set for different scenarios.

[0068] Accordingly, once the feature information of the comparison region in the image to be detected is determined, the similarity between the feature information of the comparison region in the image to be detected and the feature information of the comparison region in the qualified image can be determined. Based on this similarity and the similarity threshold in the current scene, it can be determined whether the target to be detected in the image to be detected is qualified or has defects.

[0069] In some embodiments, the aforementioned feature information may include edge features, color features, and some or all of other feature information extracted by the trained neural network.

[0070] It should be noted that when multiple different feature information exists, during defect detection, the similarity of each different feature information can be determined separately in the manner described above. If the similarity of each different feature information exceeds the corresponding threshold, the target in the image to be detected is determined to be qualified; otherwise, the target in the image to be detected is determined to be defective. Alternatively, a comprehensive similarity can be determined based on the similarity of each different feature information, and if the comprehensive similarity exceeds the corresponding threshold, the target in the image to be detected is determined to be qualified; otherwise, the target in the image to be detected is determined to be defective.

[0071] For example, a weighted algorithm can be used to determine the overall similarity based on the similarity corresponding to various different feature information.

[0072] In some embodiments, the aforementioned similarity may include correlation coefficient, cosine similarity, and some or all of the similarity predicted by a trained neural network, or corresponding variants thereof.

[0073] It should be noted that, when multiple similarities exist, during defect detection, the different similarities corresponding to the image to be detected can be determined separately according to the above method. If all similarities exceed the corresponding threshold, the target in the image to be detected is determined to be qualified; otherwise, the target in the image to be detected is determined to be defective. Alternatively, a comprehensive similarity can be determined based on the various similarities corresponding to the image to be detected, and if the comprehensive similarity exceeds the corresponding threshold, the target in the image to be detected is determined to be qualified; otherwise, the target in the image to be detected is determined to be defective.

[0074] In some embodiments, the training sample data of the target to be detected may further include at least one defect image of the target to be detected;

[0075] The above-mentioned determination of whether the target in the image to be detected is qualified or defective, based on the feature information of the comparison region in the image to be detected and the feature information of the comparison region in the qualified image, may include:

[0076] The score of the image to be detected is determined based on the first distance between the feature information of the comparison region in the image to be detected and the feature information of the comparison region in the qualified image, and the second distance between the feature information of the comparison region in the image to be detected and the feature information of the comparison region in the defective image.

[0077] Based on the score of the image to be detected and a preset score threshold, it is determined whether the target in the image to be detected is qualified or has defects.

[0078] For example, in order to improve the accuracy of defect detection, when training the defect detection model, the training sample data of the target to be detected may include not only at least one qualified image of the target to be detected, but also at least one defective image of the target to be detected. Thus, the defect detection model can learn the feature information of the qualified image of the target to be detected and the feature information of the defective image. For example, it can learn the feature information of the contrast region of the qualified image of the target to be detected and the feature information of the contrast region of the defective image.

[0079] Accordingly, in the defect detection process, for any image to be detected, the distance between the feature information of the comparison region in the image to be detected and the feature information of the comparison region in the qualified image (referred to as the first distance in this paper), and the distance between the feature information of the comparison region in the image to be detected and the feature information of the comparison region in the defective image (referred to as the second distance in this paper) can be determined respectively, and the score of the image to be detected can be determined based on the first distance and the second distance.

[0080] For example, the distances mentioned above may include, but are not limited to, Euclidean distance, Mahalanobis distance, Manhattan distance, Hamming distance, or their corresponding variants.

[0081] For example, when the defect detection model has learned the feature information of multiple qualified images, the first distance can be determined based on the average distance between the feature information of the comparison area in the image to be detected and the feature information of some or all qualified images.

[0082] For example, the average of the smallest n1 (n1 is less than or equal to the number of qualified images) distances between the feature information of the comparison region in the image to be detected and the feature information of each qualified image can be determined as the first distance mentioned above.

[0083] Similarly, when the defect detection model has learned the feature information of multiple defect images, the first distance can be determined based on the average distance between the feature information of the comparison area in the image to be detected and the feature information of some or all defect images.

[0084] For example, the average of the smallest n² (n² is less than or equal to the number of defective images) distances between the feature information of the comparison region in the image to be detected and the feature information of each defective image can be determined as the first distance mentioned above.

[0085] For example, the score of the image to be detected can be determined based on the first distance and the second distance mentioned above.

[0086] For example, the score of the image to be detected is negatively correlated with the first distance and positively correlated with the second distance.

[0087] For example, the score of the image to be detected and a preset score threshold can be used to determine whether the target in the image to be detected is qualified or has defects.

[0088] For example, if the score of the image to be detected exceeds a preset score threshold, the target in the image to be detected is determined to be qualified; otherwise, the target in the image to be detected is determined to be defective.

[0089] For example, since the requirements for qualified targets differ in different scenarios, the scoring thresholds in different scenarios may not be exactly the same.

[0090] In some embodiments, during the training process, the defect detection method provided in this application may further include:

[0091] The trained defect detection model is used to detect defects in the test sample data.

[0092] If the defect detection results of the test samples do not meet the requirements, new training sample data will be generated based on the inaccurate defect detection sample data in the test sample data, and the defect detection model will be trained again.

[0093] For example, considering that in real-world application scenarios, there may be a significant difference between the target to be detected and the training target, and when the defect detection result of the target to be detected is inconsistent with the expected result, such as when the accuracy rate is lower than the preset accuracy threshold, the misjudged target can be labeled as qualified or defective, the model can be retrained, and the detection can be repeated to make the defect detection result of the target to be detected consistent with the expectation.

[0094] It should be noted that, in the embodiments of this application, when the trained defect detection model is used for defect detection tasks, if it is determined that the defect detection result of the defect detection model is inconsistent with the expected result, such as the number of detection errors exceeding the preset threshold or being significantly lower than the preset accuracy threshold, the misjudged target can be labeled as qualified or defective, and the model can be retrained.

[0095] Furthermore, during the training process of the defect detection model, if the training sample data also includes defect images, the defect detection model can also learn the feature information of defect images during training. That is, the defect detection model can learn the feature information of qualified images and the feature information of defect images.

[0096] Accordingly, during the defect detection process using the trained defect detection model, for any image to be detected, the distance between the feature information of the image to be detected and the feature information of a qualified image (i.e., the first distance mentioned above), and the distance between the feature information of the image to be detected and the feature information of a defective image (i.e., the second distance mentioned above) can be determined respectively. Based on the first distance and the second distance, the score of the image to be detected is determined. If the score exceeds the preset score threshold, the target to be detected in the image to be detected is determined to be qualified; otherwise, the target to be detected in the image to be detected is determined to have a defect.

[0097] To enable those skilled in the art to better understand the technical solutions provided in the embodiments of this application, the technical solutions provided in the embodiments of this application are described below in conjunction with specific application scenarios.

[0098] In this embodiment, an attention mechanism can be introduced into defect detection. A contrast region to be detected can be selected. During the detection process, by detecting the contrast region in the image, it can be determined whether there is a defect in the target to be detected in the image.

[0099] In addition, since the comparison area may be shifted or rotated in different images, a positioning module is introduced to improve the accuracy of defect detection. This module locates the comparison area in the image and then performs defect detection on the comparison area.

[0100] Furthermore, since different defects exhibit varying degrees of salience across different features, the solution provided in this application supports the use of various features such as edge features, color features, and other features extracted by neural networks for defect detection.

[0101] The defect detection implementation process in this embodiment will be described below.

[0102] In this embodiment, the defect detection scheme mainly includes two parts: one part is model training, and the other part is defect detection using the trained model (i.e., the defect detection model).

[0103] For example, the model can be trained using at least one qualified image of the target object, after which defect detection can be performed.

[0104] It should be noted that when there are few training images, the model's adaptability may be insufficient. When the defect detection result of the target to be detected is inconsistent with the expected result, the misjudged target can be labeled as qualified or defective, the model can be retrained, and the detection can be repeated to make the defect detection result of the target to be detected consistent with the expectation.

[0105] The implementation details of model training and defect detection are explained below.

[0106] 1. Model Training

[0107] The training data for model training can include OK images and NG images. OK images are normal images (i.e., qualified images as mentioned above), and NG images are defective images.

[0108] For example, training can be performed using at least one OK image, but multiple OK and NG images can also be used for training.

[0109] For example, in images requiring defect detection, the focus of defect detection might be limited to a small region of the image. For instance, in the case of a beverage bottle cap, the focus of defect detection might be the area where the production date is printed. Therefore, this small region can be designated as a contrast region. In this case, feature extraction can be focused on this contrast region, resulting in more accurate detection results and less time consumption.

[0110] A general training process can be as follows: Figure 2 As shown, the main steps include: setting the positioning area, setting the comparison area, and training using training sample data. These three processes will be explained in detail below:

[0111] 1.1 Setting the location area

[0112] The localization region is selected from all images that share a common area. This region needs to have specific textures or features for localization. The localization method can be based on specific geometric shapes, texture shapes, color features, features extracted by neural networks, or other methods.

[0113] The transformation matrix between the training image or the image to be detected and the reference image can be determined based on the relative positional relationship between the localization regions in the training image or the image to be detected and the localization regions in the reference image.

[0114] For example, a common method of location is to use template matching.

[0115] For example, template matching can use shape features, grayscale features, or other features for localization.

[0116] The advantage of using template matching is that it can be applied to almost all scenarios, and its positioning accuracy is very high, reaching the sub-pixel level, which makes it convenient to calculate the transformation matrix between images.

[0117] 1.2 Setting the comparison area

[0118] The contrast region is the area in an image used for defect detection. When performing defect detection, the content in the contrast region of the image can be used for detection.

[0119] For example, the contrast region only needs to be set in the reference image, and the contrast region on other images (such as training images or images to be detected) can be determined by the localization module.

[0120] like Figure 3As shown, the left image is the baseline image, where "XXXXXXXX" represents the selected localization region, and the rectangular area to the left of the localization region is the set contrast region. The right image is a training image or the image to be detected, excluding the baseline image. Given the transformation matrix between the right image and the baseline image, the contrast region in the right image can be determined based on this transformation matrix and the position of the contrast region in the baseline image.

[0121] 1.3 Training

[0122] During training, you can use at least one OK image, or you can use multiple OK images or NG images.

[0123] During training, feature calculations can be performed on the comparison region, allowing the model to learn the feature information of the comparison region from the training sample data.

[0124] For example, since different defects exhibit varying degrees of salience on different features, different feature types can be selected for calculation, such as edge features, color features, features extracted by a trained neural network, and other features set by the user.

[0125] The above features can be used individually or combined according to different weights to achieve different purposes.

[0126] For example, when the defect is caused by changes in texture or edge, edge features can be selected for defect detection; when the defect is caused by changes in color, color features can be selected for defect detection.

[0127] 2. Defect Detection

[0128] In defect detection, the location region in the image to be detected can be determined first. Then, based on the position of the location region in the image to be detected and the position of the location region in the reference image, the transformation matrix between the image to be detected and the reference image can be determined. Furthermore, the location of the location region in the image to be detected can be determined by the transformation matrix and the position of the location region in the reference image.

[0129] Once the location of the localized region in the image to be detected is determined, the feature information of the comparison region can be calculated according to the selected feature type. Finally, the similarity between the feature information of the comparison region in the image to be detected and the feature information of the comparison region learned by the model is calculated to determine whether the image to be detected is a qualified image or a defective image.

[0130] When calculating similarity, different calculation methods can be used depending on the selected features, such as correlation coefficient, cosine similarity, and similarity predicted by neural networks.

[0131] like Figure 4The image shows the results of cosine similarity calculation when using edge features. In the left image, the target to be detected has a defect, so the calculated similarity is 0, and it is judged as NG, meaning it has a defect. In the right image, the target to be detected has no defect, so the calculated similarity is 1, and it is judged as OK, meaning it is qualified.

[0132] It should be noted that since the images of targets with different defects may differ to varying degrees from normal images, and similarity can be used to measure this difference, using similarity for defect detection can effectively address the detection of targets with different defects.

[0133] For example, considering that the requirements for defects may differ in different scenarios. For instance, in some scenarios, small defects may still be acceptable, while in others, even small defects are unacceptable. Therefore, when using a defect detection model for defect detection, the similarity thresholds set for different scenarios can not be exactly the same. This allows different similarity thresholds to be used in different scenarios to adapt to the defect requirements in those scenarios.

[0134] The method provided in this application has been described above. The apparatus provided in this application is described below:

[0135] Please see Figure 5 This is a schematic diagram of the structure of a defect detection device provided in an embodiment of this application, as shown below. Figure 5 As shown, the defect detection device may include:

[0136] Training unit 510 is used to train a defect detection model using training sample data of the target to be detected during the training process, and to learn feature information of qualified images of the target to be detected using the defect detection model; wherein, the training sample data of the target to be detected includes at least one qualified image of the target to be detected, and the qualified image of the target to be detected is an image in which the target to be detected is judged to be qualified.

[0137] The detection unit 520 is used to determine whether the target to be detected in the image to be detected is qualified or has defects based on the feature information of the qualified image during the defect detection process, using the trained defect detection model.

[0138] In some embodiments, the training unit 510 uses the defect detection model to learn feature information of a qualified image of the target to be detected, including:

[0139] The defect detection model is used to learn the feature information of the contrast region in the qualified image of the target to be detected;

[0140] The detection unit 520 determines whether the image to be detected is a qualified image or a defective image based on the feature information of the qualified image, including:

[0141] Based on the feature information of the comparison region in the image to be detected and the feature information of the comparison region in the qualified image, it is determined whether the target to be detected in the image to be detected is qualified or has defects.

[0142] In some embodiments, the contrast region in an image is determined in the following way:

[0143] For any training image or image to be detected, identify the location of the localization region in the training image or image to be detected;

[0144] Based on the position of the localization region in the reference image of the target to be detected, and the position of the localization region in the training image or the image to be detected, a transformation matrix between the training image or the image to be detected and the reference image is determined;

[0145] Based on the position of the contrast region in the reference image, and based on the transformation matrix, the contrast region in the training image or the image to be detected is determined.

[0146] In some embodiments, the detection unit 520 determines whether the target to be detected in the image to be detected is qualified or has defects based on the feature information of the comparison region in the image to be detected and the feature information of the comparison region in the qualified image, including:

[0147] Based on the similarity between the feature information of the comparison region in the image to be detected and the feature information of the comparison region in the qualified image, as well as the similarity threshold in the current scene, it is determined whether the target to be detected in the image to be detected is qualified or has defects.

[0148] The similarity thresholds are not exactly the same in different scenarios.

[0149] In some embodiments, the feature information includes edge features, color features, and some or all of other feature information extracted by the trained neural network.

[0150] And / or,

[0151] The similarity includes correlation coefficient, cosine similarity, and some or all of the similarity predicted by the trained neural network, or corresponding variants.

[0152] In some embodiments, the training sample data of the target to be detected further includes at least one defect image of the target to be detected;

[0153] The detection unit 520 determines whether the target in the image to be detected is qualified or has defects based on the feature information of the comparison area in the image to be detected and the feature information of the comparison area in the qualified image, including:

[0154] The score of the image to be detected is determined based on the first distance between the feature information of the comparison area in the image to be detected and the feature information of the comparison area in the qualified image, and the second distance between the feature information of the comparison area in the image to be detected and the feature information of the comparison area in the defective image.

[0155] Based on the score of the image to be detected and a preset score threshold, it is determined whether the target in the image to be detected is qualified or has defects.

[0156] The scoring thresholds are not exactly the same in different scenarios.

[0157] In some embodiments, the training unit 510 is further configured to perform defect detection on the test sample data using the trained defect detection model; if the defect detection result of the test sample does not meet the requirements, new training sample data is generated based on the sample data of inaccurate defect detection in the test sample data, and the defect detection model is trained again.

[0158] This application provides an electronic device including a processor and a memory, wherein the memory stores machine-executable instructions that can be executed by the processor, and the processor executes the machine-executable instructions to implement the defect detection method described above.

[0159] Please see Figure 6 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. The electronic device may include a processor 601 and a memory 602 storing machine-executable instructions. The processor 601 and the memory 602 can communicate via a system bus 603. Furthermore, by reading and executing the machine-executable instructions corresponding to the defect detection logic in the memory 602, the processor 601 can execute the defect detection method described above.

[0160] The memory 602 mentioned in this document can be any electronic, magnetic, optical, or other physical storage device that can contain or store information such as executable instructions, data, etc. For example, machine-readable storage media can be: RAM (Random Access Memory), volatile memory, non-volatile memory, flash memory, storage drives (such as hard disk drives), solid-state drives, any type of storage disk (such as optical discs, DVDs, etc.), or similar storage media, or combinations thereof.

[0161] In some embodiments, a machine-readable storage medium, such as Figure 6 The memory 602 in the memory, which is a machine-readable storage medium, stores machine-executable instructions that, when executed by a processor, implement the defect detection method described above. For example, the storage medium may be ROM, RAM, CD-ROM, magnetic tape, floppy disk, or optical data storage device.

[0162] It should be noted that, in this document, relational terms such as "objective" and "target" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0163] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A defect detection method, characterized in that, include: During the training process, the defect detection model is trained using the training sample data of the target to be detected, and the defect detection model is used to learn the feature information of qualified images of the target to be detected; wherein, the training sample data of the target to be detected includes at least one qualified image of the target to be detected, and the qualified image of the target to be detected is the image in which the target to be detected is judged to be qualified. During the defect detection process, the trained defect detection model is used to determine whether the target in the image to be detected is qualified or has defects based on the feature information of the qualified image. The step of learning the feature information of the qualified image of the target to be detected using the defect detection model includes: The defect detection model is used to learn the feature information of the contrast region in the qualified image of the target to be detected; The step of determining whether the image to be detected is a qualified image or a defective image based on the feature information of the qualified image includes: Based on the feature information of the comparison region in the image to be detected and the feature information of the comparison region in the qualified image, it is determined whether the target to be detected in the image to be detected is qualified or has defects; Contrast regions in an image are determined in the following way: For any training image or image to be detected, identify the location of the localization region in the training image or image to be detected; Based on the position of the localization region in the reference image of the target to be detected, and the position of the localization region in the training image or the image to be detected, a transformation matrix between the training image or the image to be detected and the reference image is determined; Based on the position of the contrast region in the reference image, and based on the transformation matrix, the contrast region in the training image or the image to be detected is determined.

2. The method according to claim 1, characterized in that, The step of determining whether the target in the image to be detected is qualified or defective based on the feature information of the comparison region in the image to be detected and the feature information of the comparison region in the qualified image includes: Based on the similarity between the feature information of the comparison region in the image to be detected and the feature information of the comparison region in the qualified image, as well as the similarity threshold in the current scene, it is determined whether the target to be detected in the image to be detected is qualified or has defects. The similarity thresholds are not exactly the same in different scenarios.

3. The method according to claim 2, characterized in that, The feature information includes edge features, color features, and some or all of other feature information extracted by the trained neural network. And / or, The similarity includes correlation coefficient, cosine similarity, and some or all of the similarity predicted by the trained neural network, or corresponding variants.

4. The method according to claim 1, characterized in that, The training sample data of the target to be detected also includes at least one defect image of the target to be detected; The step of determining whether the target in the image to be detected is qualified or defective based on the feature information of the comparison region in the image to be detected and the feature information of the comparison region in the qualified image includes: The score of the image to be detected is determined based on the first distance between the feature information of the comparison region in the image to be detected and the feature information of the comparison region in the qualified image, and the second distance between the feature information of the comparison region in the image to be detected and the feature information of the comparison region in the defective image. Based on the score of the image to be detected and a preset score threshold, it is determined whether the target in the image to be detected is qualified or has defects. The scoring thresholds are not exactly the same in different scenarios.

5. The method according to claim 1, characterized in that, During the training process, the method further includes: The trained defect detection model is used to detect defects in the test sample data. If the defect detection results of the test sample do not meet the requirements, new training sample data will be generated based on the inaccurate defect detection sample data in the test sample data, and the defect detection model will be trained again.

6. A defect detection device, characterized in that, include: The training unit is used to train the defect detection model using training sample data of the target to be detected during the training process, and to learn the feature information of qualified images of the target to be detected using the defect detection model; wherein, the training sample data of the target to be detected includes at least one qualified image of the target to be detected, and the qualified image of the target to be detected is an image in which the target to be detected is judged to be qualified. The detection unit is used to determine whether the target to be detected in the image to be detected is qualified or has defects based on the feature information of the qualified image during the defect detection process, using the trained defect detection model. The training unit uses the defect detection model to learn feature information of the qualified image of the target to be detected, including: The defect detection model is used to learn the feature information of the contrast region in the qualified image of the target to be detected; The detection unit determines whether the image to be detected is a qualified image or a defective image based on the feature information of the qualified image, including: Based on the feature information of the comparison region in the image to be detected and the feature information of the comparison region in the qualified image, it is determined whether the target to be detected in the image to be detected is qualified or has defects; The contrast regions in the image are determined in the following way: For any training image or image to be detected, identify the location of the localization region in the training image or image to be detected; Based on the position of the localization region in the reference image of the target to be detected, and the position of the localization region in the training image or the image to be detected, a transformation matrix between the training image or the image to be detected and the reference image is determined; Based on the position of the contrast region in the reference image, and based on the transformation matrix, the contrast region in the training image or the image to be detected is determined.

7. The apparatus according to claim 6, characterized in that, The detection unit determines whether the target in the image to be detected is qualified or has defects based on the feature information of the comparison region in the image to be detected and the feature information of the comparison region in the qualified image, including: Based on the similarity between the feature information of the comparison region in the image to be detected and the feature information of the comparison region in the qualified image, as well as the similarity threshold in the current scene, it is determined whether the target to be detected in the image to be detected is qualified or has defects. The similarity thresholds are not entirely the same in different scenarios; The feature information includes edge features, color features, and some or all of other feature information extracted by the trained neural network. And / or, The similarity includes correlation coefficient, cosine similarity, and some or all of the similarity predicted by the trained neural network or corresponding variants. The training sample data of the target to be detected also includes at least one defect image of the target to be detected; The detection unit determines whether the target in the image to be detected is qualified or has defects based on the feature information of the comparison region in the image to be detected and the feature information of the comparison region in the qualified image, including: The score of the image to be detected is determined based on the first distance between the feature information of the comparison area in the image to be detected and the feature information of the comparison area in the qualified image, and the second distance between the feature information of the comparison area in the image to be detected and the feature information of the comparison area in the defective image. Based on the score of the image to be detected and a preset score threshold, it is determined whether the target in the image to be detected is qualified or has defects. The scoring thresholds are not entirely the same in different scenarios; The training unit is further configured to use the trained defect detection model to perform defect detection on the test sample data; if the defect detection result of the test sample does not meet the requirements, new training sample data is generated based on the inaccurate defect detection sample data in the test sample data, and the defect detection model is trained again.

8. An electronic device, characterized in that, The method includes a processor and a memory, the memory storing machine-executable instructions that can be executed by the processor, the processor executing the machine-executable instructions to implement the method as described in any one of claims 1-5.

9. A machine-readable storage medium, characterized in that, The machine-readable storage medium stores machine-executable instructions, which, when executed by a processor, implement the method as described in any one of claims 1-5.

Images