Defect detection method and device, computer equipment, storage medium and program product

Abstract

The invention relates to a defect detection method and device, computer equipment, a storage medium and a program product. The method comprises the following steps: acquiring a to-be-detected image obtained by shooting a to-be-detected object at a target position; obtaining a template image obtained by shooting a template object at the target position, wherein the to-be-detected object and the template object belong to the same object category; template object features of the template object are extracted from the template image, and to-be-detected object features of the to-be-detected object are extracted from the to-be-detected image; and based on the template object features and the to-be-detected object features, performing defect detection on the to-be-detected object at the target position to obtain a defect detection result of the to-be-detected object. By adopting the method, the accuracy of defect detection can be improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a defect detection method, apparatus, computer equipment, storage medium, and computer program product. Background Technology

[0002] With the development of computer technology, defect detection technology for electronic components has emerged. This technology enables the detection of defects in various electronic components, identifying flaws and imperfections, and effectively improving the quality of electronic components.

[0003] Traditional defect detection techniques typically train a deep learning-based segmentation model to extract features from the input image. The segmentation head then outputs predicted defect locations and category confidence scores to detect defects in the product. However, traditional defect detection techniques suffer from low accuracy and insufficient detection capability, easily leading to missed detections. Summary of the Invention

[0004] Therefore, it is necessary to provide a defect detection method, apparatus, computer equipment, computer-readable storage medium, and computer program product that can improve detection accuracy, as well as a defect detection model training method, apparatus, computer equipment, computer-readable storage medium, and computer program product, in response to the above-mentioned technical problems.

[0005] This application provides a defect detection method. The method includes:

[0006] Acquire the image of the object to be tested at the target location by taking a picture;

[0007] A template image is obtained by taking a picture of the template object at the target location, wherein the object to be tested and the template object belong to the same object category;

[0008] Extract the template object features of the template object from the template image, and extract the test object features of the test object from the test image;

[0009] Based on the features of the template object and the features of the object to be tested, defect detection is performed on the object to be tested at the target location to obtain the defect detection result of the object to be tested.

[0010] On the other hand, this application also provides a defect detection device. The device includes:

[0011] The first acquisition module is used to acquire the image of the object to be tested obtained by taking a picture of the object to be tested at the target location;

[0012] The second acquisition module is used to acquire a template image obtained by taking a picture of the template object at the target location, wherein the object to be tested and the template object belong to the same object category;

[0013] The extraction module is used to extract template object features of the template object from the template image and extract test object features of the test object from the test image;

[0014] The detection module is used to perform defect detection on the object under test at the target location based on the features of the template object and the features of the object under test, and obtain the defect detection result of the object under test.

[0015] On the other hand, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:

[0016] Acquire the image of the object to be tested at the target location by taking a picture;

[0017] A template image is obtained by taking a picture of the template object at the target location, wherein the object to be tested and the template object belong to the same object category;

[0018] Extract the template object features of the template object from the template image, and extract the test object features of the test object from the test image;

[0019] Based on the features of the template object and the features of the object to be tested, defect detection is performed on the object to be tested at the target location to obtain the defect detection result of the object to be tested.

[0020] On the other hand, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs the following steps:

[0021] Acquire the image of the object to be tested at the target location by taking a picture;

[0022] A template image is obtained by taking a picture of the template object at the target location, wherein the object to be tested and the template object belong to the same object category;

[0023] Extract the template object features of the template object from the template image, and extract the test object features of the test object from the test image;

[0024] Based on the features of the template object and the features of the object to be tested, defect detection is performed on the object to be tested at the target location to obtain the defect detection result of the object to be tested.

[0025] On the other hand, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, performs the following steps:

[0026] Acquire the image of the object to be tested at the target location by taking a picture;

[0027] A template image is obtained by taking a picture of the template object at the target location, wherein the object to be tested and the template object belong to the same object category;

[0028] Extract the template object features of the template object from the template image, and extract the test object features of the test object from the test image;

[0029] Based on the features of the template object and the features of the object to be tested, defect detection is performed on the object to be tested at the target location to obtain the defect detection result of the object to be tested.

[0030] The aforementioned defect detection method, apparatus, computer equipment, storage medium, and computer program products acquire a test image of the object to be tested at a target location by photographing it. The test object and a template object belong to the same object category, and the template object is used as a reference object. A template image of the template object at the target location is acquired and used as a reference image. Template object features of the template object are extracted from the template image, and test object features of the object to be tested are extracted from the test image. Based on these template object features and test object features, defect detection is performed on the test object at the target location to detect defects present in the test object, obtaining defect detection results. This method can more accurately and comprehensively detect defects present in the same location of the test object, reducing the false negative rate and effectively improving the accuracy and comprehensiveness of defect detection.

[0031] This application provides a defect detection model processing method. The method includes:

[0032] Acquire the image of the sample object to be tested obtained by taking a picture of the sample location;

[0033] A template sample image is obtained by taking a picture of the template sample object at the sample location, wherein the sample object to be tested and the template sample object belong to the same sample object category;

[0034] Extracting template sample object features of the template sample object from the template sample image, and extracting test sample object features of the test sample object from the test sample image;

[0035] Based on the features of the template sample object and the features of the sample object to be tested, defect detection is performed on the sample object to be tested at the sample location to obtain the predicted defect detection result of the sample object to be tested.

[0036] Obtain the sample defect detection result corresponding to the sample object to be tested, and train the model based on the difference between the predicted defect detection result and the sample defect detection result to obtain the defect detection model.

[0037] On the other hand, this application also provides a defect detection model processing device. The device includes:

[0038] The first sample acquisition module is used to acquire the image of the sample to be tested obtained by taking pictures of the sample to be tested at the sample location;

[0039] The second sample acquisition module is used to acquire a template sample image obtained by taking a picture of the template sample object at the sample location, wherein the sample object to be tested and the template sample object belong to the same sample object category.

[0040] The sample extraction module is used to extract template sample object features of the template sample object from the template sample image, and to extract test sample object features of the test sample object from the test sample image;

[0041] The sample detection module is used to perform defect detection on the sample object at the sample location based on the features of the template sample object and the features of the sample object to be tested, and to obtain the predicted defect detection result of the sample object to be tested;

[0042] The training module is used to obtain the sample defect detection results corresponding to the sample object to be tested, and to train the model based on the difference between the predicted defect detection results and the sample defect detection results to obtain the defect detection model.

[0043] On the other hand, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:

[0044] Acquire the image of the sample object to be tested obtained by taking a picture of the sample location;

[0045] A template sample image is obtained by taking a picture of the template sample object at the sample location, wherein the sample object to be tested and the template sample object belong to the same sample object category;

[0046] Extracting template sample object features of the template sample object from the template sample image, and extracting test sample object features of the test sample object from the test sample image;

[0047] Based on the features of the template sample object and the features of the sample object to be tested, defect detection is performed on the sample object to be tested at the sample location to obtain the predicted defect detection result of the sample object to be tested.

[0048] Obtain the sample defect detection result corresponding to the sample object to be tested, and train the model based on the difference between the predicted defect detection result and the sample defect detection result to obtain the defect detection model.

[0049] On the other hand, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs the following steps:

[0050] Acquire the image of the sample object to be tested obtained by taking a picture of the sample location;

[0051] A template sample image is obtained by taking a picture of the template sample object at the sample location, wherein the sample object to be tested and the template sample object belong to the same sample object category;

[0052] Extracting template sample object features of the template sample object from the template sample image, and extracting test sample object features of the test sample object from the test sample image;

[0053] Based on the features of the template sample object and the features of the sample object to be tested, defect detection is performed on the sample object to be tested at the sample location to obtain the predicted defect detection result of the sample object to be tested.

[0054] Obtain the sample defect detection result corresponding to the sample object to be tested, and train the model based on the difference between the predicted defect detection result and the sample defect detection result to obtain the defect detection model.

[0055] On the other hand, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, performs the following steps:

[0056] Acquire the image of the sample object to be tested obtained by taking a picture of the sample location;

[0057] A template sample image is obtained by taking a picture of the template sample object at the sample location, wherein the sample object to be tested and the template sample object belong to the same sample object category;

[0058] Extracting template sample object features of the template sample object from the template sample image, and extracting test sample object features of the test sample object from the test sample image;

[0059] Based on the features of the template sample object and the features of the sample object to be tested, defect detection is performed on the sample object to be tested at the sample location to obtain the predicted defect detection result of the sample object to be tested.

[0060] Obtain the sample defect detection result corresponding to the sample object to be tested, and train the model based on the difference between the predicted defect detection result and the sample defect detection result to obtain the defect detection model.

[0061] The aforementioned defect detection model processing method, apparatus, computer equipment, storage medium, and computer program product acquire a test sample image obtained by photographing the test sample object at the sample location. The test sample object and the template sample object belong to the same sample object category. Using the template sample object as a reference object, a template image is acquired by photographing the template sample object at the sample location. This template image is then used as a reference image. Template sample object features are extracted from the template sample image, and test sample object features are extracted from the test sample image. Based on these template sample object features and test sample object features, defect detection is performed on the test sample object at the sample location, obtaining the model's output predicted defect detection result for the test sample object. The sample defect detection result corresponding to the test sample object is obtained. The model is trained based on the difference between the predicted defect detection result and the sample defect detection result. During training, the model's output defect detection result gradually approaches the actual defect detection result, enabling the obtained defect detection model to more accurately and comprehensively detect defects present in different test objects at the same location, reducing the false negative rate and effectively improving the accuracy and comprehensiveness of defect detection. Furthermore, the defect detection model enables rapid execution of defect detection tasks, thereby improving the efficiency of defect detection. Attached Figure Description

[0062] Figure 1 This is a diagram illustrating the application environment of a defect detection method in one embodiment;

[0063] Figure 2 This is a flowchart illustrating a defect detection method in one embodiment;

[0064] Figure 3 This is a schematic diagram illustrating the geometric defect categories in one embodiment;

[0065] Figure 4 This is a schematic diagram illustrating the categories of appearance defects in one embodiment;

[0066] Figure 5 This is an architecture diagram of geometric defect detection in one embodiment;

[0067] Figure 6 This is a schematic diagram of the image segmentation process in one embodiment;

[0068] Figure 7 This is a flowchart illustrating the defect detection method in one embodiment;

[0069] Figure 8 This is a flowchart illustrating a defect detection model processing method in one embodiment;

[0070] Figure 9 This is an architecture diagram of an appearance defect detection model in one embodiment;

[0071] Figure 10 This is a schematic diagram of the appearance defect detection process in one embodiment;

[0072] Figure 11 This is a schematic diagram of defect detection results for an industrial dataset in one embodiment;

[0073] Figure 12 This is a schematic diagram of defect detection results for another industrial dataset in one embodiment;

[0074] Figure 13 This is a structural block diagram of a defect detection device in one embodiment;

[0075] Figure 14 This is a structural block diagram of a defect detection model processing device in one embodiment;

[0076] Figure 15 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0077] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0078] The defect detection method provided in this application embodiment can be applied to, for example, Figure 1In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated on server 104 or placed on the cloud or other servers. Both terminal 102 and server 104 can independently execute the defect detection method provided in this embodiment. Terminal 102 and server 104 can also work together to execute the defect detection method provided in this embodiment. When terminal 102 and server 104 work together to execute the defect detection method provided in this embodiment, terminal 102 acquires a test image of the object to be tested at the target location and sends the test image to server 104. Server 104 acquires a template image of a template object at the target location, where the object to be tested and the template object belong to the same object category. Server 104 extracts template object features from the template image and extracts test object features from the test image. Based on the template object features and the test object features, server 104 performs defect detection on the object to be tested at the target location and obtains the defect detection result of the object to be tested.

[0079] In this embodiment, the defect detection model processing method provided in this application can be applied to, for example... Figure 1 In the application environment shown, both terminal 102 and server 104 can independently execute the defect detection model processing method provided in this embodiment. Terminal 102 and server 104 can also work together to execute the defect detection model processing method provided in this embodiment. When terminal 102 and server 104 work together to execute the defect detection model processing method provided in this embodiment, terminal 102 can obtain from server 104 a test sample image obtained by photographing the test sample object at the sample location, and a template sample image obtained by photographing the template sample object at the sample location. The test sample object and the template sample object belong to the same sample object category. Terminal 102 extracts template sample object features from the template sample image and test sample object features from the test sample image. Based on the template sample object features and the test sample object features, terminal 102 performs defect detection on the test sample object at the sample location, obtaining a predicted defect detection result for the test sample object. Terminal 102 determines the sample defect detection result corresponding to the test sample object, and trains the model based on the difference between the predicted defect detection result and the sample defect detection result to obtain a defect detection model. The defect detection model can be deployed on terminal 102 or server 104.

[0080] The terminal 102 can be, but is not limited to, various desktop computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, and smart in-vehicle systems. Portable wearable devices can include smartwatches, smart bracelets, and head-mounted devices. The server 104 can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0081] In one embodiment, such as Figure 2 As shown, a defect detection method is provided, which is applied to computer equipment (such as computer equipment). Figure 1 Taking a terminal or server as an example, the following steps are included:

[0082] Step 202: Obtain the image of the object to be tested obtained by taking a picture of the object at the target location.

[0083] The object to be tested refers to the object that needs to be inspected for defects, such as electronic components, glass, ceramics, plastics, etc., but not limited to these. The target location refers to the location of the object to be tested.

[0084] The image to be tested refers to an image containing the object to be tested. The image to be tested can be of any format, such as at least one of the following: a color image, a grayscale image, a depth image, or an image corresponding to the Y component of a YUV image. In a YUV image, "Y" represents luminance (or luma), which is the grayscale value, while "U" and "V" represent chroma (or chroma), which describe the color and saturation of the image and are used to specify the color of a pixel. A color image, also known as an RGB image, refers to an image where each pixel is composed of red (R), green (G), and blue (B) components.

[0085] Specifically, the computer device can acquire a test image including the object to be tested, which is obtained by taking a picture of the object to be tested at the target location.

[0086] In this embodiment, the image to be tested can be an image directly captured by a computer device, an image obtained locally, or an image obtained from other devices.

[0087] In this embodiment, the image to be tested is an image obtained by taking a picture of the object to be tested at the target location at a preset shooting angle.

[0088] In this embodiment, the image to be tested is an image obtained by capturing the object at the target location under a preset light source. Further, the image to be tested is an image obtained by capturing the object at the target location under a preset shooting angle and a preset light source.

[0089] Step 204: Obtain a template image of the template object at the target location by taking a picture. The test object and the template object belong to the same object category.

[0090] The template object is a standard object without defects. The object under test has the same object category as the template object. The object category refers to the type of object, such as the category of electronic components or ceramics. Electronic components have many categories, such as resistors, capacitors, inductors, transformers, diodes, transistors, optoelectronic devices, electroacoustic devices, and display devices, but are not limited to these.

[0091] Specifically, the computer device can acquire a template image including a template object, which is obtained by photographing the template object at the target location.

[0092] In this embodiment, the template image can be an image directly captured by a computer device, an image obtained from a local device, or an image obtained from other devices.

[0093] In one embodiment, the template image may be a pre-stored image that includes a template object.

[0094] In this embodiment, the template image and the image to be tested have the same shooting configuration information. The shooting configuration information includes at least one of the shooting device, shooting angle, or shooting light source. For example, the template image is an image obtained by shooting a template object at a target location at a preset shooting angle.

[0095] For example, a template image is an image obtained by photographing a template object at a target location under a preset light source. Further, a template image is an image obtained by photographing a template object at a target location under a preset shooting angle and a preset light source.

[0096] Step 206: Extract template object features from the template image and extract test object features from the test image.

[0097] The features of the object to be measured include at least one of geometric features or appearance features. Geometric features refer to features related to the surface geometry of the object to be measured. Appearance features refer to features related to the appearance of the object to be measured.

[0098] Specifically, the computer device extracts the template object features of the template object from the template image, and extracts the template object features of the template object from the template image.

[0099] In this embodiment, the computer device can extract the geometric features and appearance features of the object to be tested from the image to be tested.

[0100] In this embodiment, the computer device can extract the template geometric features and template appearance features of the template object from the template image.

[0101] Step 208: Based on the features of the template object and the features of the object to be tested, perform defect detection on the object to be tested at the target location to obtain the defect detection results of the object to be tested.

[0102] The defect detection results include at least one of the following: defect category, defect location, or defect area.

[0103] Furthermore, the defect categories include at least one of the following: geometric defect category and appearance defect category.

[0104] The defect detection results include at least one of the following: geometric defect detection results or appearance defect detection results. Geometric defect detection results include at least one of the following: geometric defect category, defect location, or defect area. Appearance defect detection results include at least one of the following: appearance defect category, defect location, or defect area.

[0105] Specifically, the computer equipment performs defect detection on the object under test at the target location based on the features of the template object and the features of the object under test, in order to detect whether the object under test has geometric defects or appearance defects, and obtain the defect detection results of the object under test.

[0106] In this embodiment, the computer device segments the template image based on the features of the template object to obtain a template region including the template object. The image to be tested is then segmented based on the features of the object to be tested to obtain a test region including the object to be tested. Based on the template region and the test region, defect detection is performed on the object to be tested at the target location to obtain the defect detection result.

[0107] In one embodiment, the geometric defect category includes at least one of the following: area defect and pose defect. An area defect refers to an incomplete or excessive area of ​​the object under test. A pose defect refers to an abnormal pose of the object under test at the target location. Pose defects may include rotation angle defects and position offset defects. Figure 3 The diagram illustrates several examples of geometric defects, such as area defects caused by missing areas of the object under test, and pose defects caused by rotation or displacement of the object under test. Appearance defects include at least one of the following: color defects, smoothness defects, etc. Figure 4 As shown, several examples of appearance defects are provided, such as color defects caused by color discrepancies in the color of the object under test, and smoothness defects caused by scratches or pen marks on the object under test.

[0108] In one embodiment, when the object under test and the template object are electronic components, the appearance defect category may include at least one of color defects and smoothness defects. Color defects include at least one of color errors or color contamination. Smoothness defects include at least one of scratches, cracks, dents, material adhesion, or material shortage.

[0109] Color error refers to the incorrect use of color on the surface of electronic components. For example, the surface of an electronic component should be black, but green pigment was used, resulting in a green appearance. Color dirt refers to other stains, such as other colors, on the surface of electronic components.

[0110] A crack is a fissure that forms in a material under at least one of the effects of stress and environmental conditions. Indentation refers to a depression on the surface of an electronic component caused by external force during manufacturing. Material adhesion refers to excess material adhering to the surface of an electronic component, resulting in a raised area. Material shortage refers to incomplete or missing parts in the mold cavity during manufacturing, leading to an incomplete or defective shape. A scratch refers to surface abrasions caused by friction on an electronic component.

[0111] In one embodiment, the template object features include the template appearance features of the template object, the test object features include the test geometric features and the test appearance features of the test object, and the defect detection results include the geometric defect detection results and the appearance defect detection results;

[0112] Based on the features of the template object and the features of the object under test, defect detection is performed on the object under test at the target location to obtain the defect detection results of the object under test, including:

[0113] Based on the geometric features to be tested, geometric defects are detected on the object at the target location to obtain the geometric defect detection results of the object; based on the appearance features of the template and the appearance features to be tested, appearance defects are detected on the object at the target location to obtain the appearance defect detection results of the object.

[0114] In this embodiment, a defect detection model is deployed on the computer device, and the acquired image to be tested and template image are input into the defect detection model. The defect detection model extracts template object features from the template image and extracts test object features from the image to be tested. Based on the template object features and the test object features, the defect detection model performs defect detection on the test object at the target location and obtains the defect detection result of the test object.

[0115] In this embodiment, the defect detection model is obtained through training. The training steps include: acquiring a test sample image obtained by taking a picture of the test sample object at the sample location; acquiring a template sample image obtained by taking a picture of a template sample object at the sample location, wherein the test sample object and the template sample object belong to the same sample object category; extracting template sample object features from the template sample image and extracting test sample object features from the test sample image; performing defect detection on the test sample object at the sample location based on the template sample object features and the test sample object features to obtain a predicted defect detection result for the test sample object; obtaining the sample defect detection result corresponding to the test sample object; and training the model based on the difference between the predicted defect detection result and the sample defect detection result to obtain the defect detection model.

[0116] In the aforementioned defect detection method, an image of the object to be tested at the target location is acquired by photographing the object to be tested. The object to be tested and the template object belong to the same object category, and the template object is used as a standard reference object. A template image of the template object at the target location is acquired and used as the standard reference image. Template object features of the template object are extracted from the template image, and object features of the object to be tested are extracted from the image to be tested. Based on these template object features and object features, defect detection is performed on the object to be tested at the target location to identify defects present in the object to be tested, obtaining the defect detection results. This method can more accurately and comprehensively detect defects present in the same location, reducing the false negative rate and effectively improving the accuracy and comprehensiveness of defect detection.

[0117] In one embodiment, the template object features include template geometric features, the test object features include the test geometric features and the test appearance features of the test object, and the defect detection results include geometric defect detection results and appearance defect detection results.

[0118] Based on the features of the template object and the features of the object under test, defect detection is performed on the object under test at the target location to obtain the defect detection results of the object under test, including:

[0119] Based on the template geometric features and the geometric features to be tested, geometric defects are detected on the object to be tested at the target location to obtain the geometric defect detection results of the object to be tested; based on the appearance features to be tested, appearance defects are detected on the object to be tested at the target location to obtain the appearance defect detection results of the object to be tested.

[0120] Specifically, the computer device can extract the geometric features and appearance features of the object to be tested from the image to be tested. The computer device can also extract the template geometric features of the template object from the template image.

[0121] Computer equipment performs geometric defect detection on the object to be measured at a target location based on the geometric features of a template and the geometric features to be measured, and obtains the geometric defect detection result of the object to be measured. Geometric defect detection includes at least one of area detection or pose detection to detect whether there are defects in the geometric shape, area size, rotation angle, or position of the object to be measured. The geometric defect detection result includes at least one of area detection result or pose detection result.

[0122] Pose detection can detect the position of the object under test at a target location, as well as the rotation angle of the object under test relative to the target location. For example, the left tilt angle is 87.5°, the right tilt angle is 87.1°, the down tilt angle is 2.9°, and the up tilt angle is 1.7°.

[0123] Computer equipment performs appearance defect detection on the object at a target location based on the appearance features to be tested, and obtains the appearance defect detection results of the object. The appearance defect detection includes at least one of color detection and smoothness detection to detect whether the object has color defects, smoothness defects, etc. The appearance defect detection results include at least one of color defect detection results and smoothness defect detection results.

[0124] In this embodiment, the template object features also include template appearance features. The computer device performs appearance defect detection on the object to be tested at the target location based on the appearance features to be tested and the template appearance features, and obtains the appearance defect detection results of the object to be tested.

[0125] In one embodiment, the defect detection model includes a geometric defect detection model and an appearance defect detection model. The geometric defect detection model extracts template geometric features of the template object from the template image and extracts test geometric features of the object to be tested from the image to be tested. Based on the template geometric features and the test geometric features, the geometric defect detection model performs geometric defect detection on the object to be tested at the target location to obtain the geometric defect detection result of the object to be tested.

[0126] The appearance defect detection model extracts the appearance features of the object to be tested from the image to be tested, and performs appearance defect detection on the object to be tested at the target location based on the appearance features to be tested, thereby obtaining the appearance defect detection result of the object to be tested.

[0127] In this embodiment, the geometric defect detection model includes a geometric feature extraction layer and a geometric defect detection layer. The geometric feature extraction layer extracts template geometric features and the geometric features to be tested. The geometric defect detection layer performs geometric defect detection on the object to be tested based on the template geometric features and the geometric features to be tested, and obtains the geometric defect detection result.

[0128] In this embodiment, the appearance defect detection model includes an appearance feature extraction layer and an appearance defect detection layer. The appearance feature extraction layer extracts the appearance features to be tested, and the appearance defect detection layer performs appearance defect detection on the object to be tested based on the appearance features to be tested, thereby obtaining the appearance defect detection result.

[0129] In this embodiment, the appearance defect detection model includes an appearance feature extraction layer and an appearance defect detection layer. The appearance feature extraction layer extracts the appearance features of the template and the appearance features to be tested. The appearance defect detection layer performs appearance defect detection on the object under test based on the appearance features of the template and the appearance features to be tested, thereby obtaining the appearance defect detection result.

[0130] In this embodiment, geometric defect detection is performed on the object under test at the target location based on the template geometric features and the geometric features to be tested, to detect whether there are geometric defects in the object under test and obtain the geometric defect detection result. Based on the appearance features to be tested, appearance defect detection is performed on the object under test at the target location to detect whether there are appearance defects in the object under test and obtain the appearance defect detection result. By using geometric features and appearance features, the geometric defect detection and appearance defect detection can be complementary, thereby more accurately and comprehensively detecting defects existing in various aspects of the object under test, reducing the missed defect rate, and effectively improving the accuracy and comprehensiveness of defect detection.

[0131] In one embodiment, geometric defect detection is performed on the object to be tested at the target location based on the template geometric features and the geometric features to be tested, to obtain the geometric defect detection result of the object to be tested, including:

[0132] Based on the geometric features of the template and the geometric features to be tested, the geometric similarity between the template object in the template image and the object to be tested in the image to be tested is determined; based on the difference between the geometric similarity and the preset geometric similarity, the geometric defect detection result of the object to be tested is generated.

[0133] Geometric similarity includes at least one of area intersection-union ratio, area difference, positional difference, or rotation angle difference. Preset geometric similarity includes at least one of preset intersection-union ratio, preset area difference, preset positional difference, or preset rotation angle difference.

[0134] A larger area intersection-union ratio indicates higher geometric similarity; a smaller area difference indicates higher geometric similarity; a smaller positional difference indicates higher geometric similarity; and a smaller difference in rotation angle indicates higher geometric similarity.

[0135] Specifically, the computer device determines the geometric similarity between the template object in the template image and the object to be tested in the image based on the geometric features of the template and the geometric features to be tested. The computer then determines the difference between the geometric similarity and a preset geometric similarity, and performs geometric defect detection on the object to be tested at the target location based on this difference, thereby obtaining the geometric defect detection result of the object to be tested.

[0136] In this embodiment, when the difference between the geometric similarity and the preset geometric similarity meets the preset difference condition, it indicates that the object under test has no geometric defects. When the difference between the geometric similarity and the preset geometric similarity does not meet the preset difference condition, it indicates that the object under test has geometric defects.

[0137] For example, if the cross-union ratio of the object under test relative to the template object is less than the preset cross-union ratio, it indicates that the object under test has geometric defects; if the cross-union ratio is greater than or equal to the preset cross-union ratio, it indicates that the object under test does not have geometric defects.

[0138] Determine the area difference between the area of ​​the object to be tested and the area of ​​the template object. If the area difference is greater than the preset area difference, there is a geometric defect. If the area difference is less than or equal to the preset area difference, it means that there is no geometric defect in the object to be tested.

[0139] The positional difference between the object position of the test object and the object position of the template object is determined. If the positional difference is greater than a preset positional difference, a geometric defect exists; if the positional difference is less than or equal to the preset positional difference, the test object does not have a geometric defect. The object position of the test object can be, for example, the coordinates of the test object at the target location, and the object position of the template object can be, for example, the coordinates of the template object at the target location.

[0140] Determine the rotation angle of the object under test and the angular difference between the two. If the difference is greater than a preset rotation angle difference, a geometric defect exists; if it is less than the preset rotation angle difference, the object under test does not have a geometric defect. The rotation angle of the object under test can be, for example, the rotation angle of the object under test relative to the target position, and the rotation angle of the template object can be, for example, the rotation angle of the template object relative to the target position.

[0141] In this embodiment, the presence of a geometric defect in the object under test is indicated when at least one of the following conditions exists:

[0142] The intersection-union ratio of the object under test relative to the template object is less than the preset intersection-union ratio; the area difference between the area of ​​the object under test and the area of ​​the template object is greater than the preset area difference; the position difference between the positions of the objects under test and the positions of the objects under test is greater than the preset position difference; and the angle difference between the rotation angles of the objects under test and the rotation angles of the objects under test is greater than the preset rotation angle difference.

[0143] In this embodiment, the geometric similarity between the template object in the template image and the object to be tested in the test image is determined based on the geometric features of the template and the geometric features to be tested. Based on the difference between the geometric similarity and the preset geometric similarity, the geometric similarity can be accurately determined to determine whether there are geometric defects in the object to be tested, and a geometric defect detection result can be generated.

[0144] In one embodiment, based on the appearance features to be tested, appearance defect detection is performed on the object to be tested at the target location to obtain the appearance defect detection result of the object to be tested, including:

[0145] Based on the appearance features to be tested, connected component identification is performed on the image to be tested to obtain multiple connected components in the image to be tested; the anomaly scores of each of the multiple connected components are determined, and the appearance defect detection results of the object to be tested are generated based on the anomaly scores of each of the multiple connected components.

[0146] In this context, the outlier score of a connected component can be considered as the confidence score of that component. The confidence score of a connected component refers to the probability that the component contains defects.

[0147] Specifically, the image to be tested can be divided into multiple image blocks, and the appearance features of each image block can be extracted to obtain the appearance features to be tested, which include the appearance features of each image block. Based on the appearance features to be tested, connected component identification is performed on the image to be tested to obtain multiple connected components.

[0148] In this embodiment, the computer device performs binarization processing on the appearance features to be tested to obtain a binarized image of the same size as the image to be tested. Connected component identification is then performed on the binarized image to obtain multiple connected components. The computer device determines the anomaly score for each of the multiple connected components and generates an appearance defect detection result for the object under test based on the anomaly scores of each of the multiple connected components.

[0149] Furthermore, the anomaly score can be a confidence score. By locating the connected components through bounding boxes, and based on the confidence score corresponding to each bounding box, bounding boxes with a confidence score greater than the confidence score threshold are filtered out. The connected components in the filtered bounding boxes are the locations where defects exist.

[0150] In this embodiment, the bounding box with the highest confidence level can be selected from the confidence levels corresponding to each bounding box, and the connected components in the bounding box with the highest confidence level are the locations where defects exist.

[0151] In this embodiment, the maximum confidence value can be used as the anomaly score of the image, and the precise location of the appearance defect in the image under test can be located based on the anomaly score.

[0152] In this embodiment, based on the appearance features to be tested, connected component identification is performed on the image to be tested to obtain multiple connected components in the image to be tested, and the anomaly scores of each of the multiple connected components are determined. Based on the anomaly scores of each of the multiple connected components, the appearance defect detection result of the object to be tested is generated, which can accurately detect the appearance defects of the object to be tested.

[0153] In one embodiment, the template object features further include template appearance features. Based on the appearance features to be tested, appearance defect detection is performed on the object to be tested at the target location to obtain the appearance defect detection result of the object to be tested, including:

[0154] Based on the appearance features of the template and the appearance features to be tested, the appearance similarity between the template object in the template image and the object to be tested in the image to be tested is determined; based on the difference between the appearance similarity and the preset appearance similarity, the appearance defect detection result of the object to be tested is generated.

[0155] The appearance similarity includes at least one of color difference and smoothness difference. The preset appearance similarity includes at least one of preset color difference and preset smoothness difference.

[0156] The smaller the color difference, the higher the appearance similarity; the smaller the smoothness difference, the higher the appearance similarity.

[0157] Specifically, the computer equipment determines the appearance similarity between the template object in the template image and the object to be tested in the image based on the appearance features of the template and the appearance features to be tested. The computer equipment determines the difference between the appearance similarity and the preset appearance similarity, and performs appearance defect detection on the object to be tested at the target location based on the difference, thereby obtaining the appearance defect detection result of the object to be tested.

[0158] In this embodiment, when the difference between the appearance similarity and the preset appearance similarity meets the preset difference condition, it indicates that the object under test has no appearance defects. When the difference between the appearance similarity and the preset appearance similarity does not meet the preset difference condition, it indicates that the object under test has appearance defects.

[0159] For example, determine the color difference between the color of the object under test and the color of the template object. If the color difference is greater than the preset color difference, there is an appearance defect. If the color difference is less than or equal to the preset color difference, it means that the object under test does not have an appearance defect.

[0160] Determine the smoothness of the object under test and the difference between the smoothness of the object under test. If the difference in smoothness is greater than the preset difference in smoothness, there is an appearance defect. If the difference in smoothness is less than the preset difference in smoothness, it means that there is no appearance defect in the object under test.

[0161] In this embodiment, the presence of an appearance defect in the object under test is indicated when at least one of the following conditions exists:

[0162] The color difference between the color of the object under test and the color of the template object is greater than the preset color difference; the smoothness difference between the smoothness of the object under test and the smoothness of the object under test is greater than the preset smoothness difference.

[0163] In this embodiment, based on the appearance features of the template and the appearance features to be tested, the appearance similarity between the template object in the template image and the object to be tested in the image to be tested is determined. Based on the difference between the appearance similarity and the preset appearance similarity, it is possible to accurately determine whether the object to be tested has appearance defects and generate appearance defect detection results.

[0164] In one embodiment, based on the features of the template object and the features of the object under test, defect detection is performed on the object under test at the target location to obtain the defect detection result of the object under test, including:

[0165] The process involves: obtaining a mask image of the template image (obtained by masking the template image); segmenting the template region containing the template object from the template image based on the mask features and template object features; segmenting the test region containing the test object from the test image based on the mask features, template object features, and test object features; and performing defect detection on the test object at the target location based on the template region and the test region to obtain the defect detection result of the test object.

[0166] Specifically, the computer device performs mask annotation on the template image to obtain a mask image. The computer device extracts mask features from the mask image and, based on these mask features, extracts template object features of the template object from the template image. Based on the mask features and template object features of the mask image, the computer device performs segmentation processing on the template image to segment the template region containing the template object.

[0167] Computer equipment performs segmentation processing on the image under test based on mask features, template object features, and features of the object under test, in order to segment the test region where the object under test is located from the image under test.

[0168] Computer equipment performs defect detection on the object under test at the target location based on a template area and a test area, obtaining the defect detection result of the object under test. This defect detection can be geometric defect detection or appearance defect detection.

[0169] In one embodiment, the template object features include template geometric features, and the test object features include the test object's geometric features and test appearance features. Based on the mask features, template geometric features, and test object features, the test region where the test object is located is segmented from the test image. Based on the template region and the test region, geometric defect detection is performed on the test object at the target location to obtain the geometric defect detection result of the test object.

[0170] In this embodiment, the appearance features of the object to be tested can also be extracted from the area to be tested, so as to perform appearance defect detection on the object to be tested based on the appearance features and obtain the appearance defect detection result of the object to be tested.

[0171] In this embodiment, the template object features also include template appearance features, which can be extracted from the template area to perform appearance defect detection on the object under test based on the template appearance features and the appearance features to be tested.

[0172] like Figure 5 The diagram shown is an architecture diagram of geometric defect detection in one embodiment, including template registration, image segmentation, and geometric defect detection.

[0173] Template registration: For each electronic component at a soldering location, a template image and a mask image obtained by annotating the electronic component in the template image are required. The template image and the mask image are combined into a template pair, and the template pair is registered to the template dataset.

[0174] Image segmentation: Perform image segmentation on the image to be tested, the template image, and the mask image to obtain the segmentation result.

[0175] The execution flow of image segmentation is as follows: Figure 6 As shown, the feature extractor extracts the geometric features of the test image, the template geometric features of the template image, and the mask features of the mask image. The decoder outputs the segmentation results of the test image and the template object, respectively. The segmentation result of the test image refers to the region in the test image where the test object is located, i.e., the test region.

[0176] Geometric defect detection: Calculates the intersection-over-union ratio (IoU), area difference, and angle difference between the segmentation result of the image under test and the segmentation result of the template object. Geometric defect detection results are generated based on these IoU, area difference, and angle difference.

[0177] In this embodiment, both the feature extractor and the decoder can be based on the transformer structure.

[0178] In this embodiment, a mask image of the template image is obtained. The mask image is obtained by masking and annotating the template image to mark the position of the template object in the image. Based on the mask features and template object features, the template region where the template object is located is accurately segmented from the template image. Based on the mask features, template object features, and test object features, the test region where the test object is located is further accurately segmented from the test image. Based on the image segmentation results, the test region, and the test region, defect detection is performed on the test object at the target location, which can improve the accuracy of defect detection of the test object.

[0179] like Figure 7 The diagram illustrates a flowchart of a defect detection method in one embodiment. A computer device acquires a test image, a template image, and a mask image. Based on these images, image segmentation is performed to obtain segmentation results for the test image and the template image. The segmentation results are used to detect whether geometric defects exist in the test object within the test image. If so, the geometric defect detection result is directly output. If no geometric defects exist, the test object is further subjected to appearance defect detection, and the appearance defect detection result is output.

[0180] In one embodiment, a defect detection method is provided, applied to a computer device, comprising:

[0181] Obtain the template image obtained by taking a picture of the template object at the target location.

[0182] For each object to be tested, a test image is obtained by taking a picture of the target object at the target location, so as to obtain a test image for each object to be tested. Each object to be tested and the template object belong to the same object category.

[0183] Obtain the mask image of the template image. The mask image is obtained by masking the template image.

[0184] Extract template geometric features from the template image, extract the test geometric features and test appearance features of the test object from the test image, and extract mask features from the mask image.

[0185] Based on the mask features and template geometric features of the mask image, the template region where the template object is located is segmented from the template image.

[0186] Based on mask features, template geometric features, and the geometric features to be tested, the region to be tested where the object to be tested is located is segmented from the image to be tested.

[0187] Based on the template region and the test region, the geometric similarity between the template object in the template image and the test object in the test image is determined.

[0188] Based on the difference between the geometric similarity and the preset geometric similarity, the geometric defect detection results of the object under test are generated.

[0189] Based on the appearance features of the object under test, appearance defects are detected at the target location to obtain the appearance defect detection results of the object under test.

[0190] In one embodiment, the target location is a soldering location on a printed circuit board, the object under test is an electronic component soldered at the soldering location, the template object is a standard electronic component at the soldering location, the object category is an electronic component category, and the defect detection result includes the defect category and defect location of the electronic component under test.

[0191] Specifically, the computer equipment acquires an image of the electronic component under test (DUT) at the soldering location on a printed circuit board, and a template image of a standard electronic component at the same soldering location. The DUT and the standard electronic component belong to the same electronic component category. The computer equipment extracts the standard electronic component features from the template image and the DUT features from the DUT image. Based on the standard and DUT features, the computer equipment performs defect detection on the DUT at the soldering location, obtaining the defect detection result.

[0192] In this embodiment, an image of the electronic component under test (DUT) at the soldering location on the printed circuit board is acquired, along with a template image of a standard electronic component at the same soldering location. The DUT and the standard electronic component belong to the same category, serving as a reference for the DUT, and the image of the standard electronic component is used as a reference image for the DUT image. Standard electronic component features are extracted from the template image, and features of the DUT are extracted from the DUT image. Based on these features, defect detection is performed on the DUT at the soldering location on the printed circuit board. This method more accurately and comprehensively detects defects at each soldering location on the printed circuit board, reducing the missed defect rate and effectively improving the accuracy and comprehensiveness of electronic component defect detection.

[0193] In one embodiment, such as Figure 8 As shown, a defect detection model processing method is provided, which can be applied to computer devices (such as computer devices) Figure 1 Taking a terminal or server as an example, the following steps are included:

[0194] Step 802: Obtain the image of the sample to be tested obtained by taking a picture of the sample object at the sample location.

[0195] The sample object to be tested refers to the object that needs to be inspected for defects, such as electronic components, glass, ceramics, plastics, etc., but not limited to these. The sample location refers to the location of the sample object to be tested.

[0196] The test sample image refers to an image containing the test sample object. The test sample image can be an image of any format, such as a color image, a depth image, or a YUV image, but is not limited to these.

[0197] Specifically, the computer device can acquire a test sample image including the test sample object, which is obtained by taking a picture of the test sample object at the sample location.

[0198] In this embodiment, the image of the sample to be tested can be an image directly captured by a computer device, an image obtained locally, or an image obtained from other devices.

[0199] In this embodiment, the image of the sample to be tested is an image obtained by taking a picture of the sample object at the sample location under at least one of a preset shooting angle or a preset light source.

[0200] Step 804: Obtain the template sample image obtained by taking a picture of the template sample object at the sample location. The sample object to be tested and the template sample object belong to the same sample object category.

[0201] The template sample object is a standard object without defects, serving as a reference object for the sample object to be tested. The sample object to be tested and the template sample object share the same sample object category. The sample object category refers to the type of object, such as the category of electronic components or the category of porcelain.

[0202] Specifically, the computer device can acquire a template sample image including the template sample object, which is obtained by taking a picture of the template sample object at the sample location.

[0203] In this embodiment, the template sample image can be an image directly captured by a computer device, an image obtained locally, or an image obtained from other devices.

[0204] In one embodiment, the template sample image may be a pre-stored image that includes a template sample object.

[0205] In this embodiment, the template sample image and the sample image to be tested have the same shooting configuration information. The shooting configuration information includes at least one of the shooting device, shooting angle, or shooting light source.

[0206] Step 806: Extract template sample object features from template sample image and extract test sample object features from test sample image.

[0207] Specifically, the computer device extracts template sample object features from the template sample image and extracts template sample object features from the template sample image.

[0208] In this embodiment, the computer device can extract the geometric features and appearance features of the sample object from the sample image.

[0209] In this embodiment, the computer device can extract the template sample geometric features and template sample appearance features of the template sample object from the template sample image.

[0210] Step 808: Based on the features of the template sample object and the features of the sample object to be tested, perform defect detection on the sample object to be tested at the sample location to obtain the predicted defect detection result of the sample object to be tested.

[0211] The predicted defect detection results include at least one of the following: defect category, defect location, or defect area.

[0212] Furthermore, the defect categories include at least one of the following: geometric defect category and appearance defect category.

[0213] The predicted defect detection results include at least one of the predicted geometric defect detection results or the predicted appearance defect detection results. The predicted geometric defect detection results include at least one of the geometric defect category, defect location, or defect area. The predicted appearance defect detection results include at least one of the appearance defect category, defect location, or defect area.

[0214] Specifically, the computer equipment performs defect detection on the sample object at the sample location based on the features of the template sample object and the features of the sample object to be tested, in order to detect whether the sample object to be tested has geometric defects or appearance defects, and obtain the predicted defect detection results of the sample object to be tested.

[0215] In this embodiment, the computer device segments the template sample image based on the features of the template sample object to obtain a template sample region including the template sample object. The test sample image is then segmented based on the features of the test sample object to obtain a test sample region including the test sample object. Based on the template sample region and the test sample region, defect detection is performed on the test sample object at the sample location to obtain a predicted defect detection result for the test sample object.

[0216] In one embodiment, the geometric defect category includes at least one of the following: area defect, pose defect. The appearance defect category includes at least one of the following: color defect, smoothness defect.

[0217] In one embodiment, the template sample object features include the template sample appearance features of the template sample object, the test sample object features include the test sample geometric features and the test sample appearance features of the test sample object, and the predicted defect detection results include the predicted geometric defect detection results and the predicted appearance defect detection results;

[0218] Based on the features of the template sample object and the features of the sample object to be tested, defect detection is performed on the sample object to be tested at the sample location to obtain the predicted defect detection results of the sample object to be tested, including:

[0219] Based on the geometric features of the sample to be tested, geometric defects are detected at the sample location to obtain the predicted geometric defect detection results of the sample to be tested; based on the appearance features of the template sample and the appearance features of the sample to be tested, appearance defects are detected at the sample location to obtain the predicted appearance defect detection results of the sample to be tested.

[0220] In this embodiment, the template sample object features include the template sample geometric features, the test sample object features include the test sample geometric features and the test sample appearance features, and the predicted defect detection results include the predicted geometric defect detection results and the predicted appearance defect detection results.

[0221] Based on the features of the template sample object and the features of the sample object to be tested, defect detection is performed on the sample object to be tested at the sample location to obtain the predicted defect detection results of the sample object to be tested, including:

[0222] Based on the geometric features of the template sample and the geometric features of the sample to be tested, geometric defects are detected at the sample location to obtain the predicted geometric defect detection results of the sample to be tested; based on the appearance features of the sample to be tested, appearance defects are detected at the sample location to obtain the predicted appearance defect detection results of the sample to be tested.

[0223] In this embodiment, based on the geometric features of the template sample and the geometric features of the sample to be tested, geometric defect detection is performed on the sample object at the sample location to obtain the predicted geometric defect detection result of the sample object to be tested, including:

[0224] Based on the geometric features of the template sample and the geometric features of the test sample, the predicted geometric similarity between the template sample object in the template sample image and the test sample object in the test sample image is determined; based on the difference between the predicted geometric similarity and the preset geometric similarity, the predicted geometric defect detection result of the test sample object is generated.

[0225] In this embodiment, the template sample object features also include template sample appearance features. Based on the appearance features of the sample to be tested, appearance defect detection is performed on the sample object at the sample location to obtain the predicted appearance defect detection result of the sample object to be tested, including:

[0226] Based on the appearance features of the template sample and the appearance features of the sample to be tested, the predicted appearance similarity between the template sample object in the template sample image and the sample to be tested in the sample to be tested image is determined; based on the difference between the predicted appearance similarity and the preset appearance similarity, the predicted appearance defect detection result of the sample to be tested is generated.

[0227] In one embodiment, based on the features of the template sample object and the features of the sample object to be tested, defect detection is performed on the sample object to be tested at the sample location to obtain the predicted defect detection result of the sample object to be tested, including:

[0228] The process involves: obtaining a mask image of the template sample image (obtained by masking and annotating the template sample image); segmenting the template sample region containing the template sample object from the template sample image based on the mask features and template sample object features; segmenting the test sample region containing the test sample object from the test sample image based on the mask features, template sample object features, and test sample object features; and performing defect detection on the test sample object at the sample location based on the template sample region and the test sample region to obtain the predicted defect detection result for the test sample object.

[0229] Step 810: Obtain the sample defect detection results corresponding to the sample object to be tested, and train the model based on the difference between the predicted defect detection results and the sample defect detection results to obtain the defect detection model.

[0230] Among them, the sample defect detection result is the actual defect detection result of the sample object to be tested, and it is also the defect detection result expected to be output by the model during training. The sample defect detection result serves as the label for model training.

[0231] Specifically, the computer equipment acquires the sample defect detection results corresponding to the sample object to be tested, and determines the difference between the predicted defect detection results and the sample defect detection results. The computer equipment trains the model based on this difference to adjust the model until the training stops when the stopping condition is met, thus obtaining the defect detection model.

[0232] Specifically, the training stopping conditions can be: the number of training iterations reaches a preset number; the difference between the predicted defect detection result and the sample defect detection result is less than or equal to a preset difference; or the model loss is less than or equal to a preset loss.

[0233] In the aforementioned defect detection model processing method, an image of the test sample object at the sample location is acquired by taking a picture of the test sample object. The test sample object and the template sample object belong to the same sample object category, and the template sample object is used as the standard object. A template image of the template sample object at the sample location is acquired and used as the standard image. Template sample object features are extracted from the template sample image, and test sample object features are extracted from the test sample image. Based on these template sample object features and test sample object features, defect detection is performed on the test sample object at the sample location, obtaining the predicted defect detection result for the test sample object output by the model. The sample defect detection result corresponding to the test sample object is obtained. The model is trained based on the difference between the predicted defect detection result and the sample defect detection result, so that the defect detection result output by the model gradually approaches the real defect detection result during training. This allows the obtained defect detection model to more accurately and comprehensively detect defects existing in different test objects at the same location, reducing the false negative rate and effectively improving the accuracy and comprehensiveness of defect detection. Furthermore, the defect detection model enables rapid execution of defect detection tasks, thereby improving the efficiency of defect detection.

[0234] In one embodiment, the image of the sample object to be tested includes a positive sample object image, and the features of the sample object to be tested include positive sample object features; the method further includes:

[0235] Noise is added to the positive sample object features to obtain the negative sample object features; based on the positive sample object features and the negative sample object features respectively, the sample category of the positive sample object image is determined, and the prediction and discrimination results for the positive sample object features and the prediction and discrimination results for the negative sample object features are obtained.

[0236] The model is trained based on the difference between the predicted defect detection results and the sample defect detection results to obtain a defect detection model, including:

[0237] Obtain the expected discrimination results for positive sample object features and the expected discrimination results for negative sample object features; train the model based on the differences between the predicted discrimination results and the expected discrimination results for positive sample object features, the differences between the predicted discrimination results and the expected discrimination results for negative sample object features, and the differences between the predicted defect detection results and the sample defect detection results to obtain a defect detection model.

[0238] Specifically, computer equipment can add noise to the features of positive sample objects to obtain the features of negative sample objects. Based on the features of positive sample objects, the computer equipment determines the sample category of the positive sample object image to determine whether the positive sample object image is a positive sample or a negative sample, obtaining a prediction result for the features of positive sample objects. The prediction result represents whether the positive sample object image is classified as a positive sample or a negative sample.

[0239] The sample category indicates whether the sample object image belongs to a positive sample or a negative sample, that is, whether the sample object image is a normal image or an abnormal image.

[0240] Computer equipment uses the features of negative sample objects to determine the sample category of positive sample object images, thereby determining whether a positive sample object image is a positive sample or a negative sample, and obtaining a prediction and discrimination result based on the features of negative sample objects.

[0241] The expected discrimination result for positive sample object features refers to the result the model expects to produce when using positive sample object features to discriminate positive sample object images. The expected discrimination result for negative sample object features refers to the result the model expects to produce when using negative sample object features to discriminate positive sample object images.

[0242] A defect detection model is obtained by training the model based on the differences between the predicted and expected discrimination results for positive sample object features, the differences between the predicted and expected discrimination results for negative sample object features, and the differences between the predicted defect detection results and the sample defect detection results.

[0243] In this embodiment, negative sample object features are obtained by adding noise to the positive sample object features, eliminating the need to collect negative samples for defect labeling during model training and greatly reducing data requirements. Based on both positive and negative sample object features, the sample category of the positive sample object image is determined, obtaining the predicted discrimination results for each feature. Model training is then performed based on the differences between the predicted and expected discrimination results for positive sample object features, the differences between the predicted and expected discrimination results for negative sample object features, and the differences between the predicted defect detection results and the sample defect detection results. This allows the model's predicted discrimination results for positive and negative samples to gradually approach the true results during training, thereby improving the model's accuracy and robustness.

[0244] In one embodiment, the image of the sample object to be tested includes a positive sample object image, and the features of the sample object to be tested include positive sample object features; the method further includes:

[0245] A negative sample object image is generated based on a positive sample object image, and negative sample object features are extracted from the negative sample object image. Based on the positive sample object features, the sample category of the positive sample object image is determined, and the prediction and discrimination result of the positive sample object image is obtained. Based on the negative sample object features, the sample category of the negative sample object image is determined, and the prediction and discrimination result of the negative sample object image is obtained.

[0246] The model is trained based on the difference between the predicted defect detection results and the sample defect detection results to obtain a defect detection model, including:

[0247] Obtain the expected discrimination results of positive sample object images and negative sample object images; train the model based on the differences between the predicted discrimination results and expected discrimination results of positive sample object images, the differences between the predicted discrimination results and expected discrimination results of negative sample object images, and the differences between the predicted defect detection results and sample defect detection results to obtain a defect detection model.

[0248] Specifically, the computer device generates a negative sample object image based on the positive sample object image, and extracts negative sample object features from the negative sample object image. Based on the positive sample object features, the sample category of the positive sample object image is determined, and a prediction result for the positive sample object image is obtained. Similarly, based on the negative sample object features, the sample category of the negative sample object image is determined, and a prediction result for the negative sample object image is obtained.

[0249] The computer equipment acquires the expected discrimination results for positive sample object images and the expected discrimination results for negative sample object images. Based on the differences between the predicted and expected discrimination results for positive sample object images, the differences between the predicted and expected discrimination results for negative sample object images, and the differences between the predicted defect detection results and the sample defect detection results, a defect detection model is trained to obtain the model.

[0250] Defect detection models include geometric defect detection models and appearance defect detection models. For example... Figure 9The diagram shows the architecture of an appearance defect detection model in one embodiment. Positive sample object images or positive sample object features are obtained from positive sample object images in the positive sample dataset. These positive sample object images or positive sample object features, along with noise data, are input into the appearance defect detection model. The appearance defect detection model generates pseudo-features, i.e., negative sample object features, based on the positive sample object features and the noise data. Specifically, the noise data can be data that causes the test sample object in the positive sample object image to rotate or shift. For example, the noise data could cause the left, right, lower, and upper angles of the test sample object in the positive sample object image to rotate by 87.5°, 87.1°, 2.9°, and 1.7°, respectively. The features obtained after this rotation are the pseudo-features. The appearance defect detection model determines whether the positive sample object image is a normal image or an abnormal image based on both the positive sample object features and the negative sample object features.

[0251] In this embodiment, a positive sample dataset is used to train the appearance defect detection model. The appearance defect detection process is as follows: Figure 10 As shown, the appearance defect detection model includes a feature extractor and a discriminator. The feature extractor adds noise to the features of positive sample objects to generate pseudo-features. These pseudo-features are the features of negative sample objects. Both the positive and negative sample object features are input into the discriminator for training. The discriminator then uses these features to determine whether a positive sample object image is a positive or negative sample. A positive sample indicates that the positive sample object image is classified as a normal image, while a negative sample indicates that the positive sample object image is classified as an abnormal image. This allows the discriminator to learn the ability to distinguish between normal and abnormal images during training.

[0252] Both the feature extractor and the discriminator employ convolutional networks. During training, the convolutional network of the feature extractor divides the positive sample object image into multiple image patches, and adds noise to a portion of each patch to obtain a noisy patch patch. This results in a negative sample object image that includes both the un-noisy image patch and the patch patch. Negative sample object features are then extracted from the negative sample object image.

[0253] Understandably, the noise addition operation is only used during training. Once the model is trained, there is no need to generate pseudo-features. The discriminator can be used directly to determine whether the input object image is a normal object or an abnormal image based on the object features.

[0254] The discriminator binarizes either the negative or positive sample object image to obtain a binarized image. Connected component detection is then performed on this binarized image to obtain each connected component. The size of the binarized image is the same as that of the negative sample object image. A connected component is a region in the binarized image that has the same pixel value. Each connected component is located using bounding boxes, and the confidence level corresponding to the bounding box is determined. The confidence level corresponding to the bounding box represents the probability of a defect existing within the bounding box. The confidence level corresponding to the bounding box is the confidence level of the connected component within that bounding box. The confidence level can be calculated based on the area of ​​the connected component; the larger the area of ​​the connected component, the higher the confidence level.

[0255] The discriminator can determine the anomaly score of the binarized image based on the confidence level of each bounding box. Based on the anomaly score, it can determine whether the negative or positive sample object image corresponding to the binarized image is a normal image or an anomaly image, thus enabling the discriminator to determine whether an image is a normal image or an anomaly image.

[0256] In this embodiment, the discriminator can perform a weighted summation of the confidence scores corresponding to each bounding box to obtain the anomaly score of the binarized image. Alternatively, based on the confidence scores corresponding to each bounding box, bounding boxes with confidence scores greater than a confidence threshold are selected, and the confidence scores of the selected bounding boxes are weighted summation to obtain the anomaly score of the binarized image. Alternatively, the highest confidence score among the confidence scores corresponding to each bounding box can be selected as the anomaly score of the binarized image.

[0257] In this embodiment, negative sample object images are generated based on positive sample object images, eliminating the need to collect negative samples for defect labeling during model training and significantly reducing data requirements. Negative sample object features are extracted from the negative sample object images. Based on the positive sample object features, the sample category of the positive sample object image is determined, obtaining the predicted discrimination result for the positive sample object image. Similarly, based on the negative sample object features, the sample category of the negative sample object image is determined, obtaining the predicted discrimination result for the negative sample object image. Model training is performed based on the differences between the predicted and expected discrimination results for positive and negative sample objects, the differences between the predicted and expected discrimination results for negative sample objects, and the differences between the predicted defect detection results and the sample defect detection results. This allows the model to gradually approach the true results in predicting and discriminating between positive and negative samples during training, thereby improving the model's accuracy and robustness.

[0258] In one embodiment, the predicted defect detection result includes the predicted geometric defect detection result and the predicted appearance defect detection result; the sample defect detection result corresponding to the sample object to be tested is obtained, and the model is trained based on the difference between the predicted defect detection result and the sample defect detection result to obtain the defect detection model, including:

[0259] Obtain the sample geometric defect detection results and sample appearance defect detection results corresponding to the sample object to be tested; determine the geometric defect detection loss of the sample object to be tested based on the sample geometric defect detection results and the predicted geometric defect detection results; determine the appearance defect detection loss of the sample object to be tested based on the sample appearance defect detection results and the predicted appearance defect detection results; train the model based on the geometric defect detection loss and the appearance defect detection loss to obtain the defect detection model.

[0260] Among them, the sample geometric defect detection result is the actual geometric defect detection result of the sample object under test, and it is also the geometric defect detection result expected to be output by the model training. The sample appearance defect detection result is the actual appearance defect detection result of the sample object under test, and it is also the appearance defect detection result expected to be output by the model training. The sample geometric defect detection result and the sample appearance defect detection result serve as labels for model training.

[0261] Specifically, the computer equipment can acquire the detection results of geometric defects and appearance defects of the sample object under test. Based on the geometric defect detection results and the predicted geometric defect detection results, the computer equipment determines the geometric defect detection loss of the sample object under test. Based on the appearance defect detection results and the predicted appearance defect detection results, the computer equipment determines the appearance defect detection loss of the sample object under test.

[0262] Computer equipment trains a model based on geometric defect detection loss and appearance defect detection loss to obtain a defect detection model.

[0263] In this embodiment, the geometric defect detection results and appearance defect detection results of the sample object to be tested are obtained. Based on the geometric defect detection results and the predicted geometric defect detection results, the geometric defect detection loss of the sample object to be tested is determined. Based on the appearance defect detection results and the predicted appearance defect detection results, the appearance defect detection loss of the sample object to be tested is determined. The model is trained based on the geometric defect detection loss and the appearance defect detection loss, so that the geometric defect detection results predicted by the model gradually approach the real geometric defect detection results during the training process, and the appearance defect detection results gradually approach the real appearance defect detection results. Finally, the defect detection model that has been trained can more accurately detect the geometric defects and appearance defects of the sample object to be tested.

[0264] This embodiment provides a defect detection model processing method and defect detection, which can be applied to various defect detection scenarios, such as defect detection scenarios for daily necessities, various industrial devices, electronic products, and electronic components.

[0265] This embodiment provides an application scenario for a defect detection model processing method and a defect detection application scenario, specifically an application scenario for defect detection of electronic components on printed circuit boards, including:

[0266] A printed circuit board assembly (PCBA) is the result of soldering or assembling electronic components onto a printed circuit board (PCB).

[0267] Printed circuit board (PCB) assembly defect detection aims to identify component defects that occur during the manufacturing process of PCB assemblies. Common electronic components include resistors and capacitors, and each component may exhibit various defects such as dents, scratches, discoloration, missing parts, and misalignment. This solution utilizes computer image technology and image-based defect detection methods to perform defect detection on PCBAs.

[0268] A component part number is a designation for each type of electronic component. Components of the same type have the same part number, while components of different types have different part numbers.

[0269] A printed circuit board has many soldering positions, each with a unique corresponding position number, also known as a reference locator. Electronic components of the same type can be mounted on different reference locators.

[0270] The sample location is the soldering location on the printed circuit board, the test sample object is the electronic component to be tested soldered at the soldering location, the template object is the standard electronic component at the soldering location, the sample object category is the electronic component category, and the predicted defect detection result includes the defect category and defect location of the electronic component to be tested.

[0271] The specific processing steps for this application scenario include the following parts:

[0272] Data collection: Collect template images of standard electronic components at each reference number on the printed circuit board, and collect test images of the electronic components under test at each reference number.

[0273] Template registration: For each electronic component at a reference number, a template image and a mask image obtained by marking the electronic component in the template image are required. Specifically, an image of a standard electronic component at the same part number-reference number on the printed circuit board is taken as the template image, and images of other electronic components under test are taken at the same part number-reference number as the test images.

[0274] The template image and the mask image are combined into a template pair, and the template pair is registered in the template dataset.

[0275] The defect detection models to be trained include geometric defect detection models and appearance defect detection models.

[0276] Geometric Defect Detection: Based on a geometric defect detection model, image segmentation is performed on the test image, template image, and mask image to obtain segmentation results. Specifically, a feature extractor extracts the geometric features of the test image, the template geometric features of the template image, and the mask features of the mask image. A decoder outputs the segmentation results of the test image and the template object, respectively. The segmentation result of the test image refers to the region where the electronic component to be tested is located, i.e., the test region. The intersection-over-union ratio (IoU), area difference, position difference, and rotation angle difference between the segmentation result of the test image and the segmentation result of a standard electronic component are calculated. Geometric defect detection results are generated based on the IoU, area difference, position difference, and rotation angle difference.

[0277] Appearance defect detection: Assuming the electronic component under test has no geometric defects, the appearance defect detection model is trained based on collected positive samples. The appearance defect detection model includes a feature extractor and a discriminator, both of which employ convolutional networks. The feature extractor adds noise to the positive sample object features in the positive sample object image to generate pseudo-negative sample features. These original positive sample features and pseudo-negative sample features are then input into the discriminator for training, enabling the discriminator to identify abnormal samples.

[0278] The positive sample object image is divided into multiple image blocks by a convolutional network with a feature extractor. Noise is added to a portion of these image blocks to obtain noise-added patch blocks, thereby obtaining a negative sample object image that includes both the image blocks without noise and the patch blocks.

[0279] The discriminator binarizes either the negative or positive sample object image to obtain a binarized image. The discriminator then performs connected component detection on the binarized image to obtain each connected component. Each connected component is located using bounding boxes, and the confidence score for each bounding box is determined.

[0280] The discriminator can determine the anomaly score of the binarized image based on the confidence level of each bounding box. Based on the anomaly score, it can determine whether the negative or positive sample object image corresponding to the binarized image is a normal image or an anomaly image, thus enabling the discriminator to determine whether an image is a normal image or an anomaly image.

[0281] In this embodiment, when a geometric defect is found in the electronic component under test during geometric defect detection, the appearance defect detection process can be skipped, and the image under test can be directly identified as an abnormal image and the corresponding defect category can be output.

[0282] In this embodiment, a deep learning-based defect detection method is proposed for printed circuit boards, which can detect and locate defects in electronic components, improving the practicality and robustness of defect detection. Furthermore, only positive samples need to be collected during model training, eliminating the need to collect negative samples for defect labeling, thus significantly reducing data requirements.

[0283] In addition, the defect detection method of this embodiment has a certain ability to detect new defects that have not appeared before, and can be adapted to the detection of different types of electronic components. It is also applicable to various types of defect detection and has strong versatility.

[0284] like Figure 11 As shown, in one embodiment, the defect detection results using the defect detection model on industrial dataset 1 are presented in the table, which provides the false detection rate and detection rate of abnormal defects.

[0285] like Figure 12 As shown, this is a defect detection result on industrial dataset 2 using a defect detection model in one embodiment. The table shows the false detection rate and detection rate of abnormal defects.

[0286] AUPR (Area Under the Precision-Recall Curve) focuses on the accuracy of predicting positive samples. Precision (TP) is the proportion of samples predicted as positive by the model that were actually positive. Recall (TPI) is the proportion of samples correctly identified as positive by the model out of all actual positive samples.

[0287] AURoc (Area Under the Receiver Operating Characteristic curve, i.e., the area under the ROC curve)

[0288] AUROC reflects the performance of a classifier by measuring the area between the receiver operating characteristic curve and the coordinate axis; a larger AUROC value indicates a higher accuracy.

[0289] from Figure 11 and Figure 12 The data shown demonstrates a very high defect detection rate and a very low false detection rate for electronic components. This indicates that the defect detection model proposed in this application can accurately detect and locate anomalies in various types of electronic components, thereby improving the practicality and robustness of defect detection for each type of electronic component.

[0290] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0291] Based on the same inventive concept, this application also provides a defect detection device for implementing the defect detection method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more defect detection device embodiments provided below can be found in the limitations of the defect detection method described above, and will not be repeated here.

[0292] In one embodiment, such as Figure 13 As shown, a defect detection device 1300 is provided, comprising:

[0293] The first acquisition module 1302 is used to acquire the image of the object to be tested obtained by taking a picture of the object to be tested at the target location.

[0294] The second acquisition module 1304 is used to acquire a template image obtained by taking a picture of the template object at the target location. The test object and the template object belong to the same object category.

[0295] Extraction module 1306 is used to extract template object features of template objects from template images and extract test object features of test objects from test images.

[0296] The detection module 1308 is used to perform defect detection on the object under test at the target location based on the features of the template object and the features of the object under test, and obtain the defect detection result of the object under test.

[0297] In one embodiment, the template object features include template geometric features, the test object features include the test geometric features and the test appearance features of the test object, and the defect detection results include geometric defect detection results and appearance defect detection results.

[0298] The detection module 1308 is also used to perform geometric defect detection on the object to be tested at the target location based on the geometric features of the template and the geometric features to be tested, and to obtain the geometric defect detection result of the object to be tested; and to perform appearance defect detection on the object to be tested at the target location based on the appearance features to be tested, and to obtain the appearance defect detection result of the object to be tested.

[0299] In one embodiment, the detection module 1308 is further configured to determine the geometric similarity between the template object in the template image and the object to be tested in the image to be tested based on the template geometric features and the geometric features to be tested; and generate a geometric defect detection result of the object to be tested based on the difference between the geometric similarity and the preset geometric similarity.

[0300] In one embodiment, the detection module 1308 is further configured to perform connected component recognition on the image to be tested based on the appearance features to be tested, obtain multiple connected components in the image to be tested; determine the anomaly score of each of the multiple connected components, and generate the appearance defect detection result of the object to be tested based on the anomaly score of each of the multiple connected components.

[0301] In one embodiment, the template object features further include template appearance features. The detection module 1308 is also used to determine the appearance similarity between the template object in the template image and the object to be tested in the image to be tested based on the template appearance features and the appearance features to be tested; and to generate the appearance defect detection result of the object to be tested based on the difference between the appearance similarity and the preset appearance similarity.

[0302] In one embodiment, the device further includes a segmentation module;

[0303] The segmentation module is used to obtain the mask image of the template image, which is obtained by masking the template image; based on the mask features and template object features of the mask image, the template region where the template object is located is segmented from the template image; based on the mask features, template object features and test object features, the test region where the test object is located is segmented from the test image.

[0304] The detection module 1308 is also used to perform defect detection on the object to be tested at the target location based on the template area and the area to be tested, and to obtain the defect detection result of the object to be tested.

[0305] In one embodiment, the target location is a soldering location on a printed circuit board, the object under test is an electronic component soldered at the soldering location, the template object is a standard electronic component at the soldering location, the object category is an electronic component category, and the defect detection result includes the defect category and defect location of the electronic component under test.

[0306] Based on the same inventive concept, this application also provides a defect detection model processing apparatus for implementing the defect detection model processing method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more defect detection model processing apparatus embodiments provided below can be found in the limitations of the defect detection model processing method described above, and will not be repeated here.

[0307] In one embodiment, such as Figure 14 As shown, a defect detection model processing device 1400 is provided, comprising:

[0308] The first sample acquisition module 1402 is used to acquire the image of the sample to be tested obtained by taking a picture of the sample object at the sample location.

[0309] The second sample acquisition module 1404 is used to acquire a template sample image obtained by taking a picture of the template sample object at the sample location. The sample object to be tested and the template sample object belong to the same sample object category.

[0310] The sample extraction module 1406 is used to extract template sample object features from the template sample image and test sample object features from the test sample image.

[0311] The sample detection module 1408 is used to perform defect detection on the sample object at the sample location based on the features of the template sample object and the features of the sample object to be tested, and to obtain the predicted defect detection result of the sample object to be tested.

[0312] The training module 1410 is used to obtain the sample defect detection results corresponding to the sample object to be tested, and to train the model based on the difference between the predicted defect detection results and the sample defect detection results to obtain the defect detection model.

[0313] In one embodiment, the image of the sample object to be tested includes a positive sample object image, and the features of the sample object to be tested include positive sample object features; the apparatus further includes:

[0314] The sample extraction module 1406 is also used to add noise to the features of positive sample objects to obtain the features of negative sample objects;

[0315] The discrimination module is used to distinguish the sample category of the positive sample object image based on the positive sample object features and the negative sample object features respectively, and to obtain the prediction discrimination results for the positive sample object features and the prediction discrimination results for the negative sample object features.

[0316] The training module 1410 is also used to obtain the expected discrimination results for positive sample object features and the expected discrimination results for negative sample object features; and to train the model based on the differences between the predicted discrimination results and the expected discrimination results for positive sample object features, the differences between the predicted discrimination results and the expected discrimination results for negative sample object features, and the differences between the predicted defect detection results and the sample defect detection results, so as to obtain a defect detection model.

[0317] In one embodiment, the image of the sample object to be tested includes a positive sample object image, and the features of the sample object to be tested include positive sample object features; the apparatus further includes:

[0318] The sample extraction module 1406 is also used to generate a negative sample object image based on the positive sample object image and extract negative sample object features from the negative sample object image.

[0319] The discrimination module is used to determine the sample category of a positive sample object image based on the features of the positive sample object, and obtain the predicted discrimination result of the positive sample object image; to determine the sample category of a negative sample object image based on the features of the negative sample object, and obtain the predicted discrimination result of the negative sample object image; and to obtain the expected discrimination result of the positive sample object image and the expected discrimination result of the negative sample object image.

[0320] The training module 1410 is also used to train the model based on the difference between the predicted and expected discrimination results of positive sample object images, the difference between the predicted and expected discrimination results of negative sample object images, and the difference between the predicted defect detection results and the sample defect detection results, so as to obtain a defect detection model.

[0321] In one embodiment, the predicted defect detection results include the predicted geometric defect detection results and the predicted appearance defect detection results, and the sample defect detection results include the sample geometric defect detection results and the sample appearance defect detection results.

[0322] The training module 1410 is also used to determine the geometric defect detection loss of the sample object to be tested based on the sample geometric defect detection results and the predicted geometric defect detection results; to determine the appearance defect detection loss of the sample object to be tested based on the sample appearance defect detection results and the predicted appearance defect detection results; and to train the model based on the geometric defect detection loss and the appearance defect detection loss to obtain a defect detection model.

[0323] Each module in the aforementioned defect detection device and defect detection model processing device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0324] In one embodiment, a computer device is provided, which may be a terminal or a server. Taking a terminal as an example, its internal structure diagram can be as follows: Figure 15 As shown, the computer device includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor provides computational and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage medium. The input / output interface is used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When executed by the processor, the computer program implements a defect detection method and a defect detection model processing method. The display unit of the computer device is used to form a visually visible image. It can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0325] Those skilled in the art will understand that Figure 15 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0326] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0327] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0328] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0329] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0330] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0331] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0332] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A defect detection method, characterized in that, The method includes: Acquire the image of the object to be tested at the target location by taking a picture; A template image is obtained by taking a picture of the template object at the target location, wherein the object to be tested and the template object belong to the same object category; Extract the template object features of the template object from the template image, and extract the test object features of the test object from the test image; Based on the features of the template object and the features of the object to be tested, defect detection is performed on the object to be tested at the target location to obtain the defect detection result of the object to be tested.

2. The method according to claim 1, characterized in that, The template object features include template geometric features, the test object features include the test object geometric features and the test appearance features of the test object, and the defect detection results include geometric defect detection results and appearance defect detection results; The step of performing defect detection on the object under test at the target location based on the features of the template object and the features of the object under test, and obtaining the defect detection result of the object under test, includes: Based on the template geometric features and the geometric features to be tested, geometric defect detection is performed on the object to be tested at the target location to obtain the geometric defect detection result of the object to be tested. Based on the appearance features to be tested, appearance defects are detected on the object to be tested at the target location to obtain the appearance defect detection results of the object to be tested.

3. The method according to claim 2, characterized in that, The step of performing geometric defect detection on the object to be tested at the target location based on the template geometric features and the geometric features to be tested, and obtaining the geometric defect detection result of the object to be tested, includes: Based on the template geometric features and the test geometric features, determine the geometric similarity between the template object in the template image and the test object in the test image; Based on the difference between the geometric similarity and the preset geometric similarity, the geometric defect detection result of the object to be tested is generated.

4. The method according to claim 2, characterized in that, The step of detecting appearance defects in the object at the target location based on the appearance features to be tested, and obtaining the appearance defect detection result of the object to be tested, includes: Based on the appearance features to be tested, connected component identification is performed on the image to be tested to obtain multiple connected components in the image to be tested. Determine the anomaly scores of each of the multiple connected components, and generate the appearance defect detection results of the object under test based on the anomaly scores of each of the multiple connected components.

5. The method according to claim 2, characterized in that, The template object features also include template appearance features. The step of performing appearance defect detection on the object at the target location based on the appearance features to be tested, and obtaining the appearance defect detection result of the object to be tested, includes: Based on the appearance features of the template and the appearance features to be tested, determine the appearance similarity between the template object in the template image and the object to be tested in the image to be tested; Based on the difference between the appearance similarity and the preset appearance similarity, the appearance defect detection result of the object to be tested is generated.

6. The method according to claim 1, characterized in that, The step of performing defect detection on the object under test at the target location based on the features of the template object and the features of the object under test, and obtaining the defect detection result of the object under test, includes: Obtain the mask image of the template image, wherein the mask image is obtained by masking the template image; Based on the mask features of the mask image and the features of the template object, the template region where the template object is located is segmented from the template image; Based on the mask features, the template object features, and the features of the object to be tested, the region to be tested where the object to be tested is located is segmented from the image to be tested; Based on the template area and the area to be tested, defect detection is performed on the object to be tested at the target location to obtain the defect detection result of the object to be tested.

7. The method according to any one of claims 1 to 6, characterized in that, The target location is the soldering location on the printed circuit board, the object to be tested is the electronic component to be tested soldered at the soldering location, the template object is the standard electronic component at the soldering location, the object category is the electronic component category, and the defect detection result includes the defect category and defect location of the electronic component to be tested.

8. A defect detection model processing method, characterized in that, The method includes: Acquire the image of the sample object to be tested obtained by taking a picture of the sample location; A template sample image is obtained by taking a picture of the template sample object at the sample location, wherein the sample object to be tested and the template sample object belong to the same sample object category; Extracting template sample object features of the template sample object from the template sample image, and extracting test sample object features of the test sample object from the test sample image; Based on the features of the template sample object and the features of the sample object to be tested, defect detection is performed on the sample object to be tested at the sample location to obtain the predicted defect detection result of the sample object to be tested. Obtain the sample defect detection result corresponding to the sample object to be tested, and train the model based on the difference between the predicted defect detection result and the sample defect detection result to obtain the defect detection model.

9. The method according to claim 8, characterized in that, The image of the sample object to be tested includes positive sample object images, and the features of the sample object to be tested include positive sample object features; the method further includes: Noise is added to the positive sample object features to obtain negative sample object features; Based on the positive sample object features and the negative sample object features respectively, the sample category of the positive sample object image is determined, and the prediction and discrimination results for the positive sample object features and the prediction and discrimination results for the negative sample object features are obtained. The step of training a model based on the difference between the predicted defect detection result and the sample defect detection result to obtain a defect detection model includes: Obtain the expected discrimination result for the positive sample object features and the expected discrimination result for the negative sample object features; A defect detection model is obtained by training the model based on the differences between the predicted and expected discrimination results for the features of the positive sample object, the differences between the predicted and expected discrimination results for the features of the negative sample object, and the differences between the predicted defect detection results and the sample defect detection results.

10. The method according to claim 8, characterized in that, The predicted defect detection results include predicted geometric defect detection results and predicted appearance defect detection results, and the sample defect detection results include sample geometric defect detection results and sample appearance defect detection results. The step of training a model based on the difference between the predicted defect detection result and the sample defect detection result to obtain a defect detection model includes: Based on the sample geometric defect detection results and the predicted geometric defect detection results, determine the geometric defect detection loss of the sample object to be tested; Based on the sample appearance defect detection results and the predicted appearance defect detection results, determine the appearance defect detection loss of the sample object to be tested; The model is trained based on the geometric defect detection loss and the appearance defect detection loss to obtain a defect detection model.

11. A defect detection device, characterized in that, The device includes: The first acquisition module is used to acquire the image of the object to be tested obtained by taking a picture of the object to be tested at the target location; The second acquisition module is used to acquire a template image obtained by taking a picture of the template object at the target location, wherein the object to be tested and the template object belong to the same object category; The extraction module is used to extract template object features of the template object from the template image and extract test object features of the test object from the test image; The detection module is used to perform defect detection on the object under test at the target location based on the features of the template object and the features of the object under test, and obtain the defect detection result of the object under test.

12. A defect detection model processing device, characterized in that, The device includes: The first sample acquisition module is used to acquire the image of the sample to be tested obtained by taking pictures of the sample to be tested at the sample location; The second sample acquisition module is used to acquire a template sample image obtained by taking a picture of the template sample object at the sample location, wherein the sample object to be tested and the template sample object belong to the same sample object category. The sample extraction module is used to extract template sample object features of the template sample object from the template sample image, and to extract test sample object features of the test sample object from the test sample image; The sample detection module is used to perform defect detection on the sample object at the sample location based on the features of the template sample object and the features of the sample object to be tested, and to obtain the predicted defect detection result of the sample object to be tested; The training module is used to obtain the sample defect detection results corresponding to the sample object to be tested, and to train the model based on the difference between the predicted defect detection results and the sample defect detection results to obtain the defect detection model.

13. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 10.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 10.

15. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 10.

Images