Defect detection method and device

Abstract

A defect detection method and device, relating to the field of computer vision, can reduce the computational complexity, reduce costs, improve detection speed, and obtain more accurate defect detection results in the defect detection process. The defect detection method comprises: first, obtaining information of multiple units to be tested in the image frame to be detected, and at least one unit template; second, according to the information of the multiple units to be tested, the multiple units to be tested are respectively template matched with the unit template to obtain a suspicious unit, and the suspicious unit is a unit to be tested whose matching degree with the unit template is lower than a preset threshold among the multiple units to be tested; finally, a machine learning algorithm is used to classify the suspicious unit to obtain the defect detection type of the suspicious unit.

Description

Technical Field

[0001] The present disclosure relates to the field of computer vision technology, and in particular to a defect detection method and device. Background Art

[0002] At present, there are many methods for defect detection. The traditional defect detection method is through manual naked eye detection. With the increasing application of computer vision, machine learning and computer vision technology can be used for defect detection to improve the detection speed and quality. However, when the resolution of the image frame to be detected is high, the amount of calculation is large and the calculation time is long, resulting in a slow detection speed, which is difficult to match the speed requirements of the industrial production line. If the resolution of the image frame to be detected is reduced, the accuracy of the defect detection results will be affected and the detection requirements cannot be met. Summary of the invention

[0003] The embodiments of the present disclosure provide a defect detection method and device, which reduce the computational complexity in the defect detection process, reduce costs, increase detection speed, and obtain more accurate defect detection results.

[0004] To achieve the above objectives, the embodiments of the present disclosure adopt the following technical solutions:

[0005] On the one hand, a defect detection method is provided. The method includes: obtaining information of multiple units to be tested in an image frame to be tested and at least one unit template; according to the information of the multiple units to be tested, performing template matching on the multiple units to be tested and the unit template respectively to obtain suspicious units, wherein the suspicious units are units to be tested whose matching degree with the unit template is lower than a preset threshold among the multiple units to be tested; and classifying the suspicious units using a machine learning algorithm to obtain the defect detection type of the suspicious units.

[0006] In some embodiments, obtaining the at least one unit template includes: randomly selecting M first units to be tested and a unit group to be tested from multiple units to be tested in the image frame to be detected, each of the unit groups to be tested including N second units to be tested, and each of the first units to be tested corresponding to one of the unit groups to be tested; respectively calculating the similarities between each of the first units to be tested and the N second units to be tested corresponding to the first unit to be tested, to obtain the N similarities corresponding to each of the first units to be tested; calculating the average similarity corresponding to the first units to be tested based on the N similarities corresponding to each of the first units to be tested; and determining the first unit to be tested with the highest average similarity among the M first units to be tested as the unit template.

[0007] In some embodiments, the information of the plurality of units under test includes at least one of position information and offset of at least one of the units under test.

[0008] In some embodiments, the method further includes: determining whether the suspicious cell is a normal cell or a defective cell according to the defect detection type of the suspicious cell; and when the suspicious cell is a defective cell, generating alarm information and outputting the defect detection type of the suspicious cell.

[0009] On the other hand, a defect detection device is provided, comprising: a processor and a transceiver; the transceiver is configured to obtain information of multiple units to be tested in an image frame to be tested, and at least one unit template; the processor is configured to: perform template matching on the multiple units to be tested with the unit templates respectively according to the information of the multiple units to be tested, to obtain suspicious units, wherein the suspicious units are units to be tested among the multiple units to be tested whose matching degree with the unit template is lower than a preset threshold; and classify the suspicious units using a machine learning algorithm to obtain the defect detection type of the suspicious units.

[0010] In some embodiments, the processor is configured to: randomly select M first units to be tested and a unit group to be tested from multiple units to be tested in the image frame to be detected, each of the unit groups to be tested including N second units to be tested, and each of the first units to be tested corresponds to one of the unit groups to be tested; respectively calculate the similarity between each of the first units to be tested and the N second units to be tested corresponding to the first unit to be tested, and obtain the N similarities corresponding to each of the first units to be tested; calculate the average similarity corresponding to the first units to be tested based on the N similarities corresponding to each of the first units to be tested; and determine the first unit to be tested with the highest average similarity among the M first units to be tested as the unit template.

[0011] In some embodiments, the information of the plurality of units under test includes at least one of position information and offset of at least one of the units under test.

[0012] In some embodiments, the processor is further configured to: determine whether the suspicious cell is a normal cell or a defective cell according to the defect detection type of the suspicious cell; and when the suspicious cell is a defective cell, generate alarm information and output the defect detection type of the suspicious cell.

[0013] The defect detection method provided by the embodiment of the present disclosure realizes rapid defect detection by combining traditional computer vision methods with deep learning technology. Template matching is used to screen the units to be tested in the image to obtain suspicious units, and the suspicious units are classified by machine learning algorithms to obtain the defect detection type of the suspicious units. In this way, the units to be tested that are rotated or changed in size that cannot be detected by template matching can be re-detected to obtain accurate detection results. Using template matching to filter out normal units and only classifying suspicious units can reduce the amount of calculation, reduce the calculation complexity, and improve the defect detection speed.

[0014] On the other hand, an electronic device is provided, wherein the electronic device comprises: a processor; and a memory arranged to store computer executable instructions, wherein the executable instructions, when executed, cause the processor to perform the defect detection method as described in any of the above embodiments.

[0015] On the other hand, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores one or more programs, and when the one or more programs are executed by a processor, the defect detection method as described in any of the above embodiments is implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the present disclosure, the following briefly introduces the drawings required to be used in some embodiments of the present disclosure. Obviously, the drawings described below are only drawings of some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can also be obtained based on these drawings. In addition, the drawings described below can be regarded as schematic diagrams, and are not limitations on the actual size of the product involved in the embodiments of the present disclosure, the actual process of the method, the actual timing of the signal, etc.

[0017] Figure 1 is a flow chart of a defect detection method according to some embodiments;

[0018] Figure 2 is a schematic diagram of an application of a defect detection method according to some embodiments;

[0019] Figure 3 is a flowchart of an adaptive acquisition unit template according to some embodiments;

[0020] Figure 4 is a schematic diagram of another application of a defect detection method according to some embodiments;

[0021] Figure 5 is a schematic diagram of an application of yet another defect detection method according to some embodiments;

[0022] Figure 6 is a schematic diagram of an application of yet another defect detection method according to some embodiments;

[0023] Figure 7 is a structural diagram of a defect detection device according to some embodiments. DETAILED DESCRIPTION

[0024] The following will be combined with the accompanying drawings to clearly and completely describe the technical solutions in some embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments provided by the present disclosure, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of the present disclosure.

[0025] Unless the context requires otherwise, throughout the specification and claims, the term "comprise" and other forms thereof, such as the third person singular form "comprises" and the present participle form "comprising", are to be interpreted as open, inclusive, that is, "including, but not limited to". In the description of the specification, the terms "one embodiment", "some embodiments", "exemplary embodiments", "example", "specific example" or "some examples" and the like are intended to indicate that specific features, structures, materials or characteristics associated with the embodiment or example are included in at least one embodiment or example of the present disclosure. The schematic representation of the above terms does not necessarily refer to the same embodiment or example. In addition, the specific features, structures, materials or characteristics may be included in any one or more embodiments or examples in any appropriate manner.

[0026] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present disclosure, unless otherwise specified, "plurality" means two or more.

[0027] When describing some embodiments, the expressions "coupled" and "connected" and their derivatives may be used. For example, when describing some embodiments, the term "connected" may be used to indicate that two or more components are in direct physical or electrical contact with each other. For another example, when describing some embodiments, the term "coupled" may be used to indicate that two or more components are in direct physical or electrical contact. However, the term "coupled" or "communicatively coupled" may also refer to two or more components that are not in direct contact with each other, but still cooperate or interact with each other. The embodiments disclosed herein are not necessarily limited to the contents of this document.

[0028] “At least one of A, B, and C” has the same meaning as “at least one of A, B, or C” and both include the following combinations of A, B, and C: A only, B only, C only, the combination of A and B, the combination of A and C, the combination of B and C, and the combination of A, B, and C.

[0029] “A and / or B” includes the following three combinations: A only, B only, and a combination of A and B.

[0030] As used herein, the term "if" is optionally interpreted to mean "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined that" or "if [a stated condition or event] is detected" are optionally interpreted to mean "upon determining that" or "in response to determining that" or "upon detecting [a stated condition or event]" or "in response to detecting [a stated condition or event]," depending on the context.

[0031] The use of "adapted to" or "configured to" herein is meant to be open and inclusive language that does not exclude devices adapted or configured to perform additional tasks or steps.

[0032] Additionally, the use of “based on” is meant to be open and inclusive, as a process, step, calculation, or other action “based on” one or more stated conditions or values ​​may, in practice, be based on additional conditions or values ​​beyond those stated.

[0033] As used herein, "about," "substantially," or "approximately" includes the stated value and an average value that is within an acceptable range of variation from the particular value as determined by one of ordinary skill in the art taking into account the measurements in question and the errors associated with the measurement of the particular quantity (i.e., the limitations of the measurement system).

[0034] Throughout the specification, reference to "one embodiment" means that the specific features, structures, or characteristics described in connection with the embodiment are included in at least one embodiment. Therefore, the phrase "in one embodiment" appearing in various places throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0035] The background of industrial defect detection is relatively fixed, with higher requirements for accuracy and less sample size. Generally, there are two defect detection methods. One defect detection method is to perform defect detection through template matching. However, when using template matching for defect detection, if the unit to be tested in the image frame to be detected rotates or changes in size, this method cannot identify the unit to be tested that has rotated or changed in size. Another defect detection method is to perform defect detection through machine learning and computer vision technology. Although this method can identify the unit to be tested that has rotated or changed in size in the image frame to be detected, when the resolution of the image frame to be detected is high, the calculation amount and calculation time of defect detection using this method are large, resulting in a slow detection speed, which is difficult to match the speed requirements of industrial production lines. If the resolution of the image frame to be detected is reduced, the accuracy of the defect detection results will be affected and the detection requirements cannot be met.

[0036] To this end, some embodiments of the present disclosure provide a defect detection method, which uses template matching to screen the units to be tested in the image frame to be tested, obtains suspicious units, and classifies the suspicious units using a machine learning algorithm to obtain the defect detection type of the suspicious units. In this way, normal units are filtered out by template matching, and only suspicious units are classified using a machine learning algorithm, which can not only reduce the amount of calculation, reduce the calculation complexity, and improve the defect detection speed. In addition, units to be tested that have rotated or changed in size can be detected, thereby improving the accuracy of defect detection.

[0037] In some embodiments, the defect detection method of the present disclosure is applicable to scenarios where the product has a single repeating unit, the products are neatly arranged, and the resolution of the image frame to be detected is high, such as display panels and products on an assembly line in industrial production. The following embodiments are illustrated using industrial beverage bottles as an example, and the present disclosure does not limit the application scenarios of the defect detection method.

[0038] Some embodiments of the present disclosure provide a defect detection method, such as Figure 1 As shown, the defect detection method includes steps 101 to 103.

[0039] Step 101: Obtain information of a plurality of units to be detected in an image frame to be detected and at least one unit template.

[0040] The unit to be tested in the image frame to be tested refers to the smallest repeated object to be tested in the image frame to be tested, such as Figure 2 As shown, taking the image frame to be detected as an image of a circuit diagram including multiple pixel circuits as an example, the unit to be tested is the image of the circuit diagram of each pixel circuit. The unit template is an image of a standard normal unit, and the unit template can be used as a template for template matching.

[0041] The information of the multiple units to be tested includes at least one of the position information and the offset of at least one unit to be tested. The position information of the unit to be tested can be calibrated using a bounding box. The embodiment of the present disclosure does not limit the shape of the bounding box, which is related to the type of the unit to be tested. For example, the bounding box can be circular, rectangular, elliptical, diamond, polygonal, etc. The following embodiment takes the bounding box as a rectangular bounding box as an example for illustrative description.

[0042] In some embodiments, when the information of the multiple units to be tested includes the position information of each unit to be tested, the position information of each unit to be tested can be manually calibrated. For example, the position of each unit to be tested in the image frame to be detected is manually calibrated using a rectangular bounding box.

[0043] In some embodiments, when the information of multiple units to be tested includes the position information and offset of at least one unit to be tested, the position information of the unit to be tested can be calculated from the position information of at least one unit to be tested and the offset of the adjacent unit in the image frame to be tested. For example, the positions of other units to be tested in the image frame to be tested can be determined based on the position and offset of one unit to be tested in the image frame to be tested that are manually calibrated. The position information and offset of the at least one unit to be tested can be input in advance by the user.

[0044] In some embodiments, a fixed method or an adaptive method may be used to obtain at least one unit template. The following describes in detail the implementation of obtaining at least one unit template using a fixed method and the implementation of obtaining at least one unit template using an adaptive method.

[0045] Exemplarily, the method of obtaining at least one unit template by a fixed method includes: randomly selecting an image frame to be detected, matching each unit to be detected in the image frame to be detected with a normal unit in turn, and selecting the unit to be detected with the highest degree of matching with the normal unit as the unit template. The unit template can be used as a unit template for template matching in subsequent defect detection. The method of obtaining at least one unit template by a fixed method can also include: matching multiple normal units in a normal image frame without defects with each other, and selecting the unit with the highest degree of matching as the unit template.

[0046] In some embodiments, when the adaptive method is used to obtain at least one unit template, a unit template can be selected from each image frame to be detected. For different image frames to be detected, the following steps 301 to 304 can be used to obtain the unit template corresponding to each image frame to be detected. Figure 3 As shown, the method of acquiring at least one unit template using an adaptive method may include steps 301 to 304.

[0047] Step 301: randomly select M first units to be tested from a plurality of units to be tested in an image frame to be detected.

[0048] In some embodiments, M is an integer greater than or equal to 2. In order to obtain the most standard unit template and improve the quality of defect detection, M can be set to a larger value. When M is equal to the number of units to be tested in the image frame to be detected, it is called an exhaustive method. In order to ensure the quality of detection and reduce the amount of calculation, M units to be tested are randomly selected from the units to be tested as the first unit to be tested. The present disclosure does not limit the value of M.

[0049] For example, Figure 2 The image frame to be detected shown in FIG. 1 includes 16 units to be tested. Taking M as 6 as an example, 6 units to be tested can be randomly selected from the 16 units to be tested as the first units to be tested. For example, the first units to be tested can include Figure 2 The units under test M1 to M6 are shown.

[0050] Step 302: Obtain a unit group under test corresponding to each first unit under test.

[0051] The unit group to be tested corresponding to each first unit to be tested includes N second units to be tested. In some embodiments, obtaining the unit group to be tested corresponding to each first unit to be tested may include: randomly selecting N units to be tested from other units to be tested in the image frame to be detected except the first unit to be tested as the unit group to be tested corresponding to the first unit to be tested, and the N units to be tested may also be referred to as N second units to be tested.

[0052] In some embodiments, the values ​​of M and N may be the same or different, and the present disclosure does not limit the values ​​of M and N. M and N may be preset by a user and input into the defect detection system.

[0053] For example, taking N=7 as an example, Figure 4 As shown, when the first unit under test is Figure 4 When the first unit to be tested M1 is detected, 7 units to be tested (second units to be tested) can be randomly selected from the other 15 units to be tested except the first unit to be tested M1 in the image frame to be detected as the unit group to be tested corresponding to the first unit to be tested M1. The 7 second units to be tested can be Figure 4The second unit under test N101 to the second unit under test N107 are shown. That is, the unit under test group corresponding to the first unit under test M1 includes the second unit under test N101 to the second unit under test N107. It can be understood that the method for obtaining the unit under test group corresponding to the first unit under test M2 to the first unit under test M6 is similar to the method for obtaining the unit under test pair group corresponding to the first unit under test M1, and will not be repeated here. Similarly, the unit under test group corresponding to each unit under test in the first unit under test M2 to the first unit under test M6 can also be obtained.

[0054] It should be noted that the same unit to be tested in the image frame to be tested can be the first unit to be tested or the second unit to be tested. Figure 2 As shown, the unit to be tested M2 in the image frame to be tested can be used as the first unit to be tested. Figure 2 ,like Figure 4 As shown, when the first unit under test is Figure 2 When the first unit under test M1 in the test is detected, the unit under test group corresponding to the first unit under test M1 includes 7 second units under test, which are Figure 4 The second unit under test N101 to the second unit under test N107 in the embodiment. Figure 2 The first unit under test M2 in Figure 4 The second unit under test N102 in is the same unit under test. Figure 2 The first unit under test M3 and Figure 4 The second unit under test N104 in the example is the same unit under test. That is, the same unit under test can be selected as the first unit under test, and can also be selected as the second unit under test.

[0055] In some embodiments, the N second units under test included in the unit under test groups corresponding to different first units under test may be at least partially identical or completely different. For example, the unit under test groups corresponding to different first units under test may include the same second unit under test.

[0056] Step 303: respectively calculate the similarity between each first unit under test and the N second units under test corresponding to the first unit under test, and obtain N similarities corresponding to each first unit under test.

[0057] Exemplarily, taking the calculation of the similarity of the first unit under test M1 as an example, the first unit under test M1 is matched with the second unit under test N101 to the second unit under test N107 included in the corresponding unit under test group, respectively, to obtain 7 similarity values ​​corresponding to the first unit under test M1. Similarly, the 7 similarity values ​​corresponding to each unit under test in the first unit under test M2 to the first unit under test M6 can also be calculated. That is, the 7 similarity values ​​corresponding to each unit under test in the first unit under test M1 to the first unit under test M6 can be obtained.

[0058] Step 304: Determine the first unit to be tested with the highest average similarity value among the M first units to be tested as the unit template.

[0059] In some embodiments, each of the M first units to be tested corresponds to N similarity values. Step 304 may include: calculating the average value of the N similarity values ​​corresponding to each first unit to be tested, obtaining the average similarity value corresponding to each first unit to be tested, and determining the first unit to be tested with the highest average similarity value among the average similarity values ​​corresponding to the M first units to be tested as the unit template.

[0060] Exemplary, combined Figure 2 ,like Figure 4 As shown, first, the average value of the 7 similarity values ​​corresponding to the first unit under test M1 is calculated to obtain the average similarity value corresponding to the first unit under test M1. The average value of the 7 similarity values ​​corresponding to the first unit under test M2 is calculated to obtain the average similarity value corresponding to the first unit under test M2. By analogy, the average similarity values ​​corresponding to the first unit under test M3 to the first unit under test M6 can be obtained. Then, among the average similarity values ​​corresponding to the first unit under test M1 to the first unit under test M6, the first unit under test corresponding to the average similarity value with the highest value is determined as the unit template.

[0061] It can be understood that the embodiment of the present disclosure randomly selects a first unit to be tested and a unit group to be tested corresponding to the first unit to be tested in the image frame to be tested, matches the first unit to be tested and each second unit to be tested in the corresponding unit group to be tested to obtain multiple similarities, and calculates the average similarity, and determines the first unit to be tested with the highest average similarity among the first units to be tested as the unit template. The most suitable unit can be selected as the unit template in the image frame to be tested by the adaptive method, which solves the problem of inconsistent light illumination received by the product to be tested due to various factors in daily life, resulting in uneven brightness of the unit to be tested image, thereby affecting the quality of the selected template.

[0062] For example, when the brightness of the image frame to be detected is high, if the unit to be detected with lower brightness is used as the unit template to match the unit to be detected in the image frame to be detected with higher brightness, due to the difference in brightness, the normal unit with higher brightness in the image frame to be detected may have a lower matching degree with the unit template with lower brightness, and the normal unit with higher brightness is considered to be a suspicious unit, which affects the accuracy of defect detection. When the unit template is determined in the image frame to be detected by the above-mentioned adaptive method, the unit template can be determined in combination with factors such as brightness. For example, when the brightness of most of the units to be tested in the image frame to be detected is high, the brightness of the unit template determined by the above-mentioned adaptive method is high. For another example, when the brightness of most of the units to be tested in the image frame to be detected is low, the brightness of the unit template determined by the above-mentioned adaptive method is low.

[0063] In some embodiments, before step 101 , the process may further include acquiring an image frame to be detected and performing image preprocessing on the image frame to be detected.

[0064] Step 102: Based on the information of the multiple units to be tested, the multiple units to be tested are respectively matched with unit templates to obtain suspicious units.

[0065] The above-mentioned suspicious unit is a unit to be tested whose matching degree with the unit template is lower than a preset threshold value among multiple units to be tested. It can be understood that by performing template matching between multiple units to be tested and the unit template, normal units in the image frame to be detected can be filtered to obtain units to be tested that cannot be determined as normal units in template matching. These units to be tested that cannot be determined as normal units can be called suspicious units. Since there is a probability that a suspicious unit is a normal unit, that is, a suspicious unit can include a normal unit. In other words, the suspicious unit obtained by template matching may be a normal unit or a defective unit.

[0066] In some embodiments, the position information of multiple units to be tested can be calibrated using a rectangular bounding box, and each unit to be tested calibrated by a rectangular bounding box is matched with a unit template to obtain a matching degree (also called similarity) between each unit to be tested and the unit template. A high matching degree indicates that the unit to be tested has a high similarity with the unit template, and the probability value of the unit to be tested being a normal unit is large. A low matching degree indicates that the unit to be tested has a low similarity with the unit template, and the probability value of the unit to be tested being a normal unit is small. When the matching degree between the unit to be tested and the unit template is higher than or equal to a preset threshold, the unit to be tested is determined to be a normal unit. When the matching degree between the unit to be tested and the unit template is lower than a preset threshold, the unit to be tested is determined to be a suspicious unit. The above-mentioned suspicious unit can further determine whether the suspicious unit is a defective unit through step 103.

[0067] Exemplarily, the above-mentioned preset threshold can be manually input into the defect detection system in advance. Taking the preset threshold of 90% as an example, when the matching degree between the unit to be tested and the template is greater than or equal to 90%, the unit to be tested is determined to be a normal unit. When the matching degree between the unit to be tested and the template is lower than 90%, the unit to be tested is determined to be a suspicious unit. The larger the preset threshold, the more accurate the template matching screening result, and the higher the defect detection accuracy. The present disclosure does not limit the size of the preset threshold.

[0068] Step 103: Classify the suspicious cells using a machine learning algorithm to obtain the defect detection type of the suspicious cells.

[0069] In some embodiments, the defect detection type includes a normal type and a defect type. Since a suspicious unit refers to a unit to be tested that has a low degree of matching with the unit template in template matching, and the inherent limitation of template matching is that it can only match the image of the unit to be tested within a fixed shape area, and cannot detect units that are rotated or have changed in size, a machine learning algorithm can be used to further classify the suspicious units to obtain the defect detection type of the suspicious units.

[0070] In some embodiments, the defect type can be pre-entered by the user. In several image frames to be detected, it is manually determined whether there are defective products. If there are defective products, they are manually marked. The specific steps of marking are: use a rectangular bounding box to circle the defective unit to be tested, and use numbers to number different defect types in sequence. The marked defective product image and the defect type corresponding to the number are input into the defect detection system. At least one defective product is marked for each defect type as a sample for classification.

[0071] For example, when the image frame to be inspected is an image of a circuit diagram including a plurality of pixel circuits, the defect types may include: rotation, size change, foreign matter, leakage, etc. Figure 5 As shown, a machine learning algorithm is used to classify suspicious units in the image frame to be detected, and the image frame to be detected includes four defect types. The correspondence between the four defect types and the digital numbers is: 1 represents rotation, 2 represents size change, 3 represents foreign matter, and 4 represents leakage.

[0072] In some embodiments, the machine learning algorithm includes: a convolutional neural network algorithm or a traditional machine learning algorithm. The model of the convolutional neural network algorithm includes Densenet, ResNet, Shufflenet or MobileNet. The traditional machine learning algorithm includes a support vector machine SVM algorithm or a decision tree algorithm. Select a suitable convolutional neural network algorithm model according to the complexity of the defect. If the defect complexity is high, select Densenet or ResNet. If the defect complexity is low, select Shufflenet or MobileNet. When using a traditional machine learning decision tree, combine LBP, Haar and other features.

[0073] Figure 6 The application diagram of the defect detection method provided for some embodiments of the present disclosure is as follows: Figure 6As shown, taking the machine learning algorithm as a convolutional neural network as an example, when performing defect detection on the image frame to be detected, first, the input image frame to be detected is divided into units to obtain multiple units to be tested in the image frame to be detected. Then, an adaptive method or a fixed method is used to obtain a unit template, and each unit to be tested in the image frame to be detected is matched with the unit template in turn to obtain the matching degree of each unit to be tested and the unit template. Then, according to the comparison between the matching degree of the unit to be tested and the unit template and the preset threshold, the unit to be tested is divided into normal units and suspicious units. Then, the normal units obtained by template matching are filtered out to obtain suspicious units. Finally, the convolutional neural network is used to classify the suspicious units to obtain the defect detection type of the suspicious units.

[0074] In some embodiments, the above step 103 may further include: determining whether the suspicious unit is a normal unit or a defective unit according to the defect detection type of the suspicious unit; when the suspicious unit is a defective unit, generating an alarm message and outputting the defect detection type of the suspicious unit. It is understandable that if the defect detection type of the suspicious unit is any defect type, the suspicious unit is determined to be a defective unit, an alarm message may be generated and the defect type of the suspicious unit may be output, so that relevant personnel can promptly discover defective products and handle them.

[0075] The defect detection method provided by the embodiment of the present disclosure uses template matching to screen the units to be tested in the image frame to be tested, obtains suspicious units, and classifies the suspicious units using a machine learning algorithm to obtain the defect detection type of the suspicious units. In this way, the normal units in the image frame to be tested can be filtered out by template matching, and only the suspicious units in the image frame to be tested are classified using a machine learning algorithm, which can not only reduce the amount of calculation, reduce the calculation complexity, and improve the defect detection speed. In addition, the units to be tested that have rotated or changed in size can be detected, thereby improving the accuracy of defect detection.

[0076] Some embodiments of the present disclosure provide a defect detection device, such as Figure 7 As shown, the defect detection device 700 includes a processor 701 and a transceiver 702 .

[0077] The transceiver 702 is configured to obtain information of a plurality of units to be detected in the image frame to be detected and at least one unit template.

[0078] The information of the multiple units to be tested includes at least one of position information and offset of at least one unit to be tested.

[0079] Processor 701 is configured to: perform template matching on the multiple units to be tested with the unit templates respectively according to the information of the multiple units to be tested to obtain suspicious units, where the suspicious units are the units to be tested whose matching degree with the unit template is lower than a preset threshold among the multiple units to be tested; and classify the suspicious units using a machine learning algorithm to obtain the defect detection type of the suspicious units.

[0080] In some embodiments, the machine learning algorithm includes: a convolutional neural network algorithm or a traditional machine learning algorithm, the model of the convolutional neural network algorithm includes Densenet, ResNet, Shufflenet or MobileNet, and the traditional machine learning algorithm includes a support vector machine SVM algorithm or a decision tree algorithm.

[0081] The processor 701 is configured to: randomly select M first units to be tested and a unit group to be tested from multiple units to be tested in the image frame to be detected, each unit group to be tested includes N second units to be tested, and each first unit to be tested corresponds to one unit group to be tested; respectively calculate the similarity between each first unit to be tested and the N second units to be tested corresponding to the first unit to be tested, and obtain the N similarities corresponding to each first unit to be tested; calculate the average similarity corresponding to the first unit to be tested according to the N similarities corresponding to each first unit to be tested; and determine the first unit to be tested with the highest average similarity among the M first units to be tested as the unit template.

[0082] The processor 701 is further configured to: determine whether the suspicious cell is a normal cell or a defective cell according to the defect detection type of the suspicious cell; and when the suspicious cell is a defective cell, generate alarm information and output the defect detection type of the suspicious cell.

[0083] The transceiver 702 may be a communication interface, such as an input / output interface. For example, when the defect detection device 700 is a chip, the transceiver 702 is a communication interface. The transceiver 702 may also be a radio frequency unit. For example, when the defect detection device 700 is a terminal device, the transceiver 702 may be a radio frequency unit connected to an antenna.

[0084] In some embodiments, the defect detection device 700 may further include a memory 703, which is used to store program codes and data corresponding to the defect detection device 700 executing any of the defect detection methods provided above. The memory 703 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM), etc.

[0085] Some embodiments of the present disclosure provide a computer-readable storage medium (e.g., a non-transitory computer-readable storage medium), which stores computer program instructions. When the computer program instructions are executed on a computer (e.g., a defect detection device), the computer executes the defect detection method described in any of the above embodiments.

[0086] Exemplarily, the above-mentioned computer-readable storage media may include, but are not limited to: magnetic storage devices (e.g., hard disks, floppy disks or magnetic tapes, etc.), optical disks (e.g., CDs (Compact Disks), DVDs (Digital Versatile Disks), etc.), smart cards and flash memory devices (e.g., EPROMs (Erasable Programmable Read-Only Memory), cards, sticks or key drives, etc.). The various computer-readable storage media described in the present disclosure may represent one or more devices and / or other machine-readable storage media for storing information. The term "machine-readable storage medium" may include, but is not limited to, wireless channels and various other media capable of storing, containing and / or carrying instructions and / or data.

[0087] Some embodiments of the present disclosure also provide a computer program product, for example, the computer program product is stored on a non-transitory computer-readable storage medium. The computer program product includes computer program instructions, and when the computer program instructions are executed on a computer (for example, a defect detection device), the computer program instructions cause the computer to perform the defect detection method described in the above embodiments.

[0088] Some embodiments of the present disclosure further provide a computer program. When the computer program is executed on a computer, the computer program enables the computer to execute the defect detection method described in the above embodiments.

[0089] The beneficial effects of the above-mentioned computer-readable storage medium, computer program product and computer program are the same as the beneficial effects of the defect detection method described in some of the above-mentioned embodiments, and will not be repeated here.

[0090] The above is only a specific embodiment of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any changes or substitutions that can be thought of by any person skilled in the art within the technical scope disclosed in the present disclosure should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims

1. A defect detection method, characterized in that: The method comprises: Acquire information of a plurality of units to be detected in the image frame to be detected and at least one unit template; According to the information of the multiple units to be tested, the multiple units to be tested are respectively template matched with the unit template to obtain suspicious units, wherein the suspicious units are units to be tested whose matching degree with the unit template is lower than a preset threshold among the multiple units to be tested; Using a machine learning algorithm to classify the suspicious unit to obtain a defect detection type of the suspicious unit; Acquiring the at least one unit template includes: Randomly selecting M first units to be tested and a unit group to be tested from a plurality of units to be tested in the image frame to be detected, each of the unit groups to be tested includes N second units to be tested, and each of the first units to be tested corresponds to one of the unit groups to be tested; Respectively calculating the similarity between each of the first units under test and the N second units under test corresponding to the first unit under test, to obtain N similarities corresponding to each of the first units under test; Calculate the average similarity value corresponding to the first unit to be tested according to the N similarities corresponding to each of the first unit to be tested; The first unit to be tested having the highest average similarity value among the M first units to be tested is determined as the unit template.

2. The method according to claim 1, characterized in that The information of the plurality of units under test includes at least one of position information and offset of at least one of the units under test.

3. The method according to claim 1, characterized in that The method further comprises: Determining whether the suspicious cell is a normal cell or a defective cell according to the defect detection type of the suspicious cell; When the suspicious cell is a defective cell, an alarm message is generated and a defect detection type of the suspicious cell is output.

4. A defect detection device, characterized in that: include: processors and transceivers; The transceiver is configured to obtain information of a plurality of units to be detected in the image frame to be detected and at least one unit template; The processor is configured to: According to the information of the multiple units to be tested, the multiple units to be tested are respectively template matched with the unit template to obtain suspicious units, wherein the suspicious units are units to be tested whose matching degree with the unit template is lower than a preset threshold among the multiple units to be tested; Using a machine learning algorithm to classify the suspicious unit to obtain a defect detection type of the suspicious unit; Acquiring the at least one unit template includes: Randomly selecting M first units to be tested and a unit group to be tested from a plurality of units to be tested in the image frame to be detected, each of the unit groups to be tested includes N second units to be tested, and each of the first units to be tested corresponds to one of the unit groups to be tested; Respectively calculating the similarity between each of the first units under test and the N second units under test corresponding to the first unit under test, to obtain N similarities corresponding to each of the first units under test; Calculate the average similarity value corresponding to the first unit to be tested according to the N similarities corresponding to each of the first unit to be tested; The first unit to be tested having the highest average similarity value among the M first units to be tested is determined as the unit template.

5. The device according to claim 4, characterized in that The processor is configured to: Randomly selecting M first units to be tested and a unit group to be tested from a plurality of units to be tested in the image frame to be detected, each of the unit groups to be tested includes N second units to be tested, and each of the first units to be tested corresponds to one of the unit groups to be tested; Respectively calculating the similarity between each of the first units under test and the N second units under test corresponding to the first unit under test, to obtain N similarities corresponding to each of the first units under test; Calculate the average similarity value corresponding to the first unit to be tested according to the N similarities corresponding to each of the first unit to be tested; The first unit to be tested having the highest average similarity value among the M first units to be tested is determined as the unit template.

6. The device according to claim 4, characterized in that The information of the plurality of units under test includes at least one of position information and offset of at least one of the units under test.

7. The device according to claim 4, characterized in that The processor is further configured to: Determining whether the suspicious cell is a normal cell or a defective cell according to the defect detection type of the suspicious cell; When the suspicious cell is a defective cell, an alarm message is generated and a defect detection type of the suspicious cell is output.

8. An electronic device, characterized in that: The electronic device comprises: a processor; and a memory arranged to store computer executable instructions, wherein the executable instructions, when executed, cause the processor to perform the defect detection method according to any one of claims 1 to 3.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores one or more programs, and when the one or more programs are executed by a processor, the defect detection method according to any one of claims 1 to 3 is implemented.

Images