Defect detection method, device, equipment and system based on artificial intelligence, medium and product

Abstract

The invention discloses a defect detection method, device, equipment and system based on artificial intelligence, a medium and a product. The method comprises the steps of extracting a first image feature of a to-be-detected image through a feature extraction module of a preset detection model, the to-be-detected image being an image containing a to-be-detected piece; and on the basis of a preset feature library, feature matching is performed on the first image features to determine a detection result, the detection result comprises normality or abnormity, the feature library at least comprises features of each component of the normal device extracted by the feature extraction module, and the normal device is the same as the to-be-detected device in category. Through a method of combining representation learning and an anomaly detection algorithm, image features of a to-be-detected image of a to-be-detected piece are extracted through a feature extraction module, and then feature matching is performed based on a feature library, so that anomaly detection is realized. The model training difficulty is low, the extracted feature scales are basically the same, the defect detection accuracy can be improved, and the problem that the negative samples have many types and are difficult to detect accurately can be solved.

Description

Technical Field

[0001] The present application belongs to the technical field of defect detection of printed circuit boards, and in particular, relates to an artificial intelligence-based defect detection method, a defect detection device, an electronic device, a defect detection system, a computer-readable storage medium, and a computer program product. Background Art

[0002] Currently, PCBA (Printed Circuit Board Assembly) refers to the process of soldering electronic components on a printed circuit board (PCB). PCBA is an important part of electronic product manufacturing, involving multiple steps and technologies.

[0003] However, in the process of PCBA, various defects are prone to occur, such as cold soldering, leaking soldering, component offset, etc. Therefore, how to accurately identify PCB defects is an important means to ensure the quality of the produced PCBs. Summary of the invention

[0004] The present application aims to solve at least one of the technical problems existing in the prior art. To this end, the present application proposes an artificial intelligence-based defect detection method, defect detection device, electronic device, defect detection system, computer-readable storage medium and computer program product, which can improve the accuracy of PCB defect detection and ensure the qualified rate of produced PCBs.

[0005] In a first aspect, the present application provides a defect detection method, comprising:

[0006] Extracting a first image feature of an image to be tested by a feature extraction module of a preset detection model, wherein the image to be tested is an image including the object to be tested;

[0007] Based on a preset feature library, feature matching is performed on the first image feature to determine a detection result, wherein the detection result includes normal or abnormal. The feature library at least includes the features of each component of the normal device extracted by the feature extraction module, and the normal device is of the same category as the device to be tested.

[0008] In a second aspect, the present application provides a defect detection device, the defect detection device comprising:

[0009] An extraction module, used to extract a first image feature of an image to be tested by using a feature extraction module of a preset detection model, wherein the image to be tested is an image including the object to be tested;

[0010] A matching module is used to perform feature matching on the first image feature based on a preset feature library to determine a detection result, wherein the detection result includes normal or abnormal, and the feature library at least includes the features of each component of the normal device extracted by the feature extraction module, and the normal device is of the same category as the device to be tested.

[0011] In a third aspect, the present application provides a non-transitory computer-readable storage medium having a computer program stored thereon, which implements the above-mentioned defect detection method when executed by a processor.

[0012] In a fourth aspect, the present application provides an electronic device, which is used to execute the defect detection method of any of the above-mentioned embodiments.

[0013] In a fifth aspect, the present application provides a defect detection system, which further includes an image acquisition device and the electronic device of the above-mentioned embodiment, wherein the image acquisition device is used to acquire the image to be detected.

[0014] In a sixth aspect, the present application provides a computer program product, including a computer program, which implements the above-mentioned defect detection method when executed by a processor.

[0015] The defect detection method, defect detection device, electronic device, defect detection system, computer-readable storage medium and computer program product provided in the embodiments of the present application use a method that combines representation learning and anomaly detection algorithms, and uses a feature extraction module to perform image features of the image of the test piece to be tested (corresponding to representation learning), and then performs feature matching based on a feature library to achieve anomaly detection. The feature library only needs to include features of normal devices to achieve feature matching. Therefore, the feature extraction module only needs to be trained based on positive samples (i.e., images of normal devices that do not contain defects) to obtain a feature extraction module that can accurately extract image features of normal samples. Compared with a representation learning model that requires both positive samples and a small number of negative samples that are difficult to obtain for training, the training sample requirements are lower, and the feature extraction module only needs to be able to accurately extract image features of normal devices. Even if overfitting occurs, it will not affect the feature extraction accuracy of the feature extraction module, and the scale of the feature extracted is relatively consistent.

[0016] When feature matching is performed based on accurate and scale-consistent features for anomaly detection, not only can the detection accuracy be guaranteed, but also the shortcomings of a large number of negative samples and their difficulty in obtaining can be effectively solved.

[0017] In this way, the present application combining representation learning and anomaly detection algorithms can not only avoid the impact of insufficient or insufficiently diverse training data on model training, but also reduce the false detection rate and missed detection rate when detecting a wide variety of defects, thereby improving the qualified rate of PCBA production.

[0018] Additional aspects and advantages of the embodiments of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the embodiments of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0020] Figure 1 It is a schematic diagram of an application scenario of the defect detection method provided in an embodiment of the present application;

[0021] Figure 2 It is a first flow chart of the defect detection method provided in an embodiment of the present application;

[0022] Figure 3 It is a schematic diagram of the structure of the detection model provided in the embodiment of the present application;

[0023] Figure 4 is a second flow chart of the defect detection method provided in an embodiment of the present application;

[0024] Figure 5 is a third flow chart of the defect detection method provided in the embodiment of the present application;

[0025] Figure 6 is a fourth flow chart of the defect detection method provided in an embodiment of the present application;

[0026] Figure 7 is a fifth flow chart of the defect detection method provided in the embodiment of the present application;

[0027] Figure 8 It is a structural diagram of a position attention module of a detection model provided in an embodiment of the present application;

[0028] Fig. 9 It is a structural diagram of the channel attention module of the detection model provided in the embodiment of the present application;

[0029] Fig.10 is a module schematic diagram of a defect detection device provided in an embodiment of the present application;

[0030] Fig.11 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0031] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be understood as limiting the present application.

[0032] For ease of understanding, the technical background and application scenarios of this application are first introduced below:

[0033] Defect detection solutions are generally divided into three categories:

[0034] (1) Image processing technology defect detection. Defect detection relies on predefined rules and fixed features (such as color, shape, and texture). This makes them very sensitive to changes in lighting and environment, and lacks the robustness to deal with complex and diverse defects. They perform poorly when processing high-dimensional and complex data, and usually rely on simple feature extraction methods that cannot fully utilize the information in high-dimensional data, which leads to low detection accuracy. The lack of adaptive capabilities makes it difficult to deal with new or unseen defect types, which means that whenever the production process or product design changes, the detection system needs to be redesigned and adjusted, which increases maintenance costs and time.

[0035] (2) Deep learning algorithm defect detection. Representation learning and metric learning usually require a large amount of labeled data for training, including negative sample data. However, in the field of industrial production, negative samples are of various types and difficult to obtain. If the training data is insufficient or not diverse enough, the deep learning model is prone to overfitting, resulting in poor generalization ability on new data.

[0036] (3) Detection of defects in normal sample learning algorithms (anomaly detection algorithms): Although this method can effectively solve the shortcomings of a wide variety of negative samples and their difficulty in obtaining, the current mainstream algorithms all detect from the perspective of outlier detection without considering the representation of image features, which will result in different anomaly detection results for the same image in different representation methods. Therefore, for anomaly detection algorithms, appropriate image features are very important.

[0037] See also Figure 1 , Figure 1 1 is an application scenario diagram of a defect detection method provided in an embodiment of the present application. The application scenario provided in the present application includes a terminal device 101 and a server 102, and the defect detection method provided in the present application can be executed by at least one of the terminal device 101 and the server 102.

[0038] Among them, the terminal devices may include but are not limited to: smart phones (such as Android phones, IOS phones, etc.), tablet computers, laptops, desktop computers, smart speakers, smart watches, portable personal computers, mobile Internet devices (Mobile Internet Devices, referred to as MID), intelligent voice interaction devices, smart home appliances, vehicle-mounted terminals, aircraft, wearable devices, etc., and the embodiments of the present application do not limit this.

[0039] The server can be an independent physical server or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms, but the embodiments of the present application do not limit this.

[0040] The terminal device 101 and the server 102 can obtain the image of the test piece 200 to perform defect detection on the test image, thereby obtaining a detection result.

[0041] In some embodiments, the application scenario further includes an image acquisition device 103, which is used to acquire an image of the test piece 200 to be tested and transmit it to at least one of the terminal device 101 and the server 102 for defect detection.

[0042] It should be noted that Figure 1 The number of terminal devices and servers is only used as an example, and the number of terminal devices and servers can be more or less, which is not limited here. The terminal devices and servers can be directly or indirectly connected via wired or wireless communication, which is not limited in this application.

[0043] The defect detection method involved in this application can be implemented based on cloud technology.

[0044] Among them, cloud technology refers to a hosting technology that unifies hardware, software, network and other resources within a wide area network or local area network to achieve data calculation, storage, processing and sharing.

[0045] Cloud technology is a general term for network technology, information technology, integration technology, management platform technology, application technology, etc. based on the cloud computing business model. It can form a resource pool, which can be used on demand and is flexible and convenient. Cloud computing technology will become an important support. The backend services of the technical network system require a large amount of computing and storage resources, such as video websites, picture websites and more portal websites. With the rapid development and application of the Internet industry, in the future, each item may have its own identification mark, which needs to be transmitted to the backend system for logical processing. Data of different levels will be processed separately. All kinds of industry data need strong system backing support, which can only be achieved through cloud computing.

[0046] The defect detection method of the present application can be implemented based on cloud computing. Cloud computing is a computing model that distributes computing tasks on a resource pool composed of a large number of computers, so that various application systems can obtain computing power, storage space and information services as needed. The network that provides resources is called a "cloud". The resources in the "cloud" are infinitely expandable in the eyes of users, and can be obtained at any time, used on demand, expanded at any time, and paid for by use.

[0047] As a provider of basic cloud computing capabilities, a cloud computing resource pool (referred to as a cloud platform, generally referred to as an IaaS (Infrastructure as a Service) platform) will be established, and various types of virtual resources will be deployed in the resource pool for external customers to choose to use. The cloud computing resource pool mainly includes: computing devices (virtualized machines, including operating systems), storage devices, and network devices.

[0048] The defect detection method in the embodiment of the present application can be executed by an electronic device, and the electronic device can be installed in at least one of a server and a terminal device, that is, the method can be executed by a server or a terminal device alone, or by a server and a terminal device together. Therefore, the execution subject of each step will not be repeatedly described below.

[0049] Based on the introduction of the above application scenarios, the embodiment of the present application provides a defect detection method, which is described in detail below:

[0050] See also Figure 2 , an artificial intelligence-based defect detection method provided in an embodiment of the present application is implemented by steps 011 and 012, which are described in detail below.

[0051] Step 011: extracting a first image feature of an image to be tested by a feature extraction module of a preset detection model, where the image to be tested is an image including the object to be tested.

[0052] The detection model may be a pre-trained neural network model for defect detection.

[0053] Among them, the neural network model is a computational model that imitates the structure and function of the biological nervous system and is used to recognize patterns, make decisions, or solve complex input-output mapping problems. It consists of a large number of artificial neurons (nodes) that interact with each other through weighted connections to form a multi-layer network structure. The core components of the neural network model include the input layer, hidden layer, and output layer, as well as the connection weights and activation functions between them.

[0054] See also Figure 3 Optionally, the detection model 20 includes a feature extraction module 21 and a classifier 22, the feature extraction module 21 is used to extract image features, and the classifier 22 is used to perform anomaly detection, that is, the classifier 22 is used to perform feature matching on the first image feature based on the feature library to determine the detection result.

[0055] Optionally, the feature extraction module may be a Vision Transformer (ViT) model.

[0056] Among them, the VIT model is an image recognition model based on the Transformer architecture, which can extract basic image features.

[0057] Compared with traditional CNN models, ViT is ① highly scalable and can handle input images of any size without preprocessing or cropping the input like CNN. ② ViT captures global contextual information at all positions of the input image through the self-attention mechanism, allowing it to better understand the semantic structure of the entire image. ③ Highly interpretable, ViT's attention mechanism allows adaptive weighting of different positions in the input image, so that the model can be visualized to what extent it pays attention to the input during the classification process.

[0058] Optionally, during the training of the detection model, the weight parameters of the feature extraction module are frozen, such as freezing the weights of the Vision Transformer during the training phase.

[0059] Freezing the weights of the Vision Transformer has the following effects:

[0060] (1) Stability: Freezing the weights of the pre-trained model can reduce the instability during training and allow the model to focus more on learning other parts of the detection model (such as the classifier) ​​parameters.

[0061] (2) Efficiency: Reduce the number of parameters that need to be updated, speed up training, and reduce the risk of overfitting.

[0062] (3) Feature inheritance: Utilize the powerful features learned by the pre-trained VIT model to avoid training from scratch and improve the initialization performance of the model.

[0063] This design helps the model to make full use of pre-trained features under limited data conditions, while enhancing the model's adaptability to the detection of different types of defects.

[0064] Optionally, the detection model is trained based on a training set generated from pre-collected image data of various normal devices.

[0065] It can be understood that the anomaly detection of the present application is achieved by matching the extracted image features with the features of normal devices in the feature library. Therefore, as long as the feature extraction module can accurately extract the features of normal devices, that is to say, the detection model of the present application only needs to generate a training set based on the pre-collected image data of various normal devices (i.e., positive samples) to complete the training. The number of positive samples is large and easy to obtain, which reduces the requirements for the training set and the difficulty of model training is low.

[0066] Specifically, after the image acquisition device acquires the image to be tested of the part to be tested produced by PCBA, the first image feature of the image to be tested can be extracted through the feature extraction module of the detection model. Since the first image features of different images to be tested are extracted by the same feature extraction module, the scales of the extracted features are basically consistent, which is conducive to improving the accuracy of anomaly detection by subsequent classifiers.

[0067] Moreover, the feature extraction module only needs to extract features from positive samples, thereby reducing the difficulty and cost of training. Even if the training is overfitting, the extracted features will not affect the accuracy of subsequent anomaly detection.

[0068] Step 012: Based on a preset feature library, feature matching is performed on the first image feature to determine a detection result, the detection result includes normal or abnormal, the feature library at least includes the features of each component of the normal device extracted by the feature extraction module, and the normal device is of the same category as the device to be tested.

[0069] The feature library refers to a database storing the features of various normal components.

[0070] Optionally, the feature library can be generated by extracting the features of each component of the normal device by the feature extraction module. In order to ensure the accuracy of abnormality detection, the feature library needs to at least contain the features of each component of the normal device of the same category as the device under test corresponding to the image under test.

[0071] Feature matching refers to matching the features of the feature library with the features of the first image to obtain a detection result.

[0072] For example, if the features of the matched components are different from the features of the components of the normal device (eg, the number of features of the matched components is less than the number of features of the components of the normal device), it can be determined that abnormal features exist in the device under test.

[0073] Optionally, the classifier is implemented based on a Local Outlier Factor (LOF) algorithm, that is, feature matching is implemented through the Local Outlier Factor algorithm.

[0074] Among them, the local outlier factor algorithm is an unsupervised learning method for detecting local outliers in a data set. The LOF algorithm is based on the concept of density, which identifies outliers by comparing the local density deviation of a point and its neighboring points. The LOF algorithm can effectively detect data points that have significantly lower density than other points.

[0075] In the present application, the local anomaly factor algorithm can detect the local density deviation of the first image feature and the component feature of the corresponding normal device in the feature library to identify whether there is an anomaly. If the local density deviation is greater than the preset deviation, it is determined that there is an anomaly, and if the local density deviation is less than the preset deviation, it is determined that there is no anomaly.

[0076] Specifically, by matching the component features of the normal device corresponding to the device to be tested corresponding to the first image feature in the feature library with the first image feature, for the normal device to be tested, the corresponding first image feature matches all the corresponding component features in the feature library. Therefore, if the first image feature matches all the corresponding component features, it can be determined that the first image feature is normal, that is, the device to be tested is normal. If the first image feature does not match at least one of the corresponding component features, it can be determined that the first image feature is abnormal, that is, the device to be tested is abnormal.

[0077] See also Figure 4 Optionally, step 012 includes:

[0078] Step 0121: determining feature distribution information of a target device of the same category as the device under test in a preset feature library;

[0079] Step 0122: Calculate the matching value between the device under test and the target device based on the image feature to be classified and the feature distribution information, where the image feature to be classified is the first image feature or the second image feature;

[0080] Step 0123: Based on the matching value, determining the matching result between the device under test and the target device, the matching result including matching and mismatching;

[0081] Step 0124: When the device under test matches the target device, determining that the test result is normal;

[0082] Step 0125: When the device under test and the target device do not match, the test result is determined to be abnormal.

[0083] Specifically, taking feature detection based on the LOF algorithm as an example, when performing feature matching, we can first find the feature distribution information of the target device with the same category as the device under test in the feature library based on the category information of the device under test, that is, the neighborhood feature corresponding to the first image feature of the device under test.

[0084] Then, based on the image features corresponding to the DUT (such as the first image features or other image features after the first image features are processed by other features) and feature distribution information, the matching value between the DUT and the target device is calculated, such as calculating the local anomaly factor in the LOF algorithm;

[0085] It can be understood that the local anomaly factor can be one or more. When calculating the local anomaly factor, the first image feature can be taken as a whole to calculate the local anomaly factor, or a local anomaly factor can be calculated for the characteristics of each component of the target device, thereby obtaining the local anomaly factor corresponding to the image feature corresponding to each component feature in the first image feature.

[0086] Finally, based on the matching value, the matching result between the DUT and the target device can be determined, and the matching result includes match and mismatch. If any local abnormality factor is greater than 1, there is an abnormality, that is, there is an abnormality in the DUT, and when all local abnormality factors are less than 1, it is determined that there is no abnormality in the DUT, that is, the DUT is normal.

[0087] The defect detection method provided in the embodiment of the present application uses a feature extraction module to obtain image features of the image of the test piece (corresponding representation learning), and then performs feature matching based on a feature library to achieve abnormality detection. The feature library only needs to include the features of normal devices to achieve feature matching. Therefore, the feature extraction module only needs to be trained based on positive samples (i.e., images of normal devices that do not contain defects) to obtain a feature extraction module that can accurately extract image features of normal samples. Compared with the representation learning model that requires both positive samples and a small number of negative samples that are difficult to obtain for training, the training sample requirements are lower, and the feature extraction module only needs to be able to accurately extract image features of normal devices. Even if overfitting occurs, it will not affect the feature extraction accuracy of the feature extraction module, and the scale of the feature extracted is relatively consistent.

[0088] When feature matching is performed based on accurate and scale-consistent features for anomaly detection, not only can the detection accuracy be guaranteed, but also the shortcomings of a large number of negative samples and their difficulty in obtaining can be effectively solved.

[0089] In this way, the present application combining representation learning and anomaly detection algorithms can not only avoid the impact of insufficient or insufficiently diverse training data on model training, but also reduce the false detection rate and missed detection rate when detecting a wide variety of defects, thereby improving the qualified rate of PCBA production.

[0090] See also Figure 3 and 5 In some embodiments, the detection model 20 further includes an attention module 23, and the defect detection method further includes:

[0091] Step 013: reshape the first image feature through the attention module to generate a second image feature;

[0092] Step 012: Based on a preset feature library, feature matching is performed on the first image feature to determine a detection result, including:

[0093] Step 0126: Based on a preset feature library, feature matching is performed on the second image features to determine the detection result.

[0094] Specifically, for the extracted first image features, an attention module can be set to focus on the features of the extracted image features that are highly correlated with defect inspection based on the attention mechanism, thereby improving the accuracy of defect detection.

[0095] The core idea of ​​the attention mechanism is to let the model learn to focus on the important parts of the input instead of treating all inputs equally. In this way, the model can process long sequences or high-dimensional data more efficiently and improve the accuracy of predictions.

[0096] Therefore, the first image feature is reshaped by the attention module to generate the second image feature, and then feature matching is performed based on the second image feature, thereby further improving the accuracy of defect detection.

[0097] Optionally, the image feature to be classified for feature matching is the second image feature, that is, feature matching is performed based on the second image feature reshaped by the attention module.

[0098] See also Figure 6 Optionally, the attention module 23 includes a position attention module 231 and a channel attention module 232, and step 013 includes:

[0099] Step 0131: capturing long-distance dependency information between any two positions in the first image feature through a position attention module to generate a third image feature;

[0100] Step 0132: capturing dependency information between different channels in the first image feature through a channel attention module to generate a fourth image feature;

[0101] Step 0133: Generate a second image feature based on the third image feature and the fourth image feature.

[0102] Specifically, for defect detection, position features and channel features have a greater impact on the accuracy of defect detection. Therefore, a position attention module is set to capture the long-distance dependency information between any two positions in the first image feature through the position attention module to enhance the position feature representation, thereby generating a third image feature.

[0103] The channel attention module is used to capture the dependency information between different channels in the first image feature to enhance the channel feature representation, thereby generating the fourth image feature.

[0104] Finally, by enhancing the third image feature represented by the position feature and the fourth image feature represented by the channel feature (such as superimposing the third image feature and the fourth image feature), a second image feature that simultaneously enhances the position feature representation and the channel feature representation is generated. The feature extraction accuracy is high, and the extracted features are more conducive to the accuracy of subsequent defect detection.

[0105] The following is an introduction to the specific process of feature reshaping by the position attention module and the channel attention module:

[0106] See also Figure 7 Optionally, step 0131 includes:

[0107] Step 01311: performing convolution on the first image feature to generate a fifth image feature and a sixth image feature;

[0108] Step 01312: Based on the fifth image feature and the sixth image feature, a position attention matrix is ​​calculated, where the position attention matrix is ​​used to characterize the degree of influence of each position on other positions;

[0109] Step 01313: performing convolution again on the first image feature to generate a seventh image feature;

[0110] Step 01314: Generate the third image feature based on the position attention matrix, the seventh image feature and the first image feature.

[0111] Specifically, see Figure 8 , the first image feature (such as Figure 8 After the feature A in the image is input into the position attention module, the first image feature is convolved to generate the fifth image feature (such as Figure 8 Feature B in the image) and the sixth image feature (such as Figure 8 Feature C).

[0112] Then, the fifth image feature and the sixth image feature are transposed (i.e. Figure 8 reshape&transpose) and matrix multiplication, and through activation functions (such as Figure 8 The softmax function in ) obtains the position attention matrix, which is used to characterize the influence of each position on other positions.

[0113] After that, the first image feature is convolved again to generate the seventh image feature, and the seventh image feature is matrix multiplied with the position attention matrix, the result is multiplied by a learnable scale parameter α, and then superimposed with the first image feature (such as element-by-element addition) to generate the final third image feature (such as Figure 8 In this way, features at similar locations are selectively aggregated through global context information, thereby improving the consistency of similar features.

[0114] Please continue reading Figure 7 Optionally, step 0132 includes:

[0115] Step 01321: Based on the first image feature, a channel attention matrix is ​​calculated, where the channel attention matrix is ​​used to characterize the degree of influence of each channel on other channels;

[0116] Step 01322: Generate a fourth image feature based on the channel attention matrix and the first image feature.

[0117] Specifically, in the first image feature (such as Fig. 9 After the feature A) in is input into the channel attention module, the first image feature can be directly matrix multiplied and the channel attention matrix can be obtained through the softmax function, which represents the influence of each channel on other channels.

[0118] After that, the channel attention matrix is ​​transposed (i.e. Fig. 9 The reshape & transpose in the above formula is then matrix multiplied with the first image feature, the result is multiplied by a learnable scale parameter β, and then element-wise added to the first image feature to obtain the final fourth image feature (such as Fig. 9 By emphasizing the interdependent channels in the feature map, it helps improve the feature discrimination ability of specific semantics.

[0119] In some embodiments, the detection model training process may be:

[0120] ① Collect positive sample data of components (capacitors, resistors, diodes, transistors, etc.) of various normal devices and make labels according to the classified data set.

[0121] ②Training the classification network: Freeze the weights of the vision tranformer module during the training process, and learn the representations of different components by updating the attention module (such as learnable parameters α, β, etc.), the fully connected module and the classifier parameters until the detection model converges.

[0122] ③ Build a feature library of various components, which includes component features of different types of normal devices. According to the component category, use the feature extraction module to extract the features of each type of component and store them locally or in a database.

[0123] According to the method described in the above embodiment, the present application embodiment also provides a defect detection device 300, which is used to perform the steps in the above defect detection method. Fig.10 , Fig.10 : is a schematic diagram of the structure of a defect detection device 300 provided in an embodiment of the present application. The defect detection device 300 includes:

[0124] An extraction module 301 is used to extract a first image feature of an image to be tested by using a feature extraction module of a preset detection model, where the image to be tested is an image including the object to be tested;

[0125] The matching module 302 is used to perform feature matching on the first image feature based on a preset feature library to determine the detection result, which includes normal or abnormal. The feature library at least includes the features of each component of the normal device extracted by the feature extraction module, and the normal device is of the same category as the device to be tested.

[0126] It should be noted that the specific details of each module unit in the above-mentioned defect detection device 300 have been described in detail in the embodiment of the above-mentioned defect detection method, and will not be repeated here.

[0127] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0128] In some embodiments, the defect detection device in the embodiments of the present application can be implemented in hardware, such as a chip, or a component in a chip, such as an integrated circuit or a chip; the defect detection device can also be implemented in software, such as as an application installed in a chip.

[0129] Among them, chips, also known as integrated circuits (ICs), microcircuits or microchips, are the core components of modern electronic devices. They are made of semiconductor materials (usually silicon) and are integrated with millions or even billions of microelectronic components (such as transistors, resistors, capacitors, etc.) through a series of complex manufacturing processes to form an electronic circuit with specific functions. These components and their interconnections are manufactured on the same small silicon wafer. They are small in size, light in weight, but powerful in function. They are the basis of various electronic products such as computers, smart phones, home appliances, cars, medical equipment, etc.

[0130] See also Fig.11 In some embodiments, the electronic device 400 of the embodiment of the present application includes a processor 401 and a memory 402. The memory 402 stores a computer program 403. When the computer program 403 is executed by the processor 401, the various processes of the above-mentioned defect detection method embodiment can be implemented and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0131] Please refer again Figure 1 In some embodiments, the defect detection system 1000 of the embodiment of the present application includes an image acquisition device 103 and an electronic device of the above-mentioned implementation mode (such as at least one of the terminal device 101 and the server 102), the image acquisition device 103 is used to acquire the image to be tested of the test piece, and the electronic device is used to perform abnormality detection on the image to be tested.

[0132] The image acquisition device may be an AOI (Automated Optical Inspection) device.

[0133] AOI equipment is a type of automated inspection equipment used to inspect printed circuit boards (PCBs) and other electronic components. AOI equipment uses cameras and other imaging technologies to capture images and analyzes these images through software to detect defects or substandard products.

[0134] For example, the AOI equipment can be a visible light camera, a depth camera, a spectral imaging device, etc.

[0135] An embodiment of the present application also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various processes of the above-mentioned defect detection method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0136] The processor may be the processor of the electronic device in the above embodiment. The computer readable storage medium may be a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0137] Computer readable media may include computer storage media and communication media. Computer storage media include volatile and non-volatile, removable and non-removable media implemented by any method or technology for storing information such as computer readable instructions, data structures, program modules or other data. Computer storage media include RAM, ROM, Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), flash memory or other solid-state storage technologies, CD-ROM, Digital Versatile Disc (DVD) or other optical storage, cassettes, tapes, disk storage or other magnetic storage devices. Of course, those skilled in the art will appreciate that computer storage media are not limited to the above. The above-mentioned system memory 504 and mass storage device 507 can be collectively referred to as memory.

[0138] The embodiment of the present application also provides a computer program product, including a computer program, which implements the above-mentioned defect detection method when executed by a processor. The processor may be a processor of the electronic device in the above-mentioned embodiment. When the computer program is executed by the processor, each process of the embodiment of the above-mentioned defect detection method is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0139] It is understandable that in the specific implementation of this application, data related to user identity or characteristics is involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0140] Although the embodiments of the present application have been shown and described, those skilled in the art will appreciate that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present application, and that the scope of the present application is defined by the claims and their equivalents.

Claims

1. A defect detection method based on artificial intelligence, characterized in that: include: Extracting a first image feature of an image to be tested by a feature extraction module of a preset detection model, wherein the image to be tested is an image including the object to be tested; Based on a preset feature library, feature matching is performed on the first image feature to determine a detection result, wherein the detection result includes normal or abnormal. The feature library at least includes the features of each component of the normal device extracted by the feature extraction module, and the normal device is of the same category as the device to be tested.

2. The defect detection method according to claim 1, characterized in that: The detection model is trained based on a training set generated by pre-collected image data of various normal devices.

3. The defect detection method according to claim 1, characterized in that: During the training of the detection model, the weight parameters of the feature extraction module are frozen.

4. The defect detection method according to any one of claims 1 to 3, characterized in that: The detection model also includes a classifier, which is used to perform feature matching on the first image features based on the feature library to determine the detection result.

5. The defect detection method according to claim 4, characterized in that: The classifier is implemented based on the local anomaly factor algorithm.

6. The defect detection method according to any one of claims 1 to 5, characterized in that: The detection model also includes an attention module, and the method further includes: Reshaping the first image feature through the attention module to generate a second image feature; The performing feature matching on the first image feature based on a preset feature library to determine a detection result includes: Based on a preset feature library, feature matching is performed on the second image features to determine a detection result.

7. The defect detection method according to claim 6, characterized in that: The attention module includes a position attention module and a channel attention module, and the first image feature is reshaped by the attention module to generate a second image feature, including: Capturing long-distance dependency information between any two positions in the first image feature through the position attention module to generate a third image feature; Capturing dependency information between different channels in the first image feature through the channel attention module to generate a fourth image feature; The second image feature is generated based on the third image feature and the fourth image feature.

8. The defect detection method according to claim 7, characterized in that: The capturing of long-distance dependency information between any two positions in the first image feature by the position attention module to generate a third image feature includes: Convolving the first image feature to generate a fifth image feature and a sixth image feature; Based on the fifth image feature and the sixth image feature, calculating a position attention matrix, wherein the position attention matrix is ​​used to characterize the degree of influence of each position on other positions; performing convolution again on the first image feature to generate a seventh image feature; The third image feature is generated based on the position attention matrix, the seventh image feature and the first image feature.

9. The defect detection method according to claim 7, characterized in that: The capturing, by the channel attention module, dependency information between different channels in the first image feature to generate a fourth image feature includes: Based on the first image feature, calculating a channel attention matrix, wherein the channel attention matrix is ​​used to characterize the influence of each channel on other channels; Based on the channel attention matrix and the first image feature, the fourth image feature is generated.

10. The defect detection method according to any one of claims 6 to 9, characterized in that: The feature matching of the image features to be classified based on the preset feature library to determine the detection result includes: Determine feature distribution information of target devices of the same category as the device under test in a preset feature library; Calculating a matching value between the device under test and the target device based on the image feature to be classified and the feature distribution information, wherein the image feature to be classified is the first image feature or the second image feature; Based on the matching value, determining a matching result between the device under test and the target device, wherein the matching result includes a match and a mismatch; In the case where the device under test and the target device match, determining that the detection result is normal; In the case that the device under test and the target device do not match, the detection result is determined to be abnormal.

11. A defect detection device, characterized in that: include: An extraction module, used to extract a first image feature of an image to be tested by using a feature extraction module of a preset detection model, wherein the image to be tested is an image including the object to be tested; A matching module is used to perform feature matching on the first image feature based on a preset feature library to determine a detection result, wherein the detection result includes normal or abnormal, and the feature library at least includes the features of each component of the normal device extracted by the feature extraction module, and the normal device is of the same category as the device to be tested.

12. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the defect detection method according to any one of claims 1 to 10 is implemented.

13. An electronic device, characterized in that: include: A processor and a memory, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the defect detection method according to any one of claims 1 to 10 is implemented.

14. A defect detection system, characterized in that: It comprises an image acquisition device and the electronic device as claimed in claim 13, wherein the image acquisition device is used to acquire the image to be tested.

15. A computer program product, characterized in that It comprises a computer program, which, when executed by a processor, implements the defect detection method according to any one of claims 1 to 10.

Images