PCB (Printed Circuit Board) anomaly detection method and system

Abstract

The invention relates to the field of anomaly detection, and discloses a PCB (Printed Circuit Board) anomaly detection method, which comprises the following steps of: acquiring a PCB template image and a to-be-detected PCB image; registering and aligning the image of the PCB to be detected with the image of the PCB template; obtaining one or more potential abnormal areas of the to-be-detected PCB image by comparing the edge energy; and comparing the potential abnormal area with a coordinate frame area corresponding to the PCB template image by using a measurement model, and confirming an abnormal state through a comparison result. According to the scheme, PCB abnormity detection can be realized accurately, and when a new PCB and a new component are introduced, only a current PCB normal model needs to be newly added.

Description

Technical Field

[0001] The present invention relates to the field of abnormality detection, and in particular to a PCB board abnormality detection method and system. Background Art

[0002] PCB (Printed Circuit Board) is an important component of electronic products, used to support and connect electronic components. During the production process of PCB boards, abnormality detection is required to ensure the normal welding of components on the board.

[0003] Existing common technical solutions include implementation based on traditional image processing algorithms and methods based on deep learning.

[0004] The idea of ​​template matching is based on the traditional image processing algorithm. By comparing the PCB board in the real-time production environment with the normal samples collected in advance from the production environment, the area with a difference greater than a certain threshold is defined as an abnormal area. However, the shapes of the components soldered on the normal PCB board are not completely consistent, especially the separated components. Template matching is difficult to adapt to the differences in the shapes of normal soldered components. On the other hand, template matching also has extremely demanding requirements for shooting and lighting.

[0005] The deep learning-based method is defined as: pre-defining the component types of the PCB board and collecting data covering the components to train the target detection model of the components. However, when new components are introduced into the production line, the detection model needs to collect data again for training and model update; Another PCB anomaly detection algorithm based on deep learning is to obtain the pixel distribution of a normal PCB board through model training. The local pixels of the abnormal PCB board will not satisfy the learned pixel distribution, and the abnormal area can be found. However, in the case of abundant components, it is extremely difficult for the model to learn the pixel distribution of the PCB board. Summary of the invention

[0006] The main purpose of the present invention is to solve the technical problems faced by the prior art, which are either not robust or have high deployment costs. The present invention provides a PCB board abnormality detection method, comprising the following steps: Acquire a PCB template image and an image of a PCB board to be detected; register and align the image of the PCB board to be detected with the PCB template image; obtain one or more potential abnormal areas of the image of the PCB board to be detected by comparing edge energy; use a metric model to compare the potential abnormal area with the coordinate frame area corresponding to the PCB template image, and confirm the abnormal state through the comparison result.

[0007] As a preferred technical solution, the step of acquiring the PCB template image and the image of the PCB board to be detected includes: A normal PCB board is placed horizontally in the middle of the platform, and an image of the normal PCB board is captured to obtain a PCB template image; the PCB template image is bound to the current PCB to establish an association relationship; when detecting anomalies, an image of the PCB board placed horizontally in the middle of the platform is obtained to obtain an image of the PCB board to be detected.

[0008] As a preferred technical solution, the registering and aligning the image of the PCB board to be detected with the image of the PCB template includes: Detecting corner points, edges, and spot features of the PCB board image to be detected and the PCB template image, and generating corresponding key point descriptors; performing key point matching on the PCB board image to be detected and the PCB template image according to the key point descriptors to obtain matching key point pairs, thereby establishing a corresponding relationship between the two images; calculating a transformation matrix through the matching key point pairs; applying the transformation matrix to the PCB board image to be detected, and transforming the PCB board image to be detected to a position aligned with the PCB template image.

[0009] As a preferred technical solution, the step of obtaining one or more potential abnormal areas of the image of the PCB board to be detected by comparing edge energy includes: The edge information of the PCB board image to be detected and the PCB template image is extracted by using an edge detection algorithm; for each detected edge, its edge energy is calculated; the edge energy value of each pixel point is used as the gray value of the point to construct an edge energy map of the entire image; the two edge energy maps are normalized so that the gray value ranges of the two edge energy maps are the same, and the normalized edge energy map is obtained; a difference map between the two normalized edge energy maps is calculated, and each pixel value in the difference map represents the difference in edge energy of the corresponding position in the two images; a pixel segmentation threshold is set to segment the pixels in the difference map into two categories: significant difference and insignificant difference; based on the segmentation result of the difference map, all continuous and significant pixel sets of the PCB board image to be detected are found through a connected domain analysis algorithm, and the position corresponding to the pixel set is a potential abnormal area.

[0010] As a preferred technical solution, the method of using the measurement model to compare the potential abnormal area with the coordinate frame area corresponding to the PCB template image, and confirming the abnormal state through the comparison result, includes: The potential abnormal area and the coordinate frame area corresponding to the PCB template image are preprocessed to obtain a preprocessed image; a feature vector is extracted from the preprocessed image area using a metric model; the similarity between the feature vector of the potential abnormal area and the feature vector of the coordinate frame area corresponding to the PCB template image is calculated; a similarity threshold is set, and whether it is abnormal is determined according to the similarity threshold.

[0011] As a preferred technical solution, the method of extracting a feature vector from a preprocessed image region using a metric model includes: A measurement model is selected, wherein the measurement model includes one or more of SIFT, SURF, HOG, and CNN; a preprocessed image region is input into the selected measurement model; a series of feature descriptors are generated according to the content of the preprocessed image using the measurement model; and the extracted feature descriptors are encoded by a bag of words (BoW) model and a VLAD (Vector of Locally Aggregated Descriptors) method to generate a feature vector.

[0012] As a preferred technical solution, the measurement model is trained in the following manner: Collect images of various components to ensure the diversity and richness of data to cover components of different shapes, colors and sizes; enhance the images by rotation, scaling, cropping and flipping; build a component feature extraction model based on the DINO model; for each component image, select images similar to it as positive samples and images dissimilar to it as negative samples, and the definition of similarity is set by the contour, color and texture features of the component; optimize the model parameters using the contrastive learning loss function, and the loss function is designed to enable the model to correctly distinguish between positive and negative sample pairs; set training parameters: including learning rate, batch size, and number of training rounds; train the model using the gradient descent optimization algorithm, and update the model parameters by minimizing the contrastive learning loss function; monitor the performance of the model through the validation set to ensure that the model is not overfitting or underfitting; use the test set to evaluate the trained model and calculate its performance indicators on the component feature extraction task, such as accuracy and recall; tune the model according to the evaluation results, including adjusting the model structure, parameters or training strategy, to obtain a model for local area measurement of PCB boards.

[0013] A second aspect of the present invention provides a PCB board abnormality detection device, comprising: A platform for placing the PCB board and an industrial camera that is fixed and looks down at the platform to shoot the image; The device also includes: An acquisition unit, the acquisition unit is used to acquire a PCB template image and an image of a PCB board to be detected; An alignment unit, the alignment unit is used to register and align the image of the PCB board to be detected with the image of the PCB template; An abnormal region confirmation unit, the abnormal region confirmation unit is used to obtain one or more potential abnormal regions of the image of the PCB board to be detected by comparing edge energy; The abnormal state confirmation unit is used to use the measurement model to compare the potential abnormal area with the coordinate frame area corresponding to the PCB template image, and confirm the abnormal state through the comparison result.

[0014] A third aspect of the present invention provides an electronic device, comprising: a memory and at least one processor, wherein instructions are stored in the memory, and the memory and the at least one processor are interconnected via lines; the at least one processor calls the instructions in the memory so that the electronic device executes the above-mentioned PCB board abnormality detection method.

[0015] A fourth aspect of the present invention provides a computer-readable storage medium, wherein instructions are stored in the computer-readable storage medium, and when the computer-readable storage medium is run on a computer, the computer executes the above-mentioned PCB board abnormality detection method.

[0016] The present invention has the following beneficial effects: The solution of the present invention identifies potential abnormal areas by comparing edge energy. The richness of edge information in the PCB board image is utilized. By comparing the difference in edge energy between the template image and the image to be detected, the area where abnormalities may exist can be quickly and accurately located. This method is more sensitive and accurate than the traditional difference detection method based solely on pixel values.

[0017] This solution can realize PCB board anomaly detection with high accuracy. At the same time, when new PCB boards and new components are introduced, it is only necessary to add the normal model of the current PCB board. The solution of the present invention introduces a measurement model to compare the potential abnormal area with the corresponding coordinate frame area of ​​the template image. This measurement model includes the comparison of multiple features, such as shape, texture, color, etc., and can provide more comprehensive and detailed anomaly detection. Because there are some commonalities between different components, such as the outlines of components are roughly similar and the color distribution is similar. When this measurement model is trained, it has a stronger ability to generalize-no retraining is required when new components are introduced.

[0018] The edge energy algorithm focuses on abnormal changes in edge information and has a good detection effect on certain types of anomalies (such as component missing, misalignment, etc.). However, for other anomalies (such as component color change, slight deformation, etc.), it is necessary to rely on the measurement model algorithm for comparison and analysis. Therefore, the combination of the two can cover a wider range of anomaly types. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A first flow chart of a PCB board abnormality detection method provided by an embodiment of the present invention; Figure 2A second flow chart of the PCB board abnormality detection method provided by an embodiment of the present invention; Figure 3 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0020] The embodiment of the present invention provides a method and device for detecting abnormalities of a PCB board. The method comprises: obtaining a PCB template image and an image of a PCB board to be detected; registering and aligning the image of the PCB board to be detected with the PCB template image; obtaining one or more potential abnormal areas of the image of the PCB board to be detected by comparing edge energy; using a metric model to compare the potential abnormal area with the corresponding coordinate frame area of ​​the PCB template image, and confirming the abnormal state through the comparison result. Because there are some commonalities between different components, such as the contours of the components are roughly similar, the color distribution is similar, etc. When this metric model is trained, it has a stronger ability to be generalized-no retraining is required when new components are introduced.

[0021] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0022] Edge Energy Algorithm: The edge energy algorithm focuses on the changes in edge information in the image. The components and circuits on the PCB usually have clear edge features. The integrity, continuity and clarity of these edges can reflect the quality status of the PCB. The edge energy algorithm can quickly detect abnormal changes in edge information, such as breakage, blur or misalignment, so as to locate potential problem areas.

[0023] Metrics Model: The metric model is used to extract discriminative feature vectors of image regions. Discriminative features can ensure that the cosine distance between feature vectors of similar images is close to 0, and the cosine distance between feature vectors of dissimilar images is close to 1.

[0024] The metric model algorithm focuses more on the detailed comparison and analysis of specific areas in the image. It builds a comparison model based on multiple features (such as shape, texture, color, etc.), and can accurately determine whether there is an abnormality by comparing the feature differences of the corresponding areas in the template image and the image to be detected, and can classify and identify the type of abnormality.

[0025] For ease of understanding, the specific process of the embodiment of the present invention is described below. Figure 1 , a first embodiment of the PCB board abnormality detection method in an embodiment of the present invention includes: 101. Obtain a PCB template image and an image of a PCB board to be inspected; Specifically, in an embodiment, the PCB template image is a standard or reference image of a PCB board, which is usually taken in a defect-free or ideal state. The image needs to contain all key components and layout information on the PCB board for subsequent comparison operations. The image of the PCB board to be inspected is the image of the PCB board that needs to be inspected on the actual production line. The acquisition of the image must ensure clarity and resolution so that potential defects can be captured.

[0026] 102. Register and align the image of the PCB board to be inspected with the image of the PCB template; In this embodiment, the purpose of registration is to find the spatial transformation relationship between the two images so that the image to be detected can be aligned with the template image in space. This is usually achieved through steps such as feature point detection, feature matching, and transformation matrix calculation. After the registration is completed, the image to be detected is transformed according to the calculated transformation matrix so that it is aligned with the template image. The aligned image should be consistent with the template image in spatial position, providing a basis for subsequent anomaly detection.

[0027] 103. Obtain one or more potential abnormal areas of the image of the PCB board to be inspected by comparing edge energy; In this embodiment, edge energy generally reflects the clarity and strength of the edge in the image. In the PCB image, the edge of the component should be clear and continuous. If the edge is blurred or broken, it may indicate a defect. By calculating the edge energy difference between the image to be detected and the template image in the corresponding area, potential abnormal areas can be identified. These areas are usually manifested as significant changes in edge energy.

[0028] 104. Use the measurement model to compare the potential abnormal area with the coordinate frame area corresponding to the PCB template image, and confirm the abnormal state through the comparison result.

[0029] According to the characteristics of the PCB board and the detection requirements, select the appropriate measurement model. These models include grayscale difference, texture difference, shape difference, etc. For each potential abnormal area, extract its features in the image to be detected and compare them with the features of the corresponding coordinate box area in the template image. The comparison process involves steps such as feature vector calculation and similarity measurement.

[0030] According to the comparison results, if the features of the potential abnormal area are significantly different from those in the template image, the area is confirmed to be an abnormal area. Whether the difference is significant can be determined by setting a threshold or using machine learning methods.

[0031] In the embodiment of the present invention, the edge energy algorithm has a good detection effect for components such as missing and misaligned. However, for components such as color change and slight deformation, it is necessary to rely on the metric model algorithm for comparison and analysis. Therefore, the combination of the two can cover a wider range of abnormal types.

[0032] There is another embodiment of the present invention, and its specific process is described in detail. Figure 2 : 201. Place a normal PCB board horizontally in the middle of the platform, capture an image of the normal PCB board, and obtain a PCB template image; bind the PCB template image with the current PCB to establish an association relationship; when detecting an abnormality, obtain an image of the PCB board horizontally placed in the middle of the platform to obtain an image of the PCB board to be detected.

[0033] Specifically, a normal PCB board is placed horizontally in the middle of the platform. A camera or image acquisition device is used to capture an image of the PCB board. The obtained image is the PCB template image, which is the appearance of a normal, defect-free PCB board.

[0034] Binding the captured PCB template image to the current PCB means associating this template with a specific PCB model or batch. This association is established so that in the subsequent inspection process, this template can be used as a reference to compare and detect whether there are abnormalities in other PCB boards.

[0035] During the inspection process, the PCB to be inspected is also placed horizontally in the middle of the platform. The image of the PCB to be inspected is acquired using the same image acquisition device. The image obtained is the image of the PCB to be inspected, which will be compared and analyzed with the previously established PCB template image.

[0036] 202. Detect corner points, edges, and spot features of the PCB board image to be detected and the PCB template image, and generate corresponding key point descriptors; perform key point matching on the PCB board image to be detected and the PCB template image according to the key point descriptors to obtain matching key point pairs, thereby establishing a corresponding relationship between the two images; calculate a transformation matrix through the matching key point pairs; apply the transformation matrix to the PCB board image to be detected, and transform the PCB board image to be detected to a position aligned with the PCB template image.

[0037] Corner detection: Corner points are points with dramatic changes in an image, such as the intersection of edges. Commonly used corner detection algorithms include Harris corner detection and FAST corner detection.

[0038] Edge detection: An edge is an area in an image where the grayscale changes significantly, such as the wire or component boundary on a PCB. The Canny edge detection algorithm is commonly used.

[0039] Blob detection: Blobs refer to connected areas with similar grayscale values ​​in an image, such as pads or components on a PCB. SIFT and SURF algorithms can be used for blob detection.

[0040] After detecting these features, a descriptor is generated for each feature. The descriptor encodes the pixel information around the feature for subsequent feature matching.

[0041] Using the generated key point descriptors, key point matching is performed on the PCB board image to be inspected and the PCB template image. This usually involves similarity calculations between descriptors, such as Euclidean distance or cosine similarity. The matching result is a series of corresponding key point pairs.

[0042] Through the matching key point pairs, the correspondence between the two images can be established. These correspondences can be further used to calculate the transformation matrix, which describes the geometric transformation (such as rotation, translation, scaling, etc.) from the image of the PCB board to be inspected to the image of the PCB template. Commonly used transformation models include affine transformation and perspective transformation. The specific choice depends on the degree of deformation between the images and the distribution of matching points.

[0043] The calculated transformation matrix is ​​applied to the image of the PCB board to be inspected, and it is transformed to a position aligned with the PCB template image. In this way, the same features in the two images will correspond in space, which is convenient for subsequent anomaly detection.

[0044] 203. Obtain one or more potential abnormal regions of the image of the PCB board to be inspected by comparing edge energy; Specifically, edge information of the PCB board image to be detected and the PCB template image is extracted using an edge detection algorithm; for each detected edge, its edge energy is calculated; the edge energy value of each pixel point is used as the grayscale value of the point to construct an edge energy map of the entire image; the two edge energy maps are normalized so that the grayscale value ranges of the two edge energy maps are the same, and the normalized edge energy map is obtained; a difference map between the two normalized edge energy maps is calculated, and each pixel value in the difference map represents the difference in edge energy of the corresponding position in the two images; a pixel segmentation threshold is set to segment the pixels in the difference map into two categories: significant difference and insignificant difference; based on the segmentation result of the difference map, all continuous and significant pixel sets of the PCB board image to be detected are found through a connected domain analysis algorithm, and the position corresponding to the pixel set is a potential abnormal area.

[0045] Use edge detection algorithms (such as Canny, Sobel, Prewitt, etc.) to extract edge information of the PCB board image to be inspected and the PCB template image. These algorithms can identify areas in the image where grayscale or color changes significantly, i.e., edges.

[0046] For each detected edge, calculate its edge energy. Edge energy is usually related to the strength and clarity of the edge and can be obtained by calculating the gradient or second-order derivative of the edge pixels.

[0047] The edge energy value of each pixel is used as the gray value of the point to construct the edge energy map of the entire image. In this way, each pixel in the image represents an edge energy value.

[0048] Since the PCB image to be inspected and the template image may have differences in shooting conditions, lighting and other factors, the grayscale value ranges of their edge energy maps may be different. Therefore, it is necessary to normalize the two edge energy maps to make their grayscale value ranges the same for subsequent comparison.

[0049] Calculate the difference map between the two normalized edge energy maps. Each pixel value in the difference map represents the difference in edge energy of the corresponding position in the two images. The larger the difference, the greater the difference in edge energy between the image to be detected and the template image.

[0050] Set a pixel segmentation threshold to divide the pixels in the difference map into two categories: significant difference and insignificant difference. The selection of this threshold is usually based on the needs and experience of the actual application.

[0051] Based on the segmentation results of the difference map, the connected domain analysis algorithm is used to find all continuous and significantly different pixel sets in the PCB board image to be inspected. The positions corresponding to these pixel sets are considered to be potential abnormal areas. The connected domain analysis algorithm can help identify these continuous and significantly different areas.

[0052] 204. Preprocess the potential abnormal area and the coordinate frame area corresponding to the PCB template image to obtain a preprocessed image; select a measurement model, the measurement model includes one or more of SIFT, SURF, HOG, and CNN; input the preprocessed image area into the selected measurement model; use the measurement model to generate a series of feature descriptors according to the content of the preprocessed image; encode the extracted feature descriptors through a bag-of-words model and a VLAD method to generate a feature vector; calculate the similarity between the feature vector of the potential abnormal area and the feature vector of the coordinate frame area corresponding to the PCB template image; set a similarity threshold, and judge whether it is abnormal according to the similarity threshold.

[0053] Preprocess the potential abnormal area and the coordinate frame area corresponding to the PCB template image. Preprocessing includes grayscale, noise reduction, contrast enhancement and other operations, the purpose is to improve image quality and reduce interference in subsequent feature extraction.

[0054] Choose a suitable metric model to extract image features. The metric model can be a traditional feature extraction method (such as SIFT, SURF, HOG) or a deep learning method (such as CNN). These models can extract representative feature descriptors from images.

[0055] The preprocessed image region is input into the selected metric model to generate a series of feature descriptors. These descriptors are usually able to capture local or global structural information in the image.

[0056] The extracted feature descriptors are encoded using methods such as the Bag of Words (BoW) model or VLAD to generate feature vectors. These methods can convert high-dimensional feature descriptors into low-dimensional feature vectors, which facilitates subsequent similarity calculations.

[0057] Calculate the similarity between the feature vector of the potential abnormal area and the feature vector of the coordinate frame area corresponding to the PCB template image. The similarity calculation can be based on a distance metric (such as Euclidean distance) or a similarity metric (such as cosine similarity). The metric model is used to extract discriminative feature vectors of the image area. The discriminative features can ensure that the cosine distance between feature vectors of similar images is close to 0, and the cosine distance between feature vectors of dissimilar images is close to 1.

[0058] A similarity threshold is set to determine whether the potential abnormal area is actually abnormal. If the similarity is lower than the threshold, the area is considered abnormal; otherwise, the area is considered normal.

[0059] Among them, the measurement model is trained in the following way: Collect images of various components to ensure the diversity and richness of data to cover components of different shapes, colors and sizes; enhance the images by rotation, scaling, cropping and flipping; build a component feature extraction model based on the DINO model; for each component image, select images similar to it as positive samples and images dissimilar to it as negative samples, and the definition of similarity is set by the contour, color and texture features of the component; optimize the model parameters using the contrastive learning loss function, and the loss function is designed to enable the model to correctly distinguish between positive and negative sample pairs; set training parameters: including learning rate, batch size, and number of training rounds; train the model using the gradient descent optimization algorithm, and update the model parameters by minimizing the contrastive learning loss function; monitor the performance of the model through the validation set to ensure that the model is not overfitting or underfitting; use the test set to evaluate the trained model and calculate its performance indicators on the component feature extraction task, such as accuracy and recall; tune the model according to the evaluation results, including adjusting the model structure, parameters or training strategy, to obtain a model for local area measurement of PCB boards.

[0060] The details are as follows: Collect images of various components to ensure the diversity and richness of the data, covering components of different shapes, colors, and sizes. This is to fully learn the characteristics of various components when training the model.

[0061] The image is enhanced by rotating, scaling, cropping, flipping, etc. to increase the generalization ability of the model. These operations can simulate various postures and deformations that occur in component PCB board anomaly detection in actual scenarios.

[0062] Based on the DINO model (or other applicable deep learning models), a component feature extraction model is built. The DINO model PCB board anomaly detection is a deep neural network that can extract deep features in images.

[0063] For each component image, select similar images as positive samples and dissimilar images as negative samples. The definition of similarity can be based on the component's outline, color, texture, etc. This helps the model learn how to distinguish different components.

[0064] The model parameters are optimized using the contrastive learning loss function, which aims to enable the model to correctly distinguish positive and negative sample pairs, that is, for positive sample pairs, the feature vectors output by the model should be similar; for negative sample pairs, the feature vectors output by the model should be dissimilar.

[0065] Set training parameters, including learning rate, batch size, number of training rounds, etc. The choice of these parameters will affect the training speed and effect of the model.

[0066] The model is trained using a gradient descent optimization algorithm to update the model parameters by minimizing the contrastive learning loss function.

[0067] Monitor the performance of the model through the validation set to ensure that the model is not overfitting or underfitting. Overfitting means that the model performs well on the training data but performs poorly on new data; underfitting means that the model does not perform well on the training data.

[0068] Use the test set to evaluate the trained model and calculate its performance indicators on the component feature extraction task, such as accuracy, recall, etc. These indicators can reflect the performance of the model in practical applications.

[0069] Tune the model based on the evaluation results, including adjusting the model structure, parameters or training strategy. The purpose of tuning is to further improve the performance of the model so that it can better adapt to actual application scenarios.

[0070] Through the above process, a component feature extraction model for local area measurement of PCB boards can be obtained. This model can extract representative features of components and be used in subsequent anomaly detection or recognition tasks.

[0071] The above describes the PCB board abnormality detection method in the embodiment of the present invention. The following describes the PCB board abnormality detection device in the embodiment of the present invention. Figure 3 , the first embodiment of the PCB board abnormality detection device in the embodiment of the present invention includes: A platform for placing the PCB board and an industrial camera that is fixed and looks down at the platform to shoot the image; The device also includes: An acquisition unit, the acquisition unit is used to acquire a PCB template image and an image of a PCB board to be detected; An alignment unit, the alignment unit is used to register and align the image of the PCB board to be detected with the image of the PCB template; An abnormal region confirmation unit, the abnormal region confirmation unit is used to obtain one or more potential abnormal regions of the image of the PCB board to be detected by comparing edge energy; The abnormal state confirmation unit is used to use the measurement model to compare the potential abnormal area with the coordinate frame area corresponding to the PCB template image, and confirm the abnormal state through the comparison result.

[0072] The embodiment of the present invention also provides a structure of an electronic device, which may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) (for example, one or more processors) and memories, and one or more storage media for storing applications or data (for example, one or more mass storage devices). Among them, the memory and the storage medium may be short-term storage or permanent storage. The program stored in the storage medium may include one or more modules, each of which may include a series of instruction operations in the electronic device. Furthermore, the processor may be configured to communicate with the storage medium and execute a series of instruction operations in the storage medium on the electronic device. The electronic device may also include one or more power supplies, one or more wired or wireless network interfaces, one or more input and output interfaces, and / or, one or more operating systems, such as Windows Serve, Mac OS X, Unix, Linux, FreeBSD, etc.

[0073] The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When the instructions are executed on a computer, the computer executes the steps of the PCB board abnormality detection method.

[0074] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described system, device, or unit can refer to the corresponding process in the aforementioned method embodiment and will not be repeated here.

[0075] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the whole or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0076] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A PCB board abnormality detection method, characterized in that: The following steps are involved: Obtain the PCB template image and the PCB board image to be inspected; Registering and aligning the image of the PCB board to be inspected with the image of the PCB template; By comparing the edge energy, one or more potential abnormal areas of the image of the PCB board to be inspected are obtained; The measurement model is used to compare the potential abnormal area with the corresponding coordinate frame area of ​​the PCB template image, and the abnormal state is confirmed through the comparison result.

2. A PCB board abnormality detection method according to claim 1, characterized in that: The step of acquiring the PCB template image and the image of the PCB board to be detected includes: Place a normal PCB board horizontally in the middle of the platform, capture an image of the normal PCB board, and obtain a PCB template image; Bind the PCB template image with the current PCB to establish an association relationship; During anomaly detection, an image of a PCB board placed horizontally in the middle of the platform is acquired to obtain an image of the PCB board to be detected.

3. A PCB board abnormality detection method according to claim 1, characterized in that: The registering and aligning the image of the PCB board to be inspected with the image of the PCB template includes: Detecting corner points, edges, and spot features of the PCB board image to be detected and the PCB template image, and generating corresponding key point descriptors; According to the key point descriptor, the key point matching is performed on the PCB board image to be detected and the PCB template image to obtain a matching key point pair, thereby establishing a corresponding relationship between the two images; A transformation matrix is ​​calculated by using the matched key point pairs; The transformation matrix is ​​applied to the image of the PCB board to be detected, and the image of the PCB board to be detected is transformed to a position aligned with the PCB template image.

4. A PCB board abnormality detection method according to claim 1, characterized in that: The step of obtaining one or more potential abnormal areas of the image of the PCB board to be detected by comparing the edge energy includes: Extract edge information of the PCB board image to be detected and the PCB template image using an edge detection algorithm; For each detected edge, calculate its edge energy; The edge energy value of each pixel is used as the gray value of the point to construct the edge energy map of the entire image; Normalizing the two edge energy maps so that the grayscale value ranges of the two edge energy maps are the same, thereby obtaining a normalized edge energy map; Calculating a difference map between two normalized edge energy maps, wherein each pixel value in the difference map represents the difference in edge energy of a corresponding position in the two images; Set a pixel segmentation threshold to divide the pixels in the difference map into two categories: significant difference and insignificant difference; Based on the segmentation result of the difference map, a connected domain analysis algorithm is used to find all continuous and significantly different pixel sets of the PCB board image to be detected, and the positions corresponding to the pixel sets are potential abnormal areas.

5. A PCB board abnormality detection method according to claim 1, characterized in that: The method of using the measurement model to compare the potential abnormal area with the coordinate frame area corresponding to the PCB template image, and confirming the abnormal state through the comparison result, includes: Preprocessing the potential abnormal area and the coordinate frame area corresponding to the PCB template image to obtain a preprocessed image; Extract feature vectors from preprocessed image regions using a metric model; Calculate the similarity between the feature vector of the potential abnormal area and the feature vector of the coordinate frame area corresponding to the PCB template image; A similarity threshold is set, and whether it is abnormal is determined based on the similarity threshold.

6. A PCB board abnormality detection method according to claim 5, characterized in that: The method of extracting a feature vector from a preprocessed image region using a metric model comprises: Select a measurement model, wherein the measurement model includes one or more of SIFT, SURF, HOG, and CNN; Input the preprocessed image region into the selected metric model; Using the metric model, generating a series of feature descriptors based on the content of the preprocessed image; The extracted feature descriptors are encoded through the bag-of-words model and the VLAD method to generate feature vectors.

7. A PCB board abnormality detection method according to claim 1, characterized in that: The metric model is trained in the following way: Collect images of various components to ensure data diversity and richness to cover components of different shapes, colors, and sizes; Enhance images by rotating, scaling, cropping, and flipping; Based on the DINO model, a component feature extraction model is constructed; For each component image, select similar images as positive samples and dissimilar images as negative samples. The definition of similarity is set by the contour, color, and texture features of the component. Optimize model parameters using a contrastive learning loss function that aims to enable the model to correctly distinguish between positive and negative sample pairs; Set training parameters: including learning rate, batch size, and number of training rounds; The model is trained using the gradient descent optimization algorithm and the model parameters are updated by minimizing the contrastive learning loss function; Monitor the performance of the model through the validation set to ensure that the model is not overfitting or underfitting; Use the test set to evaluate the trained model and calculate its performance indicators on the component feature extraction task, such as accuracy and recall rate; The model is tuned according to the evaluation results, including adjusting the model structure, parameters or training strategy, to obtain a model for measuring the local area of ​​the PCB board.

8. A PCB board abnormality detection device, characterized in that: The device comprises a platform for placing a PCB board and an industrial camera that is fixed and shoots the platform from above, and the device also comprises: An acquisition unit, the acquisition unit is used to acquire a PCB template image and an image of a PCB board to be detected; An alignment unit, the alignment unit is used to register and align the image of the PCB board to be detected with the image of the PCB template; An abnormal region confirmation unit, the abnormal region confirmation unit is used to obtain one or more potential abnormal regions of the image of the PCB board to be detected by comparing edge energy; The abnormal state confirmation unit is used to use the measurement model to compare the potential abnormal area with the coordinate frame area corresponding to the PCB template image, and confirm the abnormal state through the comparison result.

9. An electronic device, comprising a memory and at least one processor, wherein instructions are stored in the memory; The at least one processor calls the instruction in the memory so that the electronic device executes each step of the PCB board abnormality detection method as described in any one of claims 1-7.

10. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instruction is executed by the processor, each step of the PCB board abnormality detection method as described in any one of claims 1 to 7 is implemented.

Images