Defect detection method of printed circuit board and electronic equipment

Abstract

The invention provides a defect detection method of a printed circuit board and electronic equipment, which are used for improving the detection accuracy of the printed circuit board. Comprising the steps that a detection image of a target printed circuit board is acquired, and the detection image comprises local detection images; determining a target defect detection strategy corresponding to the element type of the element in the first local detection image by using a preset corresponding relationship between the element type and a defect detection strategy, the defect detection strategy including a template matching strategy, a connected region analysis strategy, a color comparison strategy and a deep learning strategy; the target defect detection strategy is at least one of the defect detection strategies, and the first local detection image is any one of the local detection images; performing defect detection on the first local detection image by using a target defect detection strategy to obtain a defect detection result of the first local detection image; and obtaining a defect detection result of the target printed circuit board according to the defect detection result of each local detection image.

Description

Background Art

[0002] In the manufacturing process of a Printed Circuit Board (PCB), Automated Optical Inspection (AOI) devices are widely used to detect defects on the circuit board. For example, they are used to detect defects such as poor soldering, broken wires, and short circuits. However, these optical inspection devices rely on traditional vision algorithms. For example, basic image processing, threshold segmentation, and binarization. These methods often face bottlenecks in accuracy and efficiency when dealing with complex components or complex environments. Therefore, the accuracy of PCB inspection is relatively low. Summary of the Invention

[0003] This application provides a method for detecting defects on a printed circuit board and an electronic device, which are used to improve the accuracy of printed circuit board inspection.

[0004] In a first aspect, an embodiment of this application provides a method for detecting defects on a printed circuit board. The method includes:

[0005] Obtain a detection image of a target printed circuit board sent by an Automated Optical Inspection (AOI) device, where the detection image includes local detection images; any one of the local detection images is an image corresponding to a local area of the target printed circuit board;

[0006] Using the pre-set correspondence between component types and defect detection strategies, determine the target defect detection strategy corresponding to the component type of the component in the first local detection image, where the defect detection strategies include template matching strategy, connected region analysis strategy, color comparison strategy, and deep learning strategy, the target defect detection strategy is at least one of the defect detection strategies, and the first local detection image is any one of the local detection images;

[0007] Use the target defect detection strategy to perform defect detection on the first local detection image to obtain the defect detection result of the first local detection image;

[0008] Obtain the defect detection result of the target printed circuit board according to the defect detection results of the local detection images.

[0009] In the embodiments of the present application, by using the pre-set correspondence between component types and defect detection strategies, the target defect detection strategy corresponding to the component type of the component in the first partial detection image is determined, and at least one defect detection strategy among the template matching strategy, the connected region analysis strategy, the color comparison strategy, and the deep learning strategy is used to perform defect detection on each partial detection image, so as to obtain the defect detection result of the target printed circuit board. Therefore, in the embodiments of the present application, defect detection is performed through the defect detection strategy corresponding to the component type. For complex components, defect detection can be performed by combining multiple strategies. Therefore, the accuracy of defect detection of the printed circuit board is improved.

[0010] In one embodiment, the using the target defect detection strategy to perform defect detection on the first partial detection image to obtain the defect detection result of the first partial detection image includes:

[0011] If the number of target defect detection strategies is one, the intermediate detection result obtained by the target defect detection strategy is determined as the defect detection result of the first partial detection image; or,

[0012] If the number of target defect detection strategies is multiple, and each intermediate detection result obtained by each target defect detection strategy is normal, it is determined that the defect detection result of the first partial detection image is free of defects; or,

[0013] If the number of target defect detection strategies is multiple, and there is a target intermediate detection result that identifies a defect among the intermediate detection results obtained by each target defect detection strategy, the target intermediate detection result is determined as the defect detection result of the first partial detection image.

[0014] In the embodiments of the present application, the defect detection result of the first partial detection image is determined based on the number of target defect detections and / or the intermediate detection results obtained from the number of target defect detections, ensuring the accuracy of the obtained defect detection result.

[0015] In one embodiment, each partial standard image is further included in the detection image, where any one partial standard image has a corresponding partial detection image and is the standard image of the corresponding partial detection image, and the components included in any one partial standard image are the same as the components included in its corresponding partial detection image;

[0016] The using the target defect detection strategy to perform defect detection on the first partial detection image includes:

[0017] If the target defect detection strategy includes the template matching strategy, match the local standard image corresponding to the first local detection image with each preset component template image to obtain a target component template corresponding to the first local detection image; use a target detection algorithm to perform component detection on the first local detection image to obtain the positions of the components in the first local detection image; according to the target component template and the positions of the components in the first local detection image, obtain a first intermediate detection result of the first local detection image.

[0018] If the target defect detection strategy includes the connected region analysis strategy, binarize the first local detection image to obtain a binary image of the first local detection image; use a connected component analysis algorithm to extract the solder region in the binary image of the first local detection image to obtain a target solder region in the first local detection image; compare the target solder region with the standard solder region in the local standard image corresponding to the first local detection image to obtain a second intermediate detection result of the first local detection image.

[0019] In the embodiments of the present application, the corresponding intermediate detection result is determined through the template matching strategy or the connected region analysis strategy to ensure a more suitable detection strategy for matching the first local detection image, thereby ensuring the detection accuracy of the first local detection image.

[0020] In one embodiment, using the target defect detection strategy to perform defect detection on the first local detection image includes:

[0021] If the target defect detection strategy includes a color comparison strategy, compare the gray value of the target solder region in the first local detection image with the gray value of the standard solder region in the local standard image corresponding to the first local detection image to obtain a third intermediate detection result of the first local detection image.

[0022] In the embodiments of the present application, the gray value of the target solder region in the first local detection image is compared with the gray value of the standard solder region in the local standard image corresponding to the first local detection image to obtain a third intermediate detection result of the first local detection image. This ensures that solder joints with abnormal colors can be identified, thereby ensuring the accuracy of defect detection.

[0023] In one embodiment, using the target defect detection strategy to perform defect detection on the first local detection image includes:

[0024] If the target defect detection strategy includes a deep learning strategy, input the first partial detection image into a pre-trained deep learning model for defect detection to obtain a fourth intermediate detection result of the first partial detection image.

[0025] In the embodiment of the present application, by inputting the first partial detection image into a pre-trained deep learning model for defect detection, that is, using deep learning for intelligent defect analysis, automated defect detection is realized, and the accuracy and efficiency of defect detection are improved.

[0026] In one embodiment, the deep learning model adds a centralized feature pyramid EVC module to the neck module of the target detection model, and adds a convolutional block attention module CBAM to the backbone module of the target detection model.

[0027] In the embodiment of the present application, by introducing a centralized feature pyramid module into the deep learning model, through pyramid-style feature extraction and centralized aggregation operations, the key semantic information in multi-scale features is integrated and enhanced. Effectively reduces the interference of irrelevant features, enhances the context information expression ability in target detection, and thus significantly improves the performance and adaptability of the model in multi-scale target detection tasks. Improves the accuracy of defect detection for printed circuit boards.

[0028] In one embodiment, the backbone module includes a first convolutional layer, a first feature conversion layer, a second feature conversion layer, a third feature conversion layer, a fourth feature conversion layer, a spatial pyramid pooling SPPF layer, and two CBAM layers; wherein, the first convolutional layer, the first feature conversion layer, the second feature conversion layer, the third feature conversion layer, the fourth feature conversion layer, and the SPPF layer are connected in sequence, the second feature conversion layer is also connected to one of the CBAM layers, and the third feature conversion layer is also connected to the other CBAM layer;

[0029] The neck module includes a first upsampling layer, an EVC module, a fifth feature conversion layer, a second upsampling layer, a sixth feature conversion layer, a second convolutional layer, a seventh feature conversion layer, a third convolutional layer, and an eighth feature conversion layer; the first upsampling layer, the EVC module, the fifth feature conversion layer, the second upsampling layer, the sixth feature conversion layer, the second convolutional layer, the seventh feature conversion layer, the third convolutional layer, and the eighth feature conversion layer are connected in sequence, one of the CBAM layers is also connected to the fifth feature conversion layer, and the other CBAM layer is also connected to the sixth feature conversion layer; the fifth feature conversion layer is also connected to the seventh feature conversion layer; the SPPF layer is also connected to the first upsampling layer and the eighth feature conversion layer respectively.

[0030] In one embodiment, any one of the fifth to eighth feature transformation layers includes a feature fusion module and a feature transformation module;

[0031] Inputting the first local detection image into a pre-trained deep learning model for defect detection to obtain a fourth intermediate detection result of the first local detection image, including:

[0032] Performing a convolution operation on the first local detection image by using the first convolutional layer to obtain a first intermediate feature map;

[0033] Successively performing feature transformation on the first intermediate feature map by using a target feature transformation layer to obtain a second intermediate feature map; wherein, the target feature transformation layer includes the first feature transformation layer, the second feature transformation layer, the third feature transformation layer, and the fourth feature transformation layer;

[0034] Performing a pooling operation on the second intermediate feature map by using the SPPF layer to obtain a first feature map;

[0035] Performing an upsampling operation on the first feature map by using the first upsampling layer to obtain a second feature map;

[0036] Performing feature enhancement on the second feature map by using the EVC module to obtain a third feature map;

[0037] Performing feature fusion on the third feature map and a first feature transformation map by using the feature fusion module in the fifth feature transformation layer to obtain a third intermediate feature map, wherein the first feature transformation map is obtained based on the third feature transformation module;

[0038] Performing feature transformation on the third intermediate feature map by using the feature transformation module in the fifth feature transformation layer to obtain a fourth feature map;

[0039] Performing an upsampling operation on the fourth feature map by using the second upsampling layer to obtain a fifth feature map;

[0040] Performing feature fusion on the fifth feature map and a second feature transformation map by using the feature fusion module in the sixth feature transformation layer to obtain a fourth intermediate feature map; wherein the second feature transformation map is obtained based on the second feature transformation module;

[0041] Performing feature transformation on the fourth intermediate feature map by using the feature transformation module in the sixth feature transformation layer to obtain a sixth feature map;

[0042] Performing a convolution operation on the sixth feature map by using the second convolutional layer to obtain a seventh feature map;

[0043] Use the feature fusion module in the seventh feature conversion layer to perform feature fusion on the seventh feature map and the fourth feature map to obtain a fifth intermediate feature map;

[0044] Use the feature conversion module in the seventh feature conversion layer to perform feature conversion on the fifth intermediate feature map to obtain an eighth feature map;

[0045] Use the third convolutional layer to perform a convolutional operation on the eighth feature map to obtain a ninth feature map;

[0046] Use the feature fusion module in the eighth feature conversion layer to perform feature fusion on the ninth feature map and the first feature map to obtain a sixth intermediate feature map;

[0047] Use the feature conversion module in the eighth feature conversion layer to perform feature conversion on the sixth intermediate feature map to obtain a fourth intermediate detection result of the first local detection image.

[0048] In one embodiment, the detection image further includes a global detection image of the target printed circuit board;

[0049] Before determining the target defect detection strategy corresponding to the component type of the component in the first local detection image by using the pre-set correspondence between the component type and the defect detection strategy, the method further includes:

[0050] Perform pixel screening on the first local measured image to obtain target pixel points;

[0051] If the number of the target pixel points does not meet the specified condition, then crop an image corresponding to the first local detection area from the global detection image, and use the cropped image to replace the first local measured image.

[0052] In the embodiments of the present application, the local detection image that does not meet the specified condition is replaced by the cropped image segmented from the global detection image to ensure the speed and accuracy of the defect detection of the printed circuit board.

[0053] A second aspect of the present application provides an electronic device, including a processor and a memory, and the processor and the memory are connected through a bus;

[0054] The memory stores a computer program, and the processor is configured to perform the following operations based on the computer program:

[0055] Obtain the detection image of the target printed circuit board sent by the automatic optical inspection (AOI) device, where the detection image includes local detection images; any one of the local detection images is an image corresponding to a local area of the target printed circuit board.

[0056] Using the pre-set correspondence between component types and defect detection strategies, determine the target defect detection strategy corresponding to the component type of the components in the first local detection image, where the defect detection strategies include template matching strategy, connected region analysis strategy, color comparison strategy, and deep learning strategy, the target defect detection strategy is at least one of the defect detection strategies, and the first local detection image is any one of the local detection images.

[0057] Use the target defect detection strategy to perform defect detection on the first local detection image to obtain the defect detection result of the first local detection image.

[0058] According to the defect detection results of the local detection images, obtain the defect detection result of the target printed circuit board.

[0059] According to the third aspect provided by the embodiments of the present invention, a computer storage medium is provided. The computer storage medium stores a computer program, and the computer program is used to execute the method as described in the first aspect. Description of the Drawings

[0060] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0061] Figure 1 Exemplarily shows a schematic diagram of the application scenario provided by the embodiments of the present application;

[0062] Figure 2 Exemplarily shows a schematic flowchart of the method for defect detection of a printed circuit board provided by the embodiments of the present application;

[0063] Figure 3 Exemplarily shows a schematic diagram of the user registration interface provided by the embodiments of the present application;

[0064] Figure 4 Exemplarily shows a schematic diagram of the correspondence between component types and defect detection strategies provided by the embodiments of the present application;

[0065] Figure 5Exemplarily shows a schematic flowchart of defect detection using the template matching strategy provided by an embodiment of the present application;

[0066] Figure 6 Exemplarily shows a schematic flowchart of defect detection using the connected region analysis strategy provided by an embodiment of the present application;

[0067] Figure 7 Exemplarily shows a schematic structural diagram of a deep learning model provided by an embodiment of the present application;

[0068] Figure 8 Exemplarily shows a schematic flowchart of defect detection using a deep learning model provided by an embodiment of the present application;

[0069] Figure 9 Exemplarily shows a schematic diagram of a terminal interface provided by an embodiment of the present application;

[0070] Figure 10 Exemplarily shows a schematic diagram of cropping a global detection image provided by an embodiment of the present application;

[0071] Figure 11 Exemplarily shows a schematic diagram of a defect detection device for a printed circuit board provided by an embodiment of the present application;

[0072] Figure 12 Exemplarily shows a structural diagram of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0073] To make the objectives, implementation manners, and advantages of the present application clearer, the following will clearly and completely describe the exemplary implementation manners of the present application with reference to the accompanying drawings in the exemplary embodiments of the present application. Obviously, the described exemplary embodiments are only a part of the embodiments of the present application, rather than all of the embodiments.

[0074] Based on the exemplary embodiments described in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope protected by the appended claims of the present application. In addition, although the disclosed content in the present application is introduced according to exemplary one or several examples, it should be understood that each aspect of these disclosed contents can also be independently constituted as a complete implementation manner.

[0075] It should be noted that the brief description of the terms in the present application is only for the convenience of understanding the subsequent described implementation manners, rather than intending to limit the implementation manners of the present application. Unless otherwise specified, these terms should be understood according to their ordinary and general meanings.

[0076] In the description and claims of this application and the above-mentioned drawings, terms such as "first" and "second" are used to distinguish similar objects and do not necessarily describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances so that the embodiments of the present invention described here can be implemented in an order other than those illustrated or described here. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover but not exclude inclusion. For example, a product or device comprising a series of components does not necessarily have to be limited to those components clearly listed, but may include other components not clearly listed or inherent to these products or devices.

[0077] The term "module" used in this application refers to any known or later-developed hardware, software, firmware, artificial intelligence, fuzzy logic, or a combination of hardware and / or software code that can perform functions related to that element.

[0078] The idea of the embodiments of this application is outlined below.

[0079] In the prior art, automatic optical inspection equipment is widely used to detect defects on printed circuit boards, and the optical inspection equipment relies on traditional vision algorithms. For example, basic image processing, threshold segmentation, and binarization, etc. These methods often face bottlenecks in accuracy and efficiency when dealing with complex components or complex environments. Therefore, the accuracy rate of printed circuit board inspection is relatively low.

[0080] Therefore, the embodiments of this application provide a method for detecting defects on a printed circuit board. By using the corresponding relationship between the preset component type and the defect detection strategy, the target defect detection strategy corresponding to the component type of the component in the first local detection image is determined, and at least one defect detection strategy among the template matching strategy, the connected region analysis strategy, the color comparison strategy, and the deep learning strategy is used to detect defects in each local detection image, and the defect detection result of the target printed circuit board is obtained. Thus, in the embodiments of this application, defect detection is performed through the defect detection strategy corresponding to the component type. For complex components, defect detection can be performed by combining multiple strategies. Therefore, the accuracy rate of printed circuit board defect detection is improved.

[0081] As Figure 1 shown, an application scenario of a method for detecting defects on a printed circuit board, which includes an AOI inspection device 110 and a server 120 in this application scenario. In the embodiments of this application, an electronic device is taken as an example of the server for illustration.

[0082] In a possible application scenario, the AOI device 110 inspects a target printed circuit board to obtain a detection image of the target printed circuit board. Among them, the detection image includes local detection images; any one of the local detection images is an image corresponding to a local area in the target printed circuit board; and the detection image of the target printed circuit board is sent to the server 120. After receiving the detection image of the target printed circuit board sent by the AOI device 110, the server 120 determines a target defect detection strategy corresponding to the component type of the component in the first local detection image by using the pre-set correspondence between the component type and the defect detection strategy. Among them, the defect detection strategy includes a template matching strategy, a connected region analysis strategy, a color comparison strategy, and a deep learning strategy. The target defect detection strategy is at least one of the defect detection strategies. The first local detection image is any one of the local detection images; and the server 120 uses the target defect detection strategy to perform defect detection on the first local detection image to obtain the defect detection result of the first local detection image; finally, the server 120 obtains the defect detection result of the target printed circuit board according to the defect detection results of the local detection images.

[0083] Among them, Figure 1 The AOI device 110 and the server 120 in it can perform information interaction through a communication network. Among them, the communication method adopted by the communication network can be divided into a wireless communication method or a wired communication method.

[0084] Exemplarily, the AOI device 110 can access the network through cellular mobile communication technology and communicate with the server 120. Among them, the cellular mobile communication technology, for example, includes the fifth-generation mobile communication (5th Generation Mobile Networks, 5G) technology.

[0085] Optionally, the AOI device 110 can access the network through a short-range wireless communication method and communicate with the server 120. Among them, the short-range wireless communication method, for example, includes Wireless Fidelity (Wi-Fi) technology.

[0086] Among them, in the description of this application, only a single AOI device 110 and a single server 120 are described in detail. However, those skilled in the art should understand that the shown AOI device 110 and server 120 are intended to represent the operations of the AOI device 110 and server 120 involved in the technical solution of this application. It does not imply any limitation on the number, type, or location of the AOI device 110 and server 120. It should be noted that if additional modules are added to or individual modules are removed from the illustrated environment, the underlying concept of the exemplary embodiments of this application will not be changed.

[0087] It should be noted that the method for detecting defects in a printed circuit board proposed in this application is applicable not only to Figure 1 the application scenarios shown, but also to any device for detecting defects in a printed circuit board.

[0088] Next, in combination with the above-described application scenarios, the method for detecting defects in a printed circuit board according to an exemplary embodiment of the present application will be described with reference to the accompanying drawings. It should be noted that the above application scenarios are only shown for the convenience of understanding the method and principle of the present application, and the embodiments of the present application are not limited in this regard.

[0089] Next, the method for detecting defects in a printed circuit board in the present application will be described with reference to the accompanying drawings. As Figure 2 shown, it is a schematic flowchart of the method for detecting defects in a printed circuit board, which may specifically include the following steps:

[0090] Step 201: Obtain a detection image of a target printed circuit board sent by an AOI device, where the detection image includes local detection images; wherein, any one of the local detection images is an image corresponding to a local area in the target printed circuit board;

[0091] In an embodiment of the present application, the xml file sent by the AOI device is used to obtain the detection image of the target printed circuit board. The xml file includes the location where the detection image is stored, and the detection image of the target printed circuit board is obtained from the location where the detection image is stored. The xml file in the embodiment of the present application also includes the component types corresponding to the local detection images, that is, the component types of the components included in each local detection image.

[0092] Before performing step 201 in an embodiment of the present application, user login needs to be performed first. When logging in for the first time, default administrator accounts and user accounts are set, and administrators and ordinary users can be registered through user registration on the menu bar. As Figure 3 shown, it is a schematic diagram of the user registration interface. As can be seen from Figure 3 it, when registering, administrator registration or ordinary user registration can be performed.

[0093] Step 202: Determine a target defect detection strategy corresponding to the component type of the component in the first local detection image by using the pre-set correspondence between the component type and the defect detection strategy, where the defect detection strategy includes a template matching strategy, a connected region analysis strategy, a color comparison strategy, and a deep learning strategy, the target defect detection strategy is at least one of the defect detection strategies, and the first local detection image is any one of the local detection images;

[0094] In the embodiments of the present application, the component types included in the first partial detection image may be one or more. Therefore, the corresponding target defect detection strategies for the first partial detection image may also be one or more, which are not limited in the embodiments of the present application.

[0095] For example, as Figure 4 shown, it is a schematic diagram of the correspondence between component types and defect detection strategies. As can be seen from Figure 4 it, if the component types of the first partial detection image include component type a and component type b, the corresponding target defect detection strategies are defect detection strategy 1 and defect detection strategy 3.

[0096] Step 203: Use the target defect detection strategy to perform defect detection on the first partial detection image to obtain the defect detection result of the first partial detection image;

[0097] In one embodiment, step 203 may be specifically implemented in the following ways:

[0098] Method 1: If the number of the target defect detection strategies is one, determine the intermediate detection result obtained by the target defect detection strategy as the defect detection result of the first partial detection image.

[0099] Method 2: If the number of the target defect detection strategies is multiple and all the intermediate detection results obtained by each target defect detection strategy are normal, determine that the defect detection result of the first partial detection image is no defect.

[0100] Method 3: If the number of the target defect detection strategies is multiple and there is a target intermediate detection result that identifies a defect among the intermediate detection results obtained by each target defect detection strategy, determine the target intermediate detection result as the defect detection result of the first partial detection image.

[0101] The detection images in the embodiments of the present application also include each partial standard image. Among them, any partial standard image has a corresponding partial detection image and is the standard image of the corresponding partial detection image. The components included in any partial standard image are the same as those included in its corresponding partial detection image. Next, the method of performing defect detection on the first partial detection image by each defect detection strategy in the embodiments of the present application will be described.

[0102] It should be noted that: in the embodiments of the present application, if there are multiple target defect detection strategies, multiple target defect detection strategies can be executed in parallel in the embodiments of the present application, that is, the target defect detection strategies are executed in parallel by multiple threads, which improves the efficiency of defect detection of the printed circuit board.

[0103] For example, as Figure 5As shown in the figure, it is a schematic flowchart of defect detection using the template matching strategy, which may specifically include the following steps:

[0104] Step 501: Use the local standard image corresponding to the first local detection image to match with each preset component template image to obtain the target component template corresponding to the first local detection image;

[0105] In the embodiment of the present application, the template matching algorithm is used to match the local standard image corresponding to the first local detection image with each preset component template image to determine the target component template.

[0106] It should be noted that: the template matching algorithm in the embodiment of the present application can be the template matching algorithm in opencv. The embodiment of the present application does not limit the specific manner of the template matching algorithm, and the template matching algorithm in the embodiment of the present application can be set according to the specific actual situation.

[0107] Step 502: Use the target detection algorithm to perform component detection on the first local detection image to obtain the positions of the components in the first local detection image;

[0108] In the embodiment of the present application, the positions of the components include the identifiers of each solder joint and the position coordinates of each solder joint.

[0109] The target detection algorithm in the embodiment of the present application can be set according to the specific actual situation, and the embodiment of the present application does not limit the target detection algorithm here.

[0110] Step 503: Obtain the first intermediate detection result of the first local detection image according to the target component template and the positions of the components in the first local detection image.

[0111] In the embodiment of the present application, the target component template includes the identifiers of each solder joint in the component and the position coordinates of each solder joint.

[0112] In one embodiment, step 503 can be specifically implemented as:

[0113] Traverse each solder joint in the first partial detection image. For any first target solder joint traversed, compare the identifier of the first target solder joint with the identifiers of each solder joint in the target component template to determine whether there is a second target solder joint in the identifiers of each solder joint in the target component template that is the same as the identifier of the first target solder joint. If not, determine that the first intermediate detection result of the first partial detection image is solder joint missing. If so, determine the similarity between the first target solder joint and the second target solder joint according to the position coordinates of the first target solder joint and the position coordinates of the second target solder joint. If the similarity is greater than the specified threshold, return to the step of traversing each solder joint in the first partial detection image until it is determined that each solder joint in the first partial detection image has a corresponding second target solder joint and the corresponding similarity is greater than the specified threshold, then determine that the first intermediate detection result is defect-free. If the similarity is not greater than the specified threshold, determine that the first intermediate detection result is solder joint offset. Among them, the similarity between the first target solder joint and the second target solder joint can be obtained by formula (1):

[0114] d = √(x1 - x2) 2 +(y1 - y2) 2 …(1);

[0115] Where d is the similarity between the first target solder joint and the second target solder joint, x1 is the horizontal position coordinate of the first target solder joint, y1 is the vertical position coordinate of the first target solder joint, x2 is the horizontal position coordinate of the second target solder joint, and y2 is the vertical position coordinate of the second target solder joint.

[0116] As Figure 6 shown, it is a schematic flow chart of defect detection by the connected region analysis strategy, which specifically may include the following steps:

[0117] Step 601: Binarize the first partial detection image to obtain the binary image of the first partial detection image;

[0118] The binarization method in the embodiments of the present application is: for any pixel point in the first partial detection image, if the gray value of the any pixel point is greater than the specified gray value, set the gray value of the any pixel point to the first preset gray value; if the gray value of the any pixel point is not greater than the specified gray value, set the gray value of the any pixel point to the second preset gray value.

[0119] The specified grayscale value in the embodiments of the present application can be set according to specific actual situations, and the present application does not limit the specified grayscale value. The first preset grayscale value in the embodiments of the present application is 255, and the second preset grayscale value in the embodiments of the present application is 0. However, the present application does not limit the first preset grayscale value and the second preset grayscale value.

[0120] Step 602: Use the connected component analysis algorithm to extract the solder region in the binary image of the first partial detection image, and obtain the target solder region in the first partial detection image;

[0121] The connected component analysis algorithm in the embodiments of the present application belongs to the algorithms in the prior art, and the present application will not elaborate on it here.

[0122] Step 603: Compare the target solder region with the standard solder region in the local standard image corresponding to the first partial detection image to obtain the second intermediate detection result of the first partial detection image.

[0123] In one embodiment, step 603 can be specifically implemented as follows: If the target solder region is larger than the standard solder region, determine that the second intermediate detection result is excessive solder; if the target solder region is smaller than the standard solder region, determine that the second intermediate detection result is insufficient solder; if the shapes of the target solder region and the standard solder region are different, determine that the second intermediate detection result is shape abnormal.

[0124] In one embodiment, if the target defect detection strategy includes a color comparison strategy, compare the grayscale value of the target solder region in the first partial detection image with the grayscale value of the standard solder region in the local standard image corresponding to the first partial detection image to obtain the third intermediate detection result of the first partial detection image.

[0125] In the embodiments of the present application, if the difference between the grayscale value of the target solder region in the first partial detection image and the grayscale value of the standard solder region in the local standard image corresponding to the first partial detection image is greater than the preset value, determine that the third intermediate detection result is color abnormal; if the difference is not greater than the preset value, determine that the third intermediate detection result is no abnormality.

[0126] It should be noted that: The preset value in the embodiments of the present application can be set according to specific actual situations, and the present application does not limit the specific value of the preset value here.

[0127] The grayscale values of the target solder region and the standard solder region in the embodiments of the present application can be extracted through opencv, and the present application will not elaborate on it.

[0128] In one embodiment, if the target defect detection strategy includes a deep learning strategy, the first local detection image is input into a pre-trained deep learning model for defect detection to obtain a fourth intermediate detection result of the first local detection image.

[0129] First, a brief introduction to the training of the deep learning model in the embodiments of the present application is given:

[0130] Training samples are obtained, where the training samples include local training images and corresponding labeled defect types; the training samples are input into the deep learning model for training to obtain probabilities corresponding to the detected defect types; a loss value is obtained according to the probabilities corresponding to the detected defect types; if the loss does not meet the specified conditions, after adjusting the model parameters of the deep learning model, the step of inputting the training samples into the deep learning model for training is returned, until the loss value meets the specified conditions, and then the training of the deep learning model is ended.

[0131] In the embodiments of the present application, when making training samples, data augmentation can be performed on the local training images through image processing. The image processing methods in the embodiments of the present application include, but are not limited to, adding salt-and-pepper noise, adding Gaussian noise, adding Gaussian blur, histogram equalization, vertical screen flipping, and edge enhancement.

[0132] In the embodiments of the present application, the Focal Loss function is introduced as the loss function of the deep learning model. The Focal Loss function adjusts the weights of samples through a dynamic scaling factor, reducing the contribution weight of easily distinguishable samples to the overall loss, so as to focus the model training on difficult-to-distinguish samples. This dynamic adjustment mechanism can effectively enhance the learning ability of the deep learning model for weak feature samples during the training stage, and significantly improve the object detection performance of the model in complex scenarios. Among them, formula (2) is used to determine the loss value through the Focal Loss function:

[0133] σ = -a t ·(1 - p t ) γ ·log(p t )…… (2);

[0134] where σ is the loss value, a t is a pre-set weight, p t is the probability corresponding to the detected defect type, and γ is a pre-set type balance factor.

[0135] It should be noted that: the specific values of the weights and the type balance factor in the embodiments of the present application are preset, but the embodiments of the present application do not limit the specific values of the weights and the type balance factor herein, and the specific values of the weights and the type balance factor can be set according to specific actual situations.

[0136] Thus, the Focal Loss function in the embodiments of the present application re-weights the classification loss of the target by introducing a type balance factor. In the loss calculation of high-confidence and easily distinguishable samples, by reducing their weights, it avoids the deep learning model from overly focusing on the overly dominant easily distinguishable samples in training. In the loss calculation of low-confidence and difficult-to-distinguish samples, by increasing their training weights, it enhances the model's learning ability for key samples. This mechanism effectively balances the proportion of positive and negative samples in the training process and improves the robustness of the model in complex object detection tasks.

[0137] Next, the structure of the deep learning model in the embodiments of the present application will be introduced. The deep learning model in the embodiments of the present application adds a centralized feature pyramid EVC module to the neck module of the object detection model, and adds a convolutional block attention module CBAM to the backbone module of the object detection model. As Figure 7 shown, it is a schematic diagram of the structure of the deep learning model. As can be seen from Figure 6 it, the backbone module in the deep learning model includes a first convolutional layer 701, a first feature transformation layer 702, a second feature transformation layer 703, a third feature transformation layer 704, a fourth feature transformation layer 705, a spatial pyramid pooling SPPF layer 706, a CBAM layer 707, and a CBAM layer 708; wherein, the first convolutional layer 701, the first feature transformation layer 702, the second feature transformation layer 703, the third feature transformation layer 704, the fourth feature transformation layer 705, and the SPPF layer 706 are connected in sequence, the second feature transformation layer 703 is also connected to the CBAM layer 707, and the third feature transformation layer 704 is also connected to the CBAM layer 708;

[0138] Any one of the first feature transformation layer to the fourth feature transformation layer in the embodiments of the present application includes a convolutional layer and a feature transformation module.

[0139] The neck module includes a first upsampling layer 709, an EVC module 710, a fifth feature transformation layer 711, a second upsampling layer 712, a sixth feature transformation layer 713, a second convolutional layer 714, a seventh feature transformation layer 715, a third convolutional layer 716, and an eighth feature transformation layer 717; the first upsampling layer 709, the EVC module 710, the fifth feature transformation layer 711, the second upsampling layer 712, the sixth feature transformation layer 713, the second convolutional layer 714, the seventh feature transformation layer 715, the third convolutional layer 716, and the eighth feature transformation layer 717 are connected in sequence, the CBAM layer 708 is also connected to the fifth feature transformation layer 711, and the CBAM layer 707 is also connected to the sixth feature transformation layer 713; the fifth feature transformation layer 711 is also connected to the seventh feature transformation layer 715; the SPPF layer 706 is also connected to the first upsampling layer 709 and the eighth feature transformation layer 717 respectively.

[0140] Any one of the fifth to eighth feature transformation layers in the embodiments of the present application includes a feature fusion module and a feature transformation module.

[0141] Next, a detailed description of defect detection using the deep learning model in the embodiments of the present application will be given in combination with the structure of the deep learning model, as Figure 8 shown, which specifically may include the following steps:

[0142] Step 801: Perform a convolution operation on the first local detection image using the first convolutional layer to obtain a first intermediate feature map;

[0143] Step 802: Perform feature transformation on the first intermediate feature map in sequence using the target feature transformation layer to obtain a second intermediate feature map; wherein, the target feature transformation layer includes the first feature transformation layer, the second feature transformation layer, the third feature transformation layer, and the fourth feature transformation layer;

[0144] Step 803: Perform a pooling operation on the second intermediate feature map using the SPPF layer to obtain a first feature map;

[0145] Step 804: Perform an upsampling operation on the first feature map using the first upsampling layer to obtain a second feature map;

[0146] Step 805: Perform feature enhancement on the second feature map using the EVC module to obtain a third feature map;

[0147] Step 806: Use the feature fusion module in the fifth feature transformation layer to fuse the third feature map and the first feature transformation map to obtain a third intermediate feature map, where the first feature transformation map is obtained based on the third feature transformation module;

[0148] Step 807: Use the feature transformation module in the fifth feature transformation layer to perform feature transformation on the third intermediate feature map to obtain a fourth feature map;

[0149] Step 808: Use the second upsampling layer to perform upsampling on the fourth feature map to obtain a fifth feature map;

[0150] Step 809: Use the feature fusion module in the sixth feature transformation layer to fuse the fifth feature map and the second feature transformation map to obtain a fourth intermediate feature map; where the second feature transformation map is obtained based on the second feature transformation module;

[0151] Step 810: Use the feature transformation module in the sixth feature transformation layer to perform feature transformation on the fourth intermediate feature map to obtain a sixth feature map;

[0152] Step 811: Use the second convolutional layer to perform convolution on the sixth feature map to obtain a seventh feature map;

[0153] Step 812: Use the feature fusion module in the seventh feature transformation layer to fuse the seventh feature map and the fourth feature map to obtain a fifth intermediate feature map;

[0154] Step 813: Use the feature transformation module in the seventh feature transformation layer to perform feature transformation on the fifth intermediate feature map to obtain an eighth feature map;

[0155] Step 814: Use the third convolutional layer to perform convolution on the eighth feature map to obtain a ninth feature map;

[0156] Step 815: Use the feature fusion module in the eighth feature transformation layer to fuse the ninth feature map and the first feature map to obtain a sixth intermediate feature map;

[0157] Step 816: Use the feature transformation module in the eighth feature transformation layer to perform feature transformation on the sixth intermediate feature map to obtain the fourth intermediate detection result of the first local detection image.

[0158] In any one of the first to fourth feature transformation layers in the embodiments of the present application, a feature transformation layer includes a convolutional layer and a feature transformation module. And in any one of the fifth to eighth feature transformation layers in the embodiments of the present application, a feature transformation layer includes a feature fusion module and a feature transformation module. The feature fusion module in the embodiments of the present application is used for feature fusion, and the feature transformation module in the embodiments of the present application is used for feature transformation.

[0159] The processing flow of any one of the first to fourth feature transformation layers in the embodiments of the present application is as follows: first, perform a convolution operation using the convolutional layer, and then perform feature transformation using the feature transformation layer. And the processing flow of any one of the fifth to eighth feature transformation layers in the embodiments of the present application is as follows: first, perform feature fusion using the feature fusion module, and then perform feature transformation using the feature transformation module. The feature transformation module in the embodiments of the present application is a c2f module.

[0160] Step 204: Obtain the defect detection result of the target printed circuit board according to the defect detection results of the local detection images.

[0161] In the embodiments of the present application, the defect detection can be controlled through a terminal interface and the corresponding situation can be displayed. As Figure 9 shown, it is a schematic diagram of the terminal interface. The displayed area of this terminal interface includes four parts:

[0162] (1) Tool button area: Perform operations such as starting, ending defect detection inference, clearing AI results (requiring an administrator program), and starting the rejudgment interface.

[0163] (2) Status display area: Display the inference process of defect detection.

[0164] (3) Data analysis and panel information area: The pass rate and system operation status can be viewed.

[0165] (4) Command line display area: Mainly for developers.

[0166] The defect detection result of the target printed circuit board in the embodiments of the present application includes the defect detection results of the local detection images.

[0167] The detection image in the embodiments of the present application further includes the global detection image of the target printed circuit board. As Figure 10 shown, Figure 10 the left image in it is a schematic diagram of the global detection image, which is an overall image of the target printed circuit board.

[0168] In order to further improve the accuracy of the printed circuit board, in one embodiment, before performing step 202, pixel screening is performed on the first local measured image to obtain target pixel points; if the number of the target pixel points does not meet the specified condition, a cropped image corresponding to the first local detection area is segmented from the global detection image, and the first local measured image is replaced with the cropped image. For example, as Figure 10 shown, Figure 10 the image on the right in

[0169] is the image after cropping the global detection image. In the embodiment of the present application, the connected region analysis algorithm is used to perform connected region analysis on the first local measured image to obtain the connected regions in the first local measured image, and the number of each pixel point in the connected regions is identified to obtain the number of target pixel points.

[0170] The specified condition in the embodiment of the present application is that the number of target pixel points is greater than the specified number. The specified number in the embodiment of the present application can be set according to the specific actual situation, and the specified number in the embodiment of the present application can be limited according to the specific actual situation.

[0171] The xml file in the embodiment of the present application includes the storage location of the global detection image, and the global detection image of the target printed circuit board is obtained by using the storage location of the global detection image.

[0172] Based on the same inventive concept, the above-mentioned printed circuit board defect detection method of the present disclosure can also be implemented by a printed circuit board defect detection device. The effect of the printed circuit board defect detection device is similar to that of the foregoing method, and will not be elaborated herein.

[0173] Figure 11 It is a structural schematic diagram of a printed circuit board defect detection device according to an embodiment of the present disclosure.

[0174] As Figure 11 shown, the printed circuit board defect detection device 1100 of the present disclosure may include an acquisition module 1110, a target defect detection strategy determination module 1120, a defect detection module 1130, and a defect detection result determination module 1140.

[0175] The acquisition module 1110 is configured to acquire a detection image of a target printed circuit board sent by an automatic optical inspection (AOI) device, where the detection image includes each local detection image; and any one of the local detection images is an image corresponding to a local area of the target printed circuit board;

[0176] A target defect detection strategy determination module 1120, configured to determine a target defect detection strategy corresponding to the component type of the component in the first partial detection image by using a pre-set correspondence between the component type and the defect detection strategy, where the defect detection strategy includes a template matching strategy, a connected region analysis strategy, a color comparison strategy, and a deep learning strategy, the target defect detection strategy is at least one of the defect detection strategies, and the first partial detection image is any one of the partial detection images;

[0177] A defect detection module 1130, configured to perform defect detection on the first partial detection image by using the target defect detection strategy to obtain a defect detection result of the first partial detection image;

[0178] A defect detection result determination module 1140, configured to obtain a defect detection result of the target printed circuit board according to the defect detection results of the partial detection images.

[0179] In one embodiment, the defect detection result determination module 1140 is specifically configured to:

[0180] If the number of the target defect detection strategies is one, determine the intermediate detection result obtained by the target defect detection strategy as the defect detection result of the first partial detection image; or,

[0181] If the number of the target defect detection strategies is multiple and all the intermediate detection results obtained by the target defect detection strategies are normal, determine that the defect detection result of the first partial detection image is no defect; or,

[0182] If the number of the target defect detection strategies is multiple and there is a target intermediate detection result that identifies a defect among the intermediate detection results obtained by the target defect detection strategies, determine the target intermediate detection result as the defect detection result of the first partial detection image.

[0183] In one embodiment, each partial standard image is further included in the detection image, where any one partial standard image has a corresponding partial detection image and is the standard image of the corresponding partial detection image, and the components included in any one partial standard image are the same as the components included in its corresponding partial detection image; the defect detection module 1130 is specifically configured to:

[0184] If the target defect detection strategy includes the template matching strategy, then match the local standard image corresponding to the first local detection image with each preset component template image to obtain the target component template corresponding to the first local detection image; use the target detection algorithm to perform component detection on the first local detection image to obtain the positions of the components in the first local detection image; and obtain the first intermediate detection result of the first local detection image according to the target component template and the components in the first local detection image.

[0185] If the target defect detection strategy includes the connected component analysis strategy, then perform binarization processing on the first local detection image to obtain the binarized image of the first local detection image; use the connected component analysis algorithm to extract the solder regions in the binarized image of the first local detection image to obtain the target solder region in the first local detection image; and compare the target solder region with the standard solder region in the local standard image corresponding to the first local detection image to obtain the second intermediate detection result of the first local detection image.

[0186] In one embodiment, the defect detection module 1130 is further configured to:

[0187] If the target defect detection strategy includes the color comparison strategy, then compare the gray value of the target solder region in the first local detection image with the gray value of the standard solder region in the local standard image corresponding to the first local detection image to obtain the third intermediate detection result of the first local detection image.

[0188] In one embodiment, the defect detection module 1130 is further configured to:

[0189] If the target defect detection strategy includes the deep learning strategy, then input the first local detection image into a pre-trained deep learning model for defect detection to obtain the fourth intermediate detection result of the first local detection image.

[0190] In one embodiment, the deep learning model is a target detection model with a centralized feature pyramid EVC module added to the neck module and a convolutional block attention module CBAM added to the backbone module.

[0191] In one embodiment, the backbone module includes a first convolutional layer, a first feature transformation layer, a second feature transformation layer, a third feature transformation layer, a fourth feature transformation layer, a Spatial Pyramid Pooling (SPPF) layer, and two CBAM layers. Among them, the first convolutional layer, the first feature transformation layer, the second feature transformation layer, the third feature transformation layer, the fourth feature transformation layer, and the SPPF layer are connected in sequence. The second feature transformation layer is also connected to one of the CBAM layers, and the third feature transformation layer is also connected to the other CBAM layer.

[0192] The neck module includes a first upsampling layer, an EVC module, a fifth feature transformation layer, a second upsampling layer, a sixth feature transformation layer, a second convolutional layer, a seventh feature transformation layer, a third convolutional layer, and an eighth feature transformation layer. The first upsampling layer, the EVC module, the fifth feature transformation layer, the second upsampling layer, the sixth feature transformation layer, the second convolutional layer, the seventh feature transformation layer, the third convolutional layer, and the eighth feature transformation layer are connected in sequence. One of the CBAM layers is also connected to the fifth feature transformation layer, and the other CBAM layer is also connected to the sixth feature transformation layer. The fifth feature transformation layer is also connected to the seventh feature transformation layer. The SPPF layer is also connected to the first upsampling layer and the eighth feature transformation layer respectively.

[0193] In one embodiment, any one of the fifth to eighth feature transformation layers includes a feature fusion module and a feature transformation module.

[0194] The defect detection module 1130 is further configured to:

[0195] Perform a convolution operation on the first local detection image using the first convolutional layer to obtain a first intermediate feature map.

[0196] Perform feature transformation on the first intermediate feature map sequentially using a target feature transformation layer to obtain a second intermediate feature map. Wherein, the target feature transformation layer includes the first feature transformation layer, the second feature transformation layer, the third feature transformation layer, and the fourth feature transformation layer.

[0197] Perform a pooling operation on the second intermediate feature map using the SPPF layer to obtain a first feature map.

[0198] Perform an upsampling operation on the first feature map using the first upsampling layer to obtain a second feature map.

[0199] Perform feature enhancement on the second feature map using the EVC module to obtain a third feature map.

[0200] The feature fusion module in the fifth feature conversion layer is used to perform feature fusion on the third feature map and the first feature conversion map to obtain a third intermediate feature map, where the first feature conversion map is obtained based on the third feature conversion module;

[0201] The feature conversion module in the fifth feature conversion layer is used to perform feature conversion on the third intermediate feature map to obtain a fourth feature map;

[0202] The second upsampling layer is used to perform an upsampling operation on the fourth feature map to obtain a fifth feature map;

[0203] The feature fusion module in the sixth feature conversion layer is used to perform feature fusion on the fifth feature map and the second feature conversion map to obtain a fourth intermediate feature map; where the second feature conversion map is obtained based on the second feature conversion module;

[0204] The feature conversion module in the sixth feature conversion layer is used to perform feature conversion on the fourth intermediate feature map to obtain a sixth feature map;

[0205] The second convolutional layer is used to perform a convolution operation on the sixth feature map to obtain a seventh feature map;

[0206] The feature fusion module in the seventh feature conversion layer is used to perform feature fusion on the seventh feature map and the fourth feature map to obtain a fifth intermediate feature map;

[0207] The feature conversion module in the seventh feature conversion layer is used to perform feature conversion on the fifth intermediate feature map to obtain an eighth feature map;

[0208] The third convolutional layer is used to perform a convolution operation on the eighth feature map to obtain a ninth feature map;

[0209] The feature fusion module in the eighth feature conversion layer is used to perform feature fusion on the ninth feature map and the first feature map to obtain a sixth intermediate feature map;

[0210] The feature conversion module in the eighth feature conversion layer is used to perform feature conversion on the sixth intermediate feature map to obtain a fourth intermediate detection result of the first local detection image.

[0211] In one embodiment, the detection image further includes a global detection image of the target printed circuit board; the apparatus further includes:

[0212] The pixel screening module 1150 is configured to screen pixels of the first partial measured image to obtain target pixel points before determining a target defect detection strategy corresponding to the component type of the component in the first partial detection image by using the pre-set correspondence between the component type and the defect detection strategy.

[0213] The cropping module 1160 is configured to, if the number of the target pixel points does not meet the specified condition, crop an image corresponding to the first partial detection area from the global detection image, and replace the first partial measured image with the cropped image.

[0214] After introducing a method and apparatus for defect detection of a printed circuit board according to an exemplary embodiment of the present invention, next, an electronic device according to another exemplary embodiment of the present invention will be introduced.

[0215] Those skilled in the art can understand that various aspects of the present invention can be implemented as a system, a method, or a program product. Therefore, various aspects of the present invention can be specifically implemented in the following forms, that is: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or an implementation combining hardware and software aspects, which can be collectively referred to as "circuit", "module", or "system" here.

[0216] In some possible embodiments, the electronic device according to the present invention may at least include at least one processor and at least one computer storage medium. Wherein, the computer storage medium stores program code, and when the program code is executed by the processor, the processor executes the steps in the method for defect detection of a printed circuit board according to various exemplary embodiments of the present invention described above in this specification. For example, the processor may execute steps 201-204 as shown in Figure 2 shown in.

[0217] Next, refer to Figure 12 to describe the electronic device 1200 according to this embodiment of the present invention. Figure 12 The shown electronic device 1200 is only an example and should not impose any limitation on the functions and usage scope of the embodiments of the present invention.

[0218] As Figure 12 shown, the components of the electronic device 1200 may include but are not limited to: the above at least one processor 1201, the above at least one computer storage medium 1202, and a bus 1203 connecting different system components (including the computer storage medium 1202 and the processor 1201).

[0219] The bus 1203 represents one or more of several types of bus architectures, including a computer storage media bus or a computer storage media controller, a peripheral bus, a processor bus, or a local bus using any of the various bus architectures.

[0220] The computer storage media 1202 may include a readable medium in the form of volatile computer storage media, such as random access computer storage media (RAM) 1221 and / or cache storage media 1222, and may further include read only computer storage media (ROM) 1223.

[0221] The computer storage media 1202 may also include a program / utilities 1225 having a set (at least one) of program modules 1224. Such program modules 1224 include, but are not limited to: an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment.

[0222] The electronic device 1200 may also communicate with one or more external devices 1204 (such as a keyboard, a pointing device, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 1200, and / or may communicate with any device that enables the electronic device 1200 to communicate with one or more other AR devices (such as a router, a modem, etc.). Such communication may be through an input / output (I / O) interface 1205. And, the electronic device 1200 may also communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter 1206. As shown in the figure, the network adapter 1206 communicates with other modules for the electronic device 1200 through the bus 1203. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 1200, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0223] In some possible implementation manners, each aspect of a method for detecting defects of a printed circuit board provided by the present invention may also be implemented in the form of a program product, which includes program code. When the program product runs on a computer device, the program code is used to cause the computer device to execute the steps in the method for detecting defects of a printed circuit board according to various exemplary implementation manners of the present invention described above in this specification.

[0224] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention also intends to include these modifications and variations.

Claims

1. A method for defect detection of a printed circuit board, characterized in that, The method includes: Obtaining a detection image of a target printed circuit board sent by an automatic optical inspection (AOI) device, where the detection image includes local detection images; any one of the local detection images is an image corresponding to a local area in the target printed circuit board; Using a pre-set correspondence between component types and defect detection strategies, determining a target defect detection strategy corresponding to the component type of the component in the first local detection image, where the defect detection strategies include a template matching strategy, a connected region analysis strategy, a color comparison strategy, and a deep learning strategy, the target defect detection strategy is at least one of the defect detection strategies, and the first local detection image is any one of the local detection images; Performing defect detection on the first local detection image using the target defect detection strategy to obtain a defect detection result of the first local detection image; Obtaining a defect detection result of the target printed circuit board according to the defect detection results of the local detection images.

2. The method according to claim 1, wherein The performing defect detection on the first local detection image using the target defect detection strategy to obtain a defect detection result of the first local detection image includes: If the number of target defect detection strategies is one, determining the intermediate detection result obtained by the target defect detection strategy as the defect detection result of the first local detection image; or, If the number of target defect detection strategies is multiple and all the intermediate detection results obtained by the target defect detection strategies are normal, determining that the defect detection result of the first local detection image is no defect; or, If the number of target defect detection strategies is multiple and there is a target intermediate detection result identifying a defect among the intermediate detection results obtained by the target defect detection strategies, determining the target intermediate detection result as the defect detection result of the first local detection image.

3. The method according to claim 2, wherein The detection image further includes local standard images, where any one of the local standard images has a corresponding local detection image and is the standard image of the corresponding local detection image, and the components included in any one of the local standard images are the same as the components included in its corresponding local detection image; The performing defect detection on the first local detection image using the target defect detection strategy includes: If the target defect detection strategy includes the template matching strategy, matching the local standard image corresponding to the first local detection image with preset component template images to obtain a target component template corresponding to the first local detection image; performing component detection on the first local detection image using a target detection algorithm to obtain the positions of the components in the first local detection image; obtaining a first intermediate detection result of the first local detection image according to the target component template and the positions of the components in the first local detection image; If the target defect detection strategy includes the connected region analysis strategy, binarize the first local detection image to obtain the binary image of the first local detection image; use the connected component analysis algorithm to extract the solder region in the binary image of the first local detection image to obtain the target solder region in the first local detection image; compare the target solder region with the standard solder region in the local standard image corresponding to the first local detection image to obtain the second intermediate detection result of the first local detection image.

4. The method according to claim 3, characterized in that, The defect detection of the first local detection image using the target defect detection strategy includes: If the target defect detection strategy includes a color comparison strategy, compare the gray value of the target solder region in the first local detection image with the gray value of the standard solder region in the local standard image corresponding to the first local detection image to obtain the third intermediate detection result of the first local detection image.

5. The method according to claim 1, wherein The defect detection of the first local detection image using the target defect detection strategy includes: If the target defect detection strategy includes a deep learning strategy, input the first local detection image into a pre-trained deep learning model for defect detection to obtain the fourth intermediate detection result of the first local detection image.

6. The method according to claim 5, wherein The deep learning model adds a centralized feature pyramid EVC module to the neck module of the target detection model and adds a convolutional block attention module CBAM to the backbone module of the target detection model.

7. The method according to claim 6, wherein The backbone module includes a first convolutional layer, a first feature transformation layer, a second feature transformation layer, a third feature transformation layer, a fourth feature transformation layer, a spatial pyramid pooling SPPF layer, and two CBAM layers; among them, the first convolutional layer, the first feature transformation layer, the second feature transformation layer, the third feature transformation layer, the fourth feature transformation layer, and the SPPF layer are connected in sequence, the second feature transformation layer is also connected to one of the CBAM layers, and the third feature transformation layer is also connected to the other CBAM layer; The neck module includes a first upsampling layer, an EVC module, a fifth feature transformation layer, a second upsampling layer, a sixth feature transformation layer, a second convolutional layer, a seventh feature transformation layer, a third convolutional layer, and an eighth feature transformation layer; the first upsampling layer, the EVC module, the fifth feature transformation layer, the second upsampling layer, the sixth feature transformation layer, the second convolutional layer, the seventh feature transformation layer, the third convolutional layer, and the eighth feature transformation layer are connected in sequence, one of the CBAM layers is also connected to the fifth feature transformation layer, and the other CBAM layer is also connected to the sixth feature transformation layer; the fifth feature transformation layer is also connected to the seventh feature transformation layer; the SPPF layer is also connected to the first upsampling layer and the eighth feature transformation layer respectively.

8. The method according to claim 7, wherein Any one of the fifth to eighth feature transformation layers includes a feature fusion module and a feature transformation module; Inputting the first local detection image into a pre-trained deep learning model for defect detection to obtain a fourth intermediate detection result of the first local detection image, including: Performing a convolution operation on the first local detection image using the first convolutional layer to obtain a first intermediate feature map; Successively performing feature transformation on the first intermediate feature map using a target feature transformation layer to obtain a second intermediate feature map; wherein, the target feature transformation layer includes the first feature transformation layer, the second feature transformation layer, the third feature transformation layer, and the fourth feature transformation layer; Performing a pooling operation on the second intermediate feature map using the SPPF layer to obtain a first feature map; Performing an upsampling operation on the first feature map using the first upsampling layer to obtain a second feature map; Performing feature enhancement on the second feature map using the EVC module to obtain a third feature map; Performing feature fusion on the third feature map and a first feature transformation map using the feature fusion module in the fifth feature transformation layer to obtain a third intermediate feature map, wherein the first feature transformation map is obtained based on the third feature transformation module; Performing feature transformation on the third intermediate feature map using the feature transformation module in the fifth feature transformation layer to obtain a fourth feature map; Performing an upsampling operation on the fourth feature map using the second upsampling layer to obtain a fifth feature map; Performing feature fusion on the fifth feature map and a second feature transformation map using the feature fusion module in the sixth feature transformation layer to obtain a fourth intermediate feature map; wherein the second feature transformation map is obtained based on the second feature transformation module; Performing feature transformation on the fourth intermediate feature map using the feature transformation module in the sixth feature transformation layer to obtain a sixth feature map; Performing a convolution operation on the sixth feature map using the second convolutional layer to obtain a seventh feature map; Performing feature fusion on the seventh feature map and the fourth feature map using the feature fusion module in the seventh feature transformation layer to obtain a fifth intermediate feature map; Performing feature transformation on the fifth intermediate feature map using the feature transformation module in the seventh feature transformation layer to obtain an eighth feature map; Performing a convolution operation on the eighth feature map using the third convolutional layer to obtain a ninth feature map; Performing feature fusion on the ninth feature map and the first feature map using the feature fusion module in the eighth feature transformation layer to obtain a sixth intermediate feature map; Performing feature transformation on the sixth intermediate feature map using the feature transformation module in the eighth feature transformation layer to obtain the fourth intermediate detection result of the first local detection image.

9. The method according to claim 1, wherein The detection image further includes a global detection image of the target printed circuit board; Before determining the target defect detection strategy corresponding to the component type of the component in the first local detection image by using the pre-set correspondence between the component type and the defect detection strategy, the method further includes: Perform pixel screening on the first partial measured image to obtain target pixel points; If the number of the target pixel points does not meet the specified condition, segment a cropped image corresponding to the first partial detection area from the global detection image, and use the cropped image to replace the first partial measured image.

10. An electronic device, characterized in that, It includes a processor and a memory, and the processor and the memory are connected through a bus; A computer program is stored in the memory, and the processor is configured to perform the following operations based on the computer program: Obtain a detection image of a target printed circuit board sent by an automatic optical inspection (AOI) device, where the detection image includes each partial detection image; where any one of the partial detection images is an image corresponding to a partial area in the target printed circuit board; Use the pre-set correspondence between component types and defect detection strategies to determine a target defect detection strategy corresponding to the component type of the component in the first partial detection image, where the defect detection strategies include a template matching strategy, a connected component analysis strategy, a color comparison strategy, and a deep learning strategy, the target defect detection strategy is at least one of the defect detection strategies, and the first partial detection image is any one of the partial detection images; Use the target defect detection strategy to perform defect detection on the first partial detection image to obtain a defect detection result of the first partial detection image; Obtain a defect detection result of the target printed circuit board according to the defect detection results of each partial detection image.

Images