PCBA component defect detection method and device

Abstract

The application discloses a PCBA component defect detection method and device, and the method comprises the steps of collecting a PCBA image to be detected through an image acquisition device to obtain a component image to be detected; adopting a preset YOLO detection model to perform target detection on the component image to be detected to obtain a target component image, wherein the target component image comprises a component category and a component center coordinate; adopting a preset text recognition model to perform preset angle recognition on text characters in the target component image to obtain component text information, wherein the component text information comprises text content and a text center coordinate; and performing defect comparison detection according to reference sample information, the target component image and the component text information to obtain a component defect detection result. The application can solve the technical problem that the prior art is either too complex in operation or has low adaptability and cannot meet the requirement of PCB component defect detection.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method and apparatus for detecting defects in PCBA components. Background Technology

[0002] PCB (Printed Circuit Board), also known as a printed circuit board, is a crucial component in the electronics industry. PCBs come in two types: bare PCBs and assembled PCBs (PCBAssembly, often called PCBAs). Currently, PCBs are widely used in the manufacturing of electronic products. Therefore, quality inspection and monitoring of PCBs are extremely important to ensure their proper functioning. The quality of the PCB directly affects the performance of the assembled product, and timely detection of defects in the manufacturing process is essential for the safe and normal use of the product.

[0003] In the past, PCB defect detection relied on manual visual inspection or ICT electrical inspection to check components. However, with the continuous miniaturization of electronic devices, the demand for PCBs in mechanical manufacturing has become larger, more complex, more functional, and smaller in size. For various reasons, the above two methods are no longer suitable for the needs of modern industrial production development.

[0004] Currently, most of the more effective technologies are automatic optical inspection methods based on machine vision. However, existing automatic optical inspection methods based on machine vision still have various technical problems. For example, the related optical instruments and equipment are expensive and complex to operate. The method of calculating similarity by matching feature points is time-consuming. Moreover, some methods have low adaptability. They cannot detect misassembled components or defects in various components. Therefore, they cannot truly adapt to the current PCB component defect detection scenario in PCBA images. Summary of the Invention

[0005] This application provides a PCBA component defect detection method and apparatus to solve the technical problem that existing technologies are either too complex to operate or have low adaptability, and cannot meet the requirements for PCB component defect detection.

[0006] In view of this, the first aspect of this application provides a method for detecting defects in PCBA components, including:

[0007] Images of the PCBA to be inspected are acquired using an image acquisition device to obtain images of the components to be inspected.

[0008] A preset YOLO detection model is used to perform target detection on the image of the component to be detected, and a target component image is obtained. The target component image includes the component category and the center coordinates of the component.

[0009] A preset text recognition model is used to perform preset angle recognition on the text characters in the target component image to obtain component text information, which includes text content and text center coordinates.

[0010] Defect detection is performed based on the benchmark sample information, the target component image, and the component text information to obtain the component defect detection result.

[0011] Preferably, the step of acquiring an image of the PCBA to be inspected using an image acquisition device to obtain an image of the component to be inspected further includes:

[0012] The image of the component to be detected is subjected to preprocessing and annotation operations in sequence. The preprocessing operations include cropping, affine transformation, median filtering and sharpening.

[0013] Preferably, the step of using a preset YOLO detection model to perform target detection on the image of the component to be detected, thereby obtaining a target component image, wherein the target component image includes the component category and the component center coordinates, including:

[0014] An initial YOLO detection model is constructed based on the upsampling mechanism, the SE attention mechanism, and the small object detection mechanism. The initial YOLO detection model includes multiple SPD convolutional kernels.

[0015] The initial YOLO detection model is trained for object detection using a multi-scale training method and a weighted image training strategy to obtain a preset YOLO detection model.

[0016] A preset YOLO detection model is used to perform local target detection on discrete components in the image of the component to be detected, so as to obtain the target component image, which includes the component category and the center coordinates of the component.

[0017] Preferably, the step of using a preset YOLO detection model to perform target detection on the image of the component to be detected to obtain the target component image further includes:

[0018] Based on the image of the target component, the system judges whether the component is missing or short-circuited with solder balls, and obtains preliminary defect detection results.

[0019] Preferably, the step of using a preset text recognition model to perform preset angle recognition on the text characters in the target component image to obtain component text information includes:

[0020] Based on the improved DBNet algorithm in the pre-set text recognition model, the text characters in the target component image are detected in the same row and the same component, and the component text region image is obtained. The component text region image includes a text detection box and text center coordinates.

[0021] After adjusting the component text region image to a preset angle, the preset SVTR algorithm in the preset text recognition model is used to recognize the text characters in the component text region image to obtain component text information, which includes text content and text center coordinates.

[0022] Preferably, the improved DBNet algorithm based on a preset text recognition model performs target detection on the same line and the same component in the text characters of the target component image to obtain a component text region image, including:

[0023] The DBNet algorithm is used to perform text detection on the text characters in the target component image, resulting in multiple text detection boxes.

[0024] Based on the principle of preset grayscale fluctuation, the analysis is performed to determine whether the text detection boxes in the same row are on the same component, and the text detection boxes in the same row and on the same component are merged and sorted to obtain the component text region image.

[0025] Preferably, the step of performing defect comparison detection based on benchmark sample information, the target component image, and the component text information to obtain component defect detection results includes:

[0026] Based on defect-free PCBA images, benchmark sample information is extracted to form a benchmark sample information list;

[0027] Based on the comparison threshold, the benchmark sample information is compared with the target component image and the component text information to detect various defects, and the component defect detection results are obtained.

[0028] A second aspect of this application provides a PCBA component defect detection device, comprising:

[0029] The image acquisition unit is used to acquire images of the PCBA to be inspected through an image acquisition device, thereby obtaining images of the components to be inspected.

[0030] The target detection unit is used to perform target detection on the image of the component to be detected using a preset YOLO detection model to obtain a target component image, wherein the target component image includes the component category and the center coordinates of the component;

[0031] The text recognition unit is used to perform preset angle recognition on the text characters in the target component image using a preset text recognition model to obtain component text information, wherein the component text information includes text content and text center coordinates;

[0032] The defect detection unit is used to perform defect comparison detection based on the reference sample information, the target component image and the component text information, and obtain the component defect detection result.

[0033] Preferably, the target detection unit includes:

[0034] The model building subunit is used to build an initial YOLO detection model based on the upsampling mechanism, the SE attention mechanism, and the small object detection mechanism. The initial YOLO detection model includes multiple SPD convolutional kernels.

[0035] The model training subunit is used to train the initial YOLO detection model for object detection using a multi-scale training method and a weighted image training strategy to obtain a preset YOLO detection model.

[0036] The target detection subunit is used to perform local target detection on discrete components in the image of the component to be detected using a preset YOLO detection model, so as to obtain a target component image, which includes the component category and the center coordinates of the component.

[0037] Preferably, the text recognition unit includes:

[0038] The region detection subunit is used to perform target detection on the same line and the same component in the text characters of the target component image based on the improved DBNet algorithm in the preset text recognition model, so as to obtain the component text region image, which includes a text detection box and text center coordinates.

[0039] The text recognition subunit is used to adjust the component text region image to a preset angle, and then use the preset SVTR algorithm in the preset text recognition model to recognize the text characters in the component text region image to obtain component text information, which includes text content and text center coordinates.

[0040] As can be seen from the above technical solutions, the embodiments of this application have the following advantages:

[0041] This application provides a PCBA component defect detection method, comprising: acquiring an image of the PCBA to be inspected using an image acquisition device to obtain an image of the component to be inspected; performing target detection on the image of the component to be inspected using a preset YOLO detection model to obtain a target component image, the target component image including the component category and the component center coordinates; performing preset angle recognition on the text characters in the target component image using a preset text recognition model to obtain component text information, the component text information including the text content and the text center coordinates; and performing defect comparison detection based on benchmark sample information, the target component image, and the component text information to obtain the component defect detection result.

[0042] This application provides a PCBA component defect detection method. By analyzing and recognizing PCBA images using a detection model, it can complete component defect detection without complex instruments and equipment, making it simple and easy to operate. Furthermore, the detection process not only focuses on the component area but also identifies and analyzes the text information on the component, clearly grasping the electrical properties of each component. This allows for targeted identification of various defect types and adaptability to the defect detection of diverse components. Therefore, this application solves the technical problems of existing technologies that are either overly complex to operate or have low adaptability, failing to meet the requirements of PCB component defect detection. Attached Figure Description

[0043] Figure 1 A flowchart illustrating a PCBA component defect detection method provided in this application embodiment;

[0044] Figure 2 This is another schematic flowchart of a PCBA component defect detection method provided in an embodiment of this application;

[0045] Figure 3 This is a schematic diagram of the structure of a PCBA component defect detection device provided in an embodiment of this application;

[0046] Figure 4 Example image of the target component obtained by target detection in this application embodiment;

[0047] Figure 5 Example image 2 of the target component obtained by target detection in the embodiments of this application;

[0048] Figure 6 Example diagram of parallel text detection segmentation on the same component provided in this application embodiment;

[0049] Figure 7 Example diagram of the grayscale area between two text detection boxes on the same component provided in the embodiments of this application;

[0050] Figure 8 Example diagram of the grayscale region between two text detection boxes on different components provided in the embodiments of this application;

[0051] Figure 9 Example diagram of a text detection box on the same component provided in the embodiments of this application;

[0052] Figure 10 Example diagram of the result of merging text on the same component in the same row provided in the embodiments of this application;

[0053] Figure 11 Example diagrams comparing threshold segmentation before and after, provided in embodiments of this application;

[0054] Figure 12 Example diagram of multi-line text analysis on the same component provided in this application embodiment;

[0055] Figure 13 A structural example diagram of a PCBA component defect detection system provided as an application example of this application. Detailed Implementation

[0056] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0057] For easier understanding, please refer to Figure 1 This application provides an embodiment of a PCBA component defect detection method, which includes:

[0058] Step 101: Acquire images of the PCBA to be inspected using an image acquisition device to obtain images of the components to be inspected.

[0059] The image acquisition device can be selected according to the actual situation and is not limited here. This embodiment uses a PCBA image acquisition system, which mainly includes an industrial camera, an industrial lens, a light source, an image acquisition bracket, and a computer. It uses a Hikvision MV-CH120-10GM industrial camera and an MVL-KF1228-12MP industrial lens. The camera uses a CMOS sensor, has a fast frame rate of 9.4fps, which meets the real-time requirements, and a high resolution of 4096×3000, which is suitable for photographing components with a minimum size of 0.5cm×1cm in this study, meeting the high precision requirements. The interface is GigE, which can be connected to the camera by plugging in a network cable, which is convenient and quick. It is equipped with an MVL-KF1228-12MP industrial lens, and the light source is a ring light source, which can effectively improve the brightness of the image and meet the hardware requirements of this invention. The computer is equipped with a GPU3080TI and 12G of RAM.

[0060] The MVS software allows you to directly view images of PCBA components. You can adjust the object distance, resolution, light intensity, affine transformation coefficient, etc., to ensure that the text on the components is clearly captured without overexposure, resulting in a 4090×3000 resolution image, which is the image of the component to be inspected.

[0061] Step 102: Use a preset YOLO detection model to perform target detection on the image of the component to be detected, and obtain the target component image. The target component image includes the component category and the center coordinates of the component.

[0062] The pre-set YOLO detection model in this embodiment can define the bounding box of PCB components in the image of the components to be detected and provide category labeling for the bounding target components. Selecting the center coordinates of the components for analysis during target detection is also to reduce complexity; compared to using the coordinates of the four corner points of the detection box, using the center coordinates can reduce the risk of detection errors to some extent. Furthermore, using the detection model to directly detect the target is simple and easy to execute, requiring no complex instruments or equipment, which can also reduce instrument investment costs to some extent.

[0063] Step 103: Use a preset text recognition model to perform preset angle recognition on the text characters in the target component image to obtain the component text information, which includes the text content and the text center coordinates.

[0064] The pre-built text recognition model is mainly used to identify text characters on components in target component images. Components typically have attribute data or model information marked on them, and the text content varies significantly between different PCB components. This embodiment not only detects specific component areas but also analyzes and processes the text information on the components. This is because the text character information of different components can determine the component's category or location. Based on this, it can detect whether the component is mounted backwards, missing, incorrectly assembled, has short-circuited solder balls, or incorrect polarity. In other words, the method in this embodiment is applicable to more component defect detection scenarios and has higher adaptability.

[0065] Step 104: Perform defect comparison detection based on the benchmark sample information, target component image and component text information to obtain component defect detection results.

[0066] It is understood that the benchmark sample information consists of relevant information about normal components in a standard PCBA image without defects. This information type matches or corresponds to the target component image and component text information studied in this embodiment, enabling defect detection through information comparison, thereby obtaining component defect detection results. The benchmark sample information can be obtained using the method mentioned above in this embodiment, or it can be obtained through other means, as long as the information is corresponding and can be reliably compared. No limitation is made here.

[0067] This application provides a PCBA component defect detection method that analyzes and processes PCBA images using a detection model. It eliminates the need for complex instruments and equipment, making it simple and easy to operate. Furthermore, the detection process not only focuses on the component area but also identifies and analyzes the text information on the component, clearly grasping the electrical properties of each component. This allows for targeted identification of various defect types and adaptability to the defect detection of diverse components. Therefore, this application solves the technical problems of existing technologies that are either overly complex or have low adaptability, failing to meet the requirements for PCB component defect detection.

[0068] For easier understanding, please refer to Figure 2 This application provides a second embodiment of a PCBA component defect detection method, including:

[0069] Step 201: Acquire images of the PCBA to be inspected using an image acquisition device to obtain images of the components to be inspected.

[0070] Step 202: Perform preprocessing and annotation operations on the image of the component to be inspected in sequence. The preprocessing operations include cropping, affine transformation, median filtering, and sharpening.

[0071] The preprocessing aims to achieve three main goals: first, to adjust the image size for easier processing and analysis by the model; second, to augment the data; and third, to improve image quality. For example, the cropping operation summarized in this embodiment unifies the image size, while the affine transformation is used to augment the image dataset. It is understood that augmenting the image dataset is primarily used in the model training phase, not in the detection of a single target image.

[0072] The 4090×3000 resolution images of the acquired PCBA components were cropped to the PCBA detection region of interest. The cropped image resolution was a multiple of 32, and the images were cropped to approximately 640×640. At the same time, the components in the cropped images were kept as complete as possible, as incomplete labels would affect the detection accuracy. A dataset of 6362 images was compiled.

[0073] In this embodiment, the affine transformation mainly focuses on dataset expansion. Since the PCBA samples in this embodiment are class-imbaled, a series of preprocessing techniques are used to appropriately expand the dataset, facilitating subsequent model training. The main methods include horizontal flipping, probability-based vertical flipping, random cropping, and random rotation. Other methods can also be added, such as standardization and adding three-channel images, but these are not limited here.

[0074] PCBA components are characterized by their small size, high density, and large quantity compared to other inspection objects. For example, a PCB board with an area of ​​40×60cm may contain hundreds of densely packed components, with some small components having an area of ​​only 0.5cm×0.5cm. Furthermore, this embodiment requires extracting text information for component defect detection. To improve detection reliability, the quality of the PCBA image must be guaranteed. This embodiment selects preprocessing methods to improve image quality, including contrast enhancement, sharpening, black and white contrast adjustment, median filtering, and gamma transform. Other preprocessing methods can be added as needed, and are not limited here.

[0075] The dataset includes data annotation for images of components to be inspected, covering defects such as PCB components, shorted solder balls, and missing components. It currently identifies up to 25 categories, including capacitors, resistors, chips, diodes, test points, jumpers, inductors, transistors, transformers, surface-mount diodes, tantalum electrolytic capacitors, aluminum electrolytic capacitors, cement resistors, metal film resistors, test points, chip inductors, series rectifier diodes, safety capacitors, Schottky diodes, ceramic capacitors, fuses, sockets, stacked inductors, and two defects: shorted solder balls and missing components. The dataset contains 6362 images with a total of 58480 labels. It's worth noting that annotations can include not only component categories and defect types, but also directional annotations, i.e., the angle at which components or text characters are distributed in the image, such as 0 degrees, 90 degrees, 180 degrees, and 270 degrees. A dataset with directional annotations can also be created using these as a reference.

[0076] During the model training phase, a large number of component images need to be processed, and the dataset needs to be divided into training, validation, and test sets. This embodiment provides a 6:2:2 ratio for this division, which is only a reference; other ratios can be designed based on actual needs. Furthermore, affine transformation data augmentation may not be suitable for datasets formed by orientation annotations, requiring other preprocessing methods, which are not specified here.

[0077] Step 203: Construct an initial YOLO detection model based on the upsampling mechanism, SE attention mechanism, and small object detection mechanism. The initial YOLO detection model includes multiple SPD convolutional kernels.

[0078] Step 204: Use multi-scale training and weighted image training strategies to train the initial YOLO detection model for object detection, and obtain the preset YOLO detection model.

[0079] Step 205: Use a preset YOLO detection model to perform local target detection on discrete components in the image of the component to be detected, and obtain the target component image, which includes the component category and the center coordinates of the component.

[0080] This embodiment improves upon the original YOLOv7 detection model. The small object detection layer, formed by the small object detection mechanism, allows for the concatenation of shallow and deep feature maps. Furthermore, the anchor size of the small object detection layer can be used to calculate the average size of small object candidate boxes, thus obtaining the overall size. Because the downsampling factor in the original YOLOv7 model is relatively large, the feature maps struggle to learn the feature information of small objects, making it difficult to accurately detect small components in PCBA images. Therefore, a small object detection layer is added for adjustment and improvement.

[0081] Adding an upsampling mechanism is used to avoid the problems caused by the downsampling mechanism in the original model. Overly rapid downsampling can lead to the loss of information about small targets, which upsampling can compensate for. Using an SPD convolutional kernel instead of nearest neighbor interpolation can learn to obtain the optimal upsampling method. Furthermore, the SE attention mechanism can make reasonable use of limited visual information processing resources, focusing on specific feature information and improving detection accuracy.

[0082] Multi-scale training and a weighted image strategy are introduced. Multi-scale training involves setting several images at different scales, randomly selecting one scale for training at regular intervals. During testing, several feature maps at different scales are generated, and each candidate box also has different scales on different feature maps. The candidate box closest to a fixed size (i.e., the input size of the detection head) is selected as the subsequent input. When the network downsamples, it often generates feature maps tens of times smaller than the original image, making it difficult for the detection network to capture the features of small objects. By training with larger and more varied input images through multi-scale training, the robustness of the detection model to object size can be improved to some extent.

[0083] The weighted image strategy is primarily used to address class imbalance. For example, the component dataset in this paper contains 25 categories, but the differences between these categories are significant. Training might cause the network weights to favor the more numerous categories, affecting the accuracy of the smaller categories. The principle is to perform class statistics on the images in the dataset; the more categories there are, the smaller their weights. The weights of the entire dataset are then used to sum the class weights of each image. In other words, the order in which images are used is determined by the class weight of each image and the weights used for sampling.

[0084] Local detection replaces global detection by acquiring local images of components instead of the global image. For example, a 4096×3000 resolution image can be divided into multiple local images of components, improving the low accuracy of small target detection caused by the large difference between the resolution of the global detection image and the resolution of small target components. Local detection only acquires images containing components, not the entire PCBA, excluding background and components outside the dataset to avoid irrelevant data affecting the recognition effect. Furthermore, considering that local image stitching can lead to disordered and blurred edge information, ghosting phenomena, and low accuracy, and that stitching large images is computationally intensive and time-consuming, image quality analysis and maximum suppression are also necessary.

[0085] Furthermore, this embodiment replaces the Conv with a stride of 2 in the Head module of the pre-built YOLO detection model with SPD-Conv. Traditional strided convolutional layers or pooling layers can lead to the loss of fine-grained information and insufficient feature learning, while SPD-Conv can achieve stridless convolution, thus not losing learnable information during downsampling. CBS represents the improved convolutional module introduced in YOLOv7. It consists of a Conv convolutional layer, a batch normalization layer, and a Silu activation function layer. The SPD-Conv module consists of an SPD layer and a non-strided convolutional layer. For the target component image obtained by using the pre-built YOLO detection model in this embodiment, please refer to [link to relevant documentation]. Figure 4 and Figure 5 .

[0086] Step 206: Based on the target component image, determine the missing component and short circuit of solder balls to obtain preliminary defect detection results.

[0087] Please see Figure 4 and Figure 5 Based on the images of the target components obtained from the inspection, we can initially determine the problems of missing components and short circuits of solder balls, that is, the components are not soldered to the board, and the solder ball soldering points are bulging or sticking, resulting in short circuits, etc. Other defects cannot be detected at present, so this is a preliminary defect detection result.

[0088] Step 207: Based on the improved DBNet algorithm in the preset text recognition model, perform target detection on the same line and the same component in the text characters in the target component image to obtain the component text region image, which includes the text detection box and the text center coordinates.

[0089] Further, step 207 includes:

[0090] The DBNet algorithm is used to detect text characters in the target component image, resulting in multiple text detection boxes.

[0091] Based on the principle of preset grayscale fluctuation, we analyze whether the text detection boxes in the same row are on the same component, and merge and organize the text detection boxes in the same row and on the same component to obtain the component text region image.

[0092] The original DBNet algorithm, when performing object detection, treats text in the same row or column but with slightly larger spacing as two segments. Some text can only be detected as one segment. For details on detecting text as two segments, please refer to [link / reference needed]. Figure 6This would directly change the text center coordinates, leading to errors in subsequent comparison and detection. Therefore, this embodiment improves the DBNet algorithm by determining whether the detected text is in the same row or column, and whether different text detection boxes are located on the same component. If they are not on the same component, they do not need to be merged; otherwise, they are merged to form a complete text detection box in the same row or column. Specifically, determining whether different text detection boxes are on the same component can be based on the principle that there will be no obvious grayscale fluctuations between text on the same component, while there will inevitably be relatively obvious grayscale fluctuations between text on different components.

[0093] The grayscale fluctuation determination method mainly includes: First, obtaining the center coordinates of all text detection boxes, and setting two fixed threshold values ​​σ. x and σ y Then, if the ordinates or abscissas of two different texts are the same, and the absolute value of the difference between their ordinates is less than at least one of two thresholds, then image cropping is performed based on the coordinates to obtain the image region between the two texts. Please refer to [link to relevant documentation]. Figure 7 and 8 Finally, a threshold calculation is performed on the cropped image region. If the grayscale distribution of the calculated image is uniform, it is determined that the two texts belong to the same component, and the two text detection boxes can be merged; otherwise, it means that the two texts are not on the same component.

[0094] The component text region image detection process in this embodiment can be illustrated with an example; please refer to [link / reference]. Figure 6 The component is marked A. The "P", "CS24", "16", and "48" on the same row of component A are all divided into two segments during testing; please refer to [link / reference]. Figure 9 On component B, all text in the same row was successfully detected. This is due to the unequal spacing of the text in the same row on the component. Similar situations could also be caused by other factors, such as ambient lighting. Regardless, this ultimately leads to a significant difference in the text center coordinates. The improved method presented in this embodiment can determine the grayscale area between the text "P" and "CS24," allowing the two text detection boxes to be merged to obtain... Figure 10 The detection results are shown.

[0095] Step 208: After adjusting the component text region image to a preset angle, the preset SVTR algorithm in the preset text recognition model is used to recognize the text characters in the component text region image to obtain component text information, which includes text content and text center coordinates.

[0096] The text area images of components may be distributed at different angles. To facilitate subsequent comparison and detection, this embodiment adjusts all text to the same preset angle, and does not process text already at the preset angle. The preset angle can be selected according to the actual situation. This embodiment provides an example where the preset angle is 0 degrees.

[0097] The pre-built SVTR algorithm replaces the Transformers structure with an RNN structure, which can more effectively mine the contextual information of text images and improve recognition accuracy. Understandably, all component text region images, in addition to requiring pre-built angle adjustments, also need to be padded to ensure uniform size for easy model recognition.

[0098] The text content can reflect the specific model or attribute information of the component. Based on this information, the connection requirements and location requirements of the component can be clarified, so more types of component defects can be detected.

[0099] The dataset formed by text region images can be divided into text angle dataset and text dataset according to the angle processing process. Furthermore, some text image processing operations can be performed on these datasets to improve the reliability of text recognition, such as scaling or threshold segmentation.

[0100] The scaling method mainly involves: 1. Calculating the scaling ratios of the image's length and width to the target size, finding the smallest scaling factor; 2. Scaling the image according to the smallest scaling factor found in the previous step; 3. Padding the scaled image to the target size. The image size is (640, 360), while the network input size is specified as (192, 192). First, calculate the length and width scaling ratios: 192 ÷ 640 = 0.3; 192 ÷ 360 = 0.533. Therefore, 0.3 is chosen as the scaling ratio and multiplied by the original image's length and width, resulting in an image scaled to (192, 108). The next step is to pad 108 to 128 so that it is divisible by 32, obtaining the final image size (192, 128). It should be noted that although the image input size meets the model's input requirements after adaptive padding, gray edges will appear. These gray edges are invalid areas, affecting not only the algorithm's processing speed but also the detection accuracy. Therefore, this embodiment improves the above padding algorithm by: 1. Calculating the scaling ratios of the image's length and width to the target size, and finding the minimum scaling factor; 2. Scaling the image according to the minimum scaling factor from the previous step; 3. For pixel values ​​that are not multiples of 32, if the value reduced to a multiple of 32 is less than the value increased to a multiple of 32, then the edges are uniformly cropped to the target pixel; otherwise, padding is applied to the target pixel. The original image size is 640×360. After scaling it to (192, 108), 108 is not a multiple of 32. Therefore, the pixel closest to a multiple of 32 is calculated to be 96, requiring only 12 pixels. Padding 108 to 128 requires 20 pixels. Therefore, the image is uniformly cropped to (192, 96). This improved algorithm removes some irrelevant information, allowing the main information to occupy a larger proportion of the image, which helps improve the algorithm's feature extraction capability.

[0101] Thresholding segmentation aims to maximize the grayscale difference between text and background, enabling the text orientation classifier to better capture text edge information. This embodiment selects a global thresholding segmentation algorithm. To avoid inaccurate segmentation points selected by global thresholding, please refer to [link to relevant documentation]. Figure 11 The Otsu thresholding algorithm is used. The principle is to traverse different thresholds according to the distribution of gray values ​​on the image, calculate the intra-class variance between the background and the foreground under different thresholds, and the threshold corresponding to the maximum value of the intra-class variance is the optimal segmentation threshold.

[0102] Step 209: Extract benchmark sample information based on defect-free PCBA images to form a benchmark sample information list.

[0103] The benchmark sample information in this embodiment, also known as the gold standard sample information, serves as a comparison benchmark. It is information extracted from defect-free PCBA images that corresponds to the information in this embodiment. It can be compared with the sample information to be tested using coordinate standards to detect component defects. The benchmark information list is updated with information content and categories according to the comparison and detection requirements.

[0104] Step 210: Based on the comparison threshold, the benchmark sample information is compared with the target component image and component text information to detect various defects, and the component defect detection results are obtained.

[0105] This embodiment uses multiple comparison thresholds, including a confidence threshold, a threshold for matching the center coordinates of normal components and the component under test, a threshold for matching the center coordinates of the component and the center coordinates of the text, and a text threshold. In addition, it includes three main recognition parameters: component category, text angle judgment, and text content judgment.

[0106] The information of the PCBA board under test, consisting of images and text information of the target components, is compared with the information of the benchmark sample. If any mismatch occurs during the step-by-step comparison, the result is directly obtained, and the process terminates. For example, if the category does not match in component category analysis, it is determined that the component is incorrectly assembled, and no further testing is required. The specific judgment is as follows:

[0107] 1. Compare whether the confidence level of the component under test is less than the threshold. If it is less, continue with the subsequent judgment. If it is greater, the data is determined to be invalid and the process is terminated.

[0108] 2. Compare the difference between the center coordinates of the component under test and the center coordinates of the normal component. If the difference is less than the threshold, continue with the subsequent judgment. If the difference is greater than or equal to the threshold, the data is deemed invalid and the process is terminated.

[0109] 3. Determine if the component categories are consistent. If they are inconsistent, the components are incorrectly installed.

[0110] 4. Determine if the component has text. If there is no text, proceed to point 5; otherwise, proceed to point 6.

[0111] 5. Determine whether both the component under test and the reference component at the same location have no text; otherwise, the defect type is component misassembly.

[0112] 5. If there is text, determine whether the difference between the center coordinates of the text and the center coordinates of the component is less than a threshold. If it is greater than or equal to the threshold, the data is deemed invalid and the process is terminated.

[0113] 6. Determine if the text angles of the components are consistent. If they are inconsistent, the components are determined to be reversed. If they are consistent, proceed to point 7.

[0114] 7. Determine if the text content of the component is less than a text threshold. For example, if the threshold is set to 80%, the normal text is "12345", and the defective text is detected as "12346", which meets the condition of being greater than or equal to the threshold, indicating that the component is correct. If it is less than the threshold, it proves that the component is not defective; if it is greater than or equal to the threshold, it is determined that the component is misassembled.

[0115] It should be noted that, for the above coordinate comparison and analysis process, this embodiment provides a specific text coordinate comparison and analysis scheme, which is described in detail as follows:

[0116] 1) Count the text coordinates of all components and the number of components. Let the total number of components be N.

[0117] 2) Let T be the number of single-line text components and components without text;

[0118] 3) Because single-line text can be directly matched with components, a dataset n is formed by filtering the text coordinates of multiple-line text components from all available component text coordinate information;

[0119] 4) Use the K-means++ clustering algorithm to calculate the number K of components with multi-line text. The number of components with multi-line text is equal to the total number of components N minus the number of components with single-line text and no text T. Therefore, K equals NT.

[0120] 5) Select K initial cluster centers C from the dataset n. i (i≤1≤k), and the sample points that are farther away from other cluster centers are more likely to be selected as the next cluster center. Calculate the remainder of the data objects and the cluster center C. i Using Euclidean distance, find the cluster center C that is closest to the target data object. i And assign the data objects to cluster center C. i Within the corresponding clusters, the average value of the data objects in each cluster is then calculated as the new cluster center. This process is repeated until the cluster centers no longer change or the maximum number of iterations is reached.

[0121] 6) By calculating and obtaining K multi-line text clusters, the clusters can be matched with the coordinates of the components.

[0122] It should be noted that for examples of multi-line text, please refer to [link / reference]. Figure 12The "1002" on the resistor is a single-line text, and its center coordinates correspond to the center coordinates of the resistor. However, the "106", "16K", and "847" on the tantalum electrolytic capacitor are multi-line texts and cannot be directly correlated with the center coordinates of the tantalum electrolytic capacitor. The K-means clustering algorithm in this embodiment is used to detect whether the multi-line text is on the component. Through analysis, it is found that "106", "16K", and "847" correspond to the center coordinates of the tantalum electrolytic capacitor, that is, these texts are located on the same component.

[0123] This embodiment primarily focuses on defect detection for components with text characters. If the component does not have text characters, such as an aluminum electrolytic capacitor (which can be categorized as having text or no text), even though different specifications of aluminum electrolytic capacitors have different designs, they generally use a small, differently colored area on the surface to represent the negative electrode. Therefore, polarity detection can be performed to determine if the component is installed incorrectly. In this embodiment, if the component is identified as an aluminum electrolytic capacitor without text during the comparative detection phase, an image is captured based on the PCBA component target detection results. The image is then thresholded and segmented, and an affine transformation is performed to improve the skewness during the capture. Then, feature points are calculated simultaneously on both the test and reference sample images, and a feature point matching score is calculated. If the score is higher than a preset value, the component is considered correctly mounted; otherwise, it can be inferred that the aluminum electrolytic capacitor is installed incorrectly.

[0124] Understandably, for components such as capacitors, chips, diodes, test points, jumpers, inductors, transistors, transformers, cement resistors, metal film resistors, test points, chip inductors, series rectifier diodes, safety capacitors, Schottky diodes, ceramic capacitors, fuses, sockets, and stacked inductors, which have no polarity requirements and no text, the problem of incorrect component assembly can be directly determined by the test category.

[0125] For ease of understanding, please refer to the section on applying the method in this embodiment to a real system. Figure 13 It mainly includes an image acquisition module, a light source illumination module, a motion control module, and a computer design module. The image acquisition module acquires images of the moving PCBA and transmits the images to the computer for image processing to obtain the PCB component defect detection results.

[0126] For easier understanding, please refer to Figure 3 This application also provides an embodiment of a PCBA component defect detection device, comprising:

[0127] The image acquisition unit 301 is used to acquire an image of the PCBA to be inspected through an image acquisition device to obtain an image of the component to be inspected.

[0128] The target detection unit 302 is used to perform target detection on the image of the component to be detected using a preset YOLO detection model, and obtain the target component image, which includes the component category and the center coordinates of the component.

[0129] The text recognition unit 303 is used to perform preset angle recognition on the text characters in the target component image using a preset text recognition model to obtain component text information, which includes text content and text center coordinates.

[0130] The defect detection unit 304 is used to perform defect comparison detection based on the reference sample information, the target component image and the component text information, and obtain the component defect detection result.

[0131] Furthermore, the target detection unit 302 includes:

[0132] Model building subunit 3021 is used to build an initial YOLO detection model based on the upsampling mechanism, SE attention mechanism and small object detection mechanism. The initial YOLO detection model includes multiple SPD convolutional kernels.

[0133] The model training subunit 3022 is used to train the initial YOLO detection model for object detection using a multi-scale training method and a weighted image training strategy to obtain a preset YOLO detection model.

[0134] The target detection subunit 3023 is used to perform local target detection on discrete components in the image of the component to be detected using a preset YOLO detection model, so as to obtain the target component image, which includes the component category and the center coordinates of the component.

[0135] Furthermore, the text recognition unit 303 includes:

[0136] The region detection subunit 3031 is used to perform target detection on the same line and the same component in the text characters of the target component image based on the improved DBNet algorithm in the preset text recognition model, and to obtain the component text region image, which includes the text detection box and the text center coordinates.

[0137] The text recognition subunit 3032 is used to adjust the component text region image to a preset angle, and then use the preset SVTR algorithm in the preset text recognition model to recognize the text characters in the component text region image to obtain component text information, which includes text content and text center coordinates.

[0138] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0139] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0140] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0141] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for executing all or part of the steps of the methods described in the various embodiments of this application through a computer device (which may be a personal computer, server, or network device, etc.). The aforementioned storage medium includes: USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, optical disks, and other media capable of storing program code.

[0142] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A PCBA component defect detection method, characterized in that, The application relates to a PCBA (Printed Circuit Board Assembly) defect detection method and device. An image acquisition device is used to acquire a PCBA image to be detected, and a component image to be detected is obtained; An initial YOLO detection model is constructed based on an up-sampling mechanism, an SE attention mechanism and a small target detection mechanism, wherein the initial YOLO detection model comprises a plurality of SPD convolution kernels; A preset YOLO detection model is obtained by using a multi-scale training method and a weighted image training strategy to train the initial YOLO detection model; The preset YOLO detection model is used to perform local target detection on discrete components in the component image to be detected, and a target component image is obtained; The preset text recognition model is used to perform preset angle recognition on text characters in the target component image, and component text information is obtained, wherein the component text information comprises text content and a text center coordinate, and the specific process is as follows: An improved DBNet algorithm in the preset text recognition model is used to perform target detection on text characters in the target component image in the same row and the same component, and a component text region image is obtained, wherein the component text region image comprises a text detection frame and a text center coordinate, and the specific process is as follows: A DBNet algorithm is used to perform text detection on text characters in the target component image, and a plurality of text detection frames are obtained; Whether the text detection frames in the same row are on the same component is analyzed based on a preset gray scale fluctuation principle, and the text detection frames in the same row and the same component are merged and arranged to obtain a component text region image; After the component text region image is adjusted to a preset angle, a preset SVTR algorithm in the preset text recognition model is used to recognize text characters in the component text region image, and component text information is obtained, wherein the component text information comprises text content and a text center coordinate; Defect comparison detection is performed according to reference sample information, the target component image and the component text information, and a component defect detection result is obtained. The PCBA image to be detected is acquired by using the image acquisition device, and a component image to be detected is obtained, and the method further comprises the following steps:

2. The PCBA component defect detection method of claim 1, wherein, The component image to be detected is sequentially subjected to a pretreatment operation and a labeling operation, and the pretreatment operation comprises cropping, affine transformation, median filtering and sharpening. The preset YOLO detection model is used to perform target detection on the component image to be detected, and a target component image is obtained, and the method further comprises the following steps:

3. The PCBA component defect detection method of claim 1, wherein, Component missing and tin bead short-circuit judgment are performed according to the target component image, and a preliminary defect detection result is obtained. The reference sample information is extracted based on a defect-free PCBA image, and a reference sample information list is formed.

4. The PCBA component defect detection method of claim 1, wherein, ​ ​ The reference sample information is respectively compared with the target component image and the component text information according to a comparison threshold, and component defect detection results are obtained.

5. A PCBA component defect detection device, characterized in that, The method comprises the steps of: An image acquisition unit is configured to acquire a PCBA image to be detected by an image acquisition device to obtain a component image to be detected; A target detection unit is configured to perform target detection on the component image to be detected by using a preset YOLO detection model to obtain a target component image, wherein the target component image comprises a component category and a component center coordinate. A model construction subunit is configured to construct an initial YOLO detection model based on an up-sampling mechanism, an SE attention mechanism, and a small target detection mechanism, wherein the initial YOLO detection model comprises a plurality of SPD convolution kernels; A model training subunit is configured to perform target detection training on the initial YOLO detection model by using a multi-scale training method and a weighted image training strategy to obtain a preset YOLO detection model; A target detection subunit is configured to perform local target detection on discrete components in the component image to be detected by using the preset YOLO detection model to obtain a target component image; A text recognition unit is configured to perform preset angle recognition on text characters in the target component image by using a preset text recognition model to obtain component text information, wherein the component text information comprises text content and a text center coordinate. A region detection subunit is configured to perform target detection on text characters in the target component image by using an improved DBNet algorithm in the preset text recognition model to obtain a component text region image, wherein the component text region image comprises a text detection box and a text center coordinate. A text recognition subunit is configured to perform recognition on text characters in the component text region image by using a preset SVTR algorithm in the preset text recognition model after adjusting the component text region image to a preset angle to obtain component text information, wherein the component text information comprises text content and a text center coordinate. A defect detection unit is configured to perform defect comparison detection according to reference sample information, the target component image, and the component text information to obtain component defect detection results. ​ ​

Images