Component defect detection method and system, and storage medium

Abstract

The present invention relates to the technical field of surface mount equipment, and specifically discloses a component defect detection method and system, and a storage medium, wherein the method comprises the following steps: constructing a template image according to standard component parameter information; identifying the image of the component to be tested and correcting the posture, and obtaining an ROI area according to the constructed template image; correcting the ROI area according to the template image to eliminate the offset of the pins in the ROI area; and comparing the pin information in the ROI area with the pin information in the template image to determine the component defects. The scheme does not need to use the actual captured image, but directly uses the structural information and camera information of the standard component to construct the template image, and has stronger robustness; the ROI area is extracted, located and corrected based on the template image for the image of the component to be tested, and the calculation is simple and efficient, and the result is more stable.

Description

Technical Field

[0001] The present invention relates to the technical field of surface mount equipment, and in particular to a component defect detection method and system, and a storage medium. Background Art

[0002] In surface mount technology, the placement machine, as the core production equipment, is a device that accurately places surface mount components of various shapes and specifications on the PCB pads by moving the placement head. Component defect recognition and detection usually adopts component defect detection based on deep learning or traditional manual defect detection methods. For the traditional manual defect detection method, the human eye is used to observe component photos for screening. This method is not only inefficient, but also has unsatisfactory detection results. For the component defect detection method based on deep learning, the training model is usually large, the memory usage is high, and the hardware environment requirements are high, resulting in a long final detection time and high cost, which is difficult to meet the scenarios in real life that have high requirements for detection speed.

[0003] Therefore, existing component defect identification and detection lacks automated detection technology with fast detection speed and low cost. Summary of the invention

[0004] The purpose of the present invention is to overcome the problems existing in the prior art and to provide a component defect detection method and system, and a storage medium.

[0005] In order to achieve the above object, the present invention provides a component defect detection method in a first aspect, comprising the following steps:

[0006] Constructing a template image according to standard component parameter information;

[0007] Identify the image of the component to be tested and correct its posture, and obtain the ROI area based on the template image;

[0008] Correct the ROI area according to the template image to eliminate the offset of the pins in the ROI area;

[0009] Compare the pin information in the ROI area with the pin information in the template image to determine component defects.

[0010] Preferably, the step of obtaining the ROI region according to the template image is as follows: segmenting the image according to the size of the template image, and extracting the common part with the template image from the segmented image.

[0011] Preferably, the ROI area correction according to the template image is specifically:

[0012] Divide the component pins in the ROI area to obtain the position information of the pins on the image;

[0013] Calculate the slope of the straight line formed by the center position of the pin outline point set on each side of the component respectively, and determine whether correction is needed based on the slope of the straight line. If necessary, adjust the rotation angle of the ROI area so that the slope of each straight line meets the requirements.

[0014] Preferably, judging whether correction is required based on the slope of the straight line, and if necessary, adjusting the rotation angle of the ROI area so that the slope of each straight line meets the requirement is specifically as follows:

[0015] It is determined whether the difference between the slope of the straight line and the slope of the side corresponding to the template image is less than a preset angle. If not, the difference is used as the rotation angle of the ROI area for correction.

[0016] Preferably, before respectively calculating the slope of the straight line formed by the center positions of the pin contour point sets on each side of the component, the pin contours whose edge contour point sets are less than a preset value are first removed.

[0017] Preferably, the standard component parameter information includes the pin type, pin number, pin distribution and pin size of the component.

[0018] Preferably, the comparing the pin information in the ROI area with the pin information in the template image to determine the component defect is specifically:

[0019] Compare the number of pins in the ROI area with the number of pins in the template image to see if they are consistent. If they are inconsistent, calculate the defect position information;

[0020] If they are consistent, the result is that there is no pin shortage, and then it is determined based on the pin position information whether there is pin offset, and whether there is pin warping or bending.

[0021] Preferably, the calculating defect location information is specifically:

[0022] The position distance value between adjacent pins on each side of the template image is used as the standard deviation value, and the center position of each pin on the side where the pin defect exists in the ROI area is calculated. The position difference of adjacent pins in the ROI area is calculated based on the pin center position, and each position difference is compared with the corresponding standard deviation value to obtain the defect position information.

[0023] Preferably, judging whether there is pin offset according to the pin position information is specifically:

[0024] The position distance value between adjacent pins on each side of the template image is used as the standard deviation value, and the position difference value between adjacent pins on each side of the ROI area is calculated. The position difference value is compared with the corresponding standard deviation value to see if it is within the error range. If it is not within the error range, it is determined that there is an offset between the corresponding adjacent pins.

[0025] Preferably, judging whether there is any pin warping or bending according to the pin position information is specifically as follows: calculating the acceptable fluctuation range of the pin warping or bending according to the pin length information of the template image, calculating the pin length information of each side of the ROI area, and calculating whether each pin is warped or bent according to the acceptable fluctuation range of the corresponding pin on the template image.

[0026] Preferably, the identification of the image of the component to be tested and correction of the posture are specifically: using a camera to obtain the image of the component to be tested, grayscale processing, detecting the posture information of the component to be tested, including the center point position, the minimum circumscribed rectangle size and the rotation angle information, and correcting the image of the component to be tested based on the template image.

[0027] A second aspect of the present invention provides a component defect detection system, comprising:

[0028] A template image module is used to construct a template image according to standard component parameter information;

[0029] ROI area module, used to identify the image of the component to be tested and correct the posture, and obtain the ROI area according to the template image;

[0030] The ROI area correction module is used to correct the ROI area according to the template image to eliminate the offset of the pins in the ROI area;

[0031] The defect detection module is used to compare the pin information in the ROI area with the pin information in the template image to determine the component defects.

[0032] A third aspect of the present invention provides a computer-readable storage medium, comprising a program, wherein the program can be executed by a processor to implement the steps of the above method.

[0033] Through the above technical solution, there is no need to use the actual captured images, and the structural information and camera information of the standard components can be directly used to construct the template image, which is more robust. The template image is used as a reference for ROI area extraction, positioning and correction of the image of the component to be tested, and the calculation is simple and efficient, and the result is more stable. The template image is used as a reference for component defect judgment, and there is no need to build a model like the deep learning method, which saves resources and hardware resource costs, and has a fast detection speed and low cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The specific implementation modes of the present invention are described in detail below in conjunction with the accompanying drawings.

[0035] Figure 1 It is a schematic diagram of the component defect detection process proposed by the present invention;

[0036] Figure 2 This is a schematic diagram of pin division in an embodiment of the present invention;

[0037] Figure 3This is a normal component display diagram in an embodiment of the present invention;

[0038] Figure 4 This is a diagram showing components with missing pins in an embodiment of the present invention;

[0039] Figure 5 is a diagram showing a pin offset component in an embodiment of the present invention;

[0040] Figure 6 This is a diagram showing a component with warped pins in an embodiment of the present invention;

[0041] Figure 7 It is a diagram showing the pin-bending component in an embodiment of the present invention. DETAILED DESCRIPTION

[0042] In order to further illustrate the features of the present invention, the technical solution of the present invention is described in more detail below through specific implementation methods. However, the present invention can be implemented in many other places different from the description, and those skilled in the art can make similar improvements without violating the connotation of the present invention. Therefore, the present invention is not limited by the specific implementation disclosed below.

[0043] In a first aspect, an embodiment of the present invention provides a component defect detection method, such as Figure 1 As shown, the following steps are included:

[0044] Constructing a template image according to standard component parameter information;

[0045] Exemplarily, according to the input component parameter information, the pin type, number of pins, pin distribution, and pin length and width of the component are obtained. The actual component size information is obtained by using the camera, and the component size is determined according to the input component parameter information. An identical image is drawn according to the component size, and then the pins and other conditions are drawn at the corresponding positions on this image according to the distribution and size of the pins on the component, and finally a standard template image is obtained. The construction of the template image does not require the use of the actual captured image, but directly uses the structural information and camera information of the component to construct the standard image, which is more robust and avoids the problem of differences in images taken by the same machine in different environments, such as the influence of lighting.

[0046] Identify the image of the component to be tested and correct its posture, and obtain the ROI area based on the template image;

[0047] Exemplarily, the image of the component to be tested is collected by a camera, the image is grayed, and the component detection algorithm is used to detect the component's posture information, mainly including the center point position of the component, the size of the minimum circumscribed rectangle, and the rotation angle information. The component detection algorithm refers to patents CN202310105302.4, CN202211076630.8, and CN202211229168.0. Specifically, for example, for CHIP components, the methods in CN202310105302.4 and CN202211076630.8 are used to obtain the component's posture information. For QFN components, the method in CN202211229168.0 is used to calculate the component's rotation angle. That is, the component is corrected according to the center position and rotation angle of the image, as well as the size of the minimum circumscribed rectangle of the image. The image of the component to be tested that has an angle offset relative to the template image is corrected. Specifically, the rotation matrix is ​​calculated according to the center point position and rotation angle of the component; then the obtained rotation matrix is ​​applied to the image using affine transformation to achieve rotation correction of the image.

[0048] Furthermore, the ROI region is obtained according to the template image by segmenting the template image according to its size, and extracting the common part with the template image from the segmented image. The specific operation of extracting the common part is to compare the two numbers at the same pixel position on the segmented corrected image and the template image respectively, and when both are 255, the pixel value at the position is 255, otherwise the value at the position is 0.

[0049] The technical solution of the present invention uses the template image as a reference to extract the ROI area of ​​the processed image of the component to be tested. There is no need to use Fast feature point detection, Hamming matching algorithm and RANSAC feature point screening to first perform matching and then perform differential calculation. It only needs to use the constructed template image to search for components and determine the common position area of ​​the component image. The calculation is faster and the result is more stable.

[0050] Correct the ROI area according to the template image to eliminate the offset of the pins in the ROI area;

[0051] The extracted ROI area is divided into component pin positions using a grayscale projection algorithm to obtain the position information of the pins on the ROI area image. For example, the pin positions of the four sides of the top, bottom, left, and right are calculated based on the grayscale projection method calculation results, such as Figure 2As shown, the upper pin belongs to the range of Y1 to Y2, the lower pin belongs to the range of Y3 to Y4, the left pin belongs to the range of X1 to X2, and the right pin belongs to the range of X3 to X4. For example, for SOP-8 components, there are pins on the upper and lower sides, and no pins on the left and right. Then, only the pins on the upper and lower sides need to be calculated, that is, the pin distribution within the range of Y1 to Y2 and the pin distribution within the range of Y3 to Y4. In the range of Y1 to Y2, the pin edge contour point set whose number of edge contour point sets is less than the preset value is removed, and the value is pre-set to 10 for example. Calculate the center position of the remaining existing pin contour point set, and calculate the slope of the straight line connected by these center position coordinates. Similarly, in the range of Y3 to Y4, the contour point set that meets the conditions is also calculated. Use the slope of the straight line to determine whether the extracted ROI area is corrected. If the correction condition is not met, continue to calibrate according to the calculated slope angle, and adjust the rotation angle of the ROI area so that the slopes of each straight line meet the requirements. The exemplary setting condition is that the angle corresponding to the slope of the straight line is less than 1°, that is, the difference between the slope of the straight line and the slope of the side corresponding to the template image is less than 1°. When this condition is not met, the angle between the slope of the straight line and the horizontal line is regarded as a new rotation angle, the rotation matrix under the rotation angle is calculated, and then the rotation matrix is ​​applied to the image using a reflection transformation to achieve the rotation correction operation of the image.

[0052] Compare the pin information in the ROI area with the pin information in the template image to determine component defects.

[0053] Compare the number of pins in the ROI area with the number of pins in the template image to see if they are consistent. If not, calculate the defect position information, take the position distance value between adjacent pins on each side of the template image as the standard deviation value, calculate the center position of each pin on the side where the pin defect exists in the ROI area, calculate the position difference of adjacent pins in the ROI area based on the pin center position, and compare each position difference with the corresponding standard deviation value to obtain the defect position information; if the number of pins in the ROI area is consistent with the number of pins in the template image, the judgment result is that there is no pin number defect, and then judge whether there is pin offset and whether there is pin defect based on the pin position information. The pins are warped or bent down, specifically: the position distance value between adjacent pins on each side of the template image is used as the standard deviation value, the position difference between adjacent pins on each side of the ROI area is calculated, and the size of each position difference is compared with the corresponding standard deviation value to see whether it is within the error range. If it is not within the error range, it is determined that there is an offset between the corresponding adjacent pins; the acceptable fluctuation range of the pins being warped or bent down is calculated according to the pin length information of the template image, the pin length information of each side of the ROI area is calculated, and whether each pin is warped or bent down is calculated according to the acceptable fluctuation range of the corresponding pins on the template image, and the pin positions that do not meet the conditions are recorded and output. Among them, the allowable error range of each position difference value and the corresponding standard deviation value is related to the process of the component, and 30% of the relevant parameters of the standard component is taken as an example.

[0054] Taking SOP-8 components as an example, normal components such as Figure 3 As shown, the pin missing components detected by the detection method of the present invention are as follows Figure 4 As shown, the pin offset components are Figure 5 As shown, the pin-up components are Figure 6 As shown, the pins are bent downwards as Figure 7 As shown. The detection speed is fast, and it is applicable to a wide range of component pin defect types and has accurate detection.

[0055] The component defect judgment method of the present invention does not need to build a model like the deep learning method, which effectively saves resources and hardware resource costs. Deep learning and other methods require a good hardware environment and need to be on a GPU to obtain a good running time. The effect on the CPU is not as good as the solution of the present invention.

[0056] Based on the same inventive concept, a second aspect of an embodiment of the present invention provides a component defect detection system, comprising:

[0057] A template image module is used to construct a template image according to standard component parameter information;

[0058] ROI area module, used to identify the image of the component to be tested and correct the posture, and obtain the ROI area according to the template image;

[0059] The ROI area correction module is used to correct the ROI area according to the template image to eliminate the offset of the pins in the ROI area;

[0060] The defect detection module is used to compare the pin information in the ROI area with the pin information in the template image to determine the component defects.

[0061] A third aspect of an embodiment of the present invention provides a computer-readable storage medium, including a program, wherein the program can be executed by a processor to implement the steps of the above method.

[0062] To sum up, the technical solution of the present invention does not need to use the actual captured images, but directly uses the structural information and camera information of the standard components to construct the template image, which is more robust; the ROI area extraction, positioning and correction of the image of the component to be tested are performed based on the template image, the calculation is simple and efficient, and the result is more stable; the component defect judgment is performed based on the template image, and there is no need to build a model like the deep learning method, which saves resources and hardware resource costs, and has a fast detection speed and low cost.

[0063] The preferred embodiments of the present invention are described in detail above in conjunction with the accompanying drawings, but the present invention is not limited thereto. Within the technical concept of the present invention, the technical solution of the present invention can be subjected to a variety of simple modifications, including the combination of various specific technical features in any suitable manner. In order to avoid unnecessary repetition, the present invention will not further describe various possible combinations. However, these simple modifications and combinations should also be regarded as the contents disclosed by the present invention and belong to the protection scope of the present invention.

Claims

1. A component defect detection method, characterized in that: The following steps are involved: Using a camera to obtain actual standard component parameter information, determining the component size according to the input standard component parameter information and drawing an identical image, drawing pins at corresponding positions on the image, and constructing a template image, wherein the construction of the template image does not require the use of an actual captured image; the standard component parameter information includes the component's pin type, pin number, pin distribution, and pin size; Identify the image of the component to be tested and correct its posture, correct the image of the component to be tested that has an angle offset relative to the template image, and specifically calculate the rotation matrix based on the position of the component center point and the rotation angle; then use affine transformation to apply the obtained rotation matrix to the image to achieve rotation correction of the image; segment the image according to the size of the template image, extract the common part with the template image from the segmented image, and obtain the ROI area based on the template image; the specific operation of obtaining the ROI area based on the template image is as follows: segment the image according to the size of the template image, and extract the common part with the template image from the segmented image; the specific operation of extracting the common part is as follows: compare the two numbers at the same pixel position on the segmented correction image and the template image respectively, and when both are 255, the pixel value at the position is 255, otherwise the value at the position is 0; Correct the ROI area according to the template image to eliminate the offset of the pins in the ROI area; specifically: use the grayscale projection algorithm to divide the component pin positions in the extracted ROI area to obtain the position information of the pins on the ROI area image; calculate the pin positions of the four sides of the top, bottom, left and right according to the calculation results of the grayscale projection method, remove the pin edge contour point sets whose number of edge contour point sets is less than the preset value, calculate the center positions of the remaining existing pin contour point sets, calculate the slope of the straight line connecting the coordinates of these center positions, and use the slope of the straight line to determine whether the extracted ROI area is corrected; if the correction condition is not met, continue to correct according to the calculated slope angle, adjust the rotation angle of the ROI area so that the slopes of each straight line meet the requirements, and when the correction condition is not met, take the angle between the slope of the straight line and the horizontal line as the new rotation angle, calculate the rotation matrix under the rotation angle, and then use the reflection transformation to apply the rotation matrix to the image to achieve the rotation correction operation of the image; Compare the pin information in the ROI area with the pin information in the template image to determine component defects, specifically: compare whether the number of pins in the ROI area is consistent with the number of pins in the template image. If not, calculate the defect position information; if consistent, the judgment result is that there is no pin number defect, and then judge whether there is pin offset based on the pin position information, and whether there is pin upturning or downturning.

2. The component defect detection method according to claim 1, characterized in that: The calculation of defect location information is specifically as follows: The position distance value between adjacent pins on each side of the template image is used as the standard deviation value, and the center position of each pin on the side where the pin defect exists in the ROI area is calculated. The position difference of adjacent pins in the ROI area is calculated based on the pin center position, and each position difference is compared with the corresponding standard deviation value to obtain the defect position information.

3. The component defect detection method according to claim 2, characterized in that: The method of judging whether there is pin offset according to the pin position information is specifically as follows: The position distance value between adjacent pins on each side of the template image is used as the standard deviation value, and the position difference value between adjacent pins on each side of the ROI area is calculated. The position difference value is compared with the corresponding standard deviation value to see if it is within the error range. If it is not within the error range, it is determined that there is an offset between the corresponding adjacent pins.

4. The component defect detection method according to claim 2, characterized in that: Determining whether there is any pin warping or bending according to the pin position information is specifically as follows: calculating the acceptable fluctuation range of the pin warping or bending according to the pin length information of the template image, calculating the pin length information of each side of the ROI area, and calculating whether each pin is warped or bent according to the acceptable fluctuation range of the corresponding pin on the template image.

5. The component defect detection method according to any one of claims 1 to 4, characterized in that: The method of identifying the image of the component to be tested and correcting the posture is as follows: using a camera to obtain the image of the component to be tested, graying it, detecting the posture information of the component to be tested, including the center point position, the minimum circumscribed rectangle size and the rotation angle information, and correcting the image of the component to be tested based on the template image.

6. A component defect detection system, characterized in that: include: The template image module is used to obtain the actual standard component parameter information using a camera, determine the component size according to the input standard component parameter information and draw an identical image, draw the pins at the corresponding positions on the image, and construct a template image. The construction of the template image does not require the use of the actual captured image; the standard component parameter information includes the pin type, pin number, pin distribution and pin size of the component; The ROI area module is used to identify the image of the component to be tested and correct the posture, and correct the image of the component to be tested that has an angle offset relative to the template image. Specifically, the rotation matrix is ​​calculated according to the position of the center point of the component and the rotation angle; then the obtained rotation matrix is ​​applied to the image using affine transformation to realize the rotation correction of the image; the template image is segmented according to the size of the template image, and the common part with the template image is extracted from the segmented image, and the ROI area is obtained according to the template image; the ROI area is obtained according to the template image as follows: the template image is segmented according to the size of the template image, and the common part with the template image is extracted from the segmented image; the specific operation of extracting the common part is: the two numbers at the same pixel point position on the segmented correction image and the template image are compared respectively, and when both are 255, the pixel value of the position is 255, otherwise the value of the position is 0; The ROI area correction module is used to correct the ROI area according to the template image to eliminate the offset of the pins in the ROI area; specifically: the extracted ROI area is divided into the component pin positions using the grayscale projection algorithm to obtain the position information of the pins on the ROI area image; the pin positions of the four sides of the upper, lower, left and right are calculated according to the calculation results of the grayscale projection method, the pin edge contour point sets whose number of edge contour point sets is less than the preset value are removed, the center positions of the remaining existing pin contour point sets are calculated, and the slope of the straight line connecting the coordinates of these center positions is calculated, and the slope of the straight line is used to determine whether the extracted ROI area is corrected; if the correction condition is not met, the correction is continued according to the calculated slope angle, and the rotation angle of the ROI area is adjusted so that the slopes of each straight line meet the requirements. When the correction condition is not met, the angle between the slope of the straight line and the horizontal line is regarded as the new rotation angle, and the rotation matrix under the rotation angle is calculated, and then the rotation matrix is ​​applied to the image using the reflection transformation to realize the rotation correction operation of the image; The defect detection module is used to compare the pin information in the ROI area with the pin information in the template image to determine the component defects. Specifically, the number of pins in the ROI area is compared with the number of pins in the template image to see if they are consistent. If not, the defect position information is calculated; if they are consistent, the judgment result is that there is no pin number defect, and then the pin position information is used to determine whether there is pin offset, and whether there is pin warping or bending.

7. A computer-readable storage medium, characterized in that: The method comprises a program which can be executed by a processor to implement the steps of the method according to any one of claims 1 to 5.

Images