Plug-in abnormality detection method, processor and detection camera

Abstract

This application relates to the field of device testing, specifically to a method, controller, and testing camera for detecting abnormal insertion of components. The method includes acquiring a first image of the board before insertion and a second image of the board after insertion; performing position correction on the first or second image based on a reference image in the first and second images; comparing the position-corrected first / second image with the uncorrected second / first image to obtain pin images; and determining whether the insertion is abnormal based on the geometric or structural features of the pins in the pin images. The anomaly detection process requires no manual intervention, improving the accuracy and timeliness of the detection results and increasing the efficiency of anomaly detection.

Description

Technical Field

[0001] This application relates to the field of device testing, and more specifically, to a method, controller, and testing camera for detecting abnormal insertion of plug-in components. Background Technology

[0002] In automated production lines, high yield rates, continuous production, and high efficiency are fundamental requirements. Therefore, it is necessary to inspect the products on the production line to promptly identify anomalies, allowing for troubleshooting and resolution of problems, and preventing the automated production line from producing a large number of defective products.

[0003] For plug-in assembly, robots on the production line automatically pick up the plug-ins and insert them into the insert plates. However, checking whether the plug-ins are accurately inserted into the insert plates usually relies on manual visual inspection. Because the plug-ins have many pins and are small in size, this method is inefficient and reduces the efficiency of anomaly detection. Summary of the Invention

[0004] This application provides a method, controller, and detection camera for detecting abnormal plug-in insertion, so as to at least solve the technical problem of low detection efficiency.

[0005] According to a first aspect of the embodiments of this application, a method for detecting plug-in insertion anomalies is provided. The plug-in includes a plurality of pins, and the insertion board includes a plurality of insertion areas for inserting the pins. The detection method includes:

[0006] Acquire a first image of the pre-insertion plate and a second image of the post-insertion plate, wherein the first image contains an image of the insertion area and the second image contains an image of the insertion area in which the pins are inserted;

[0007] The position of the first image or the second image is corrected based on the reference volume image in the first image and the second image.

[0008] The first image / second image after position correction is compared with the second image / first image without position correction to obtain the pin image;

[0009] Based on the geometric or structural features of the pins in the pin image, determine whether the plug-in insertion is abnormal.

[0010] Optionally, the step of performing position correction on the first image or the second image based on a reference volume image in the first image and the second image includes:

[0011] Obtain the first position coordinates of the reference body image in the first image and the second position coordinates of the reference body image in the second image, wherein the reference body includes an insertion area at a non-insertion position, the first position coordinates include the center point coordinates of the reference body image in the first image, and the second position coordinates include the center point coordinates of the reference body image in the second image;

[0012] Calculate the affine transformation matrix parameters based on the first and second position coordinates;

[0013] Construct the affine transformation matrix based on the affine transformation matrix parameters;

[0014] Multiply the first matrix corresponding to the first image by the affine transformation matrix to obtain the first image after position correction.

[0015] Optionally, the reference image includes an image of at least three insertion areas on the insert plate where no pins are inserted; the first position coordinates are... The second position coordinates are Where i is the number of insertion areas without inserted pins, and i is a positive integer;

[0016] The calculation of the affine transformation matrix parameters based on the first and second position coordinates includes:

[0017] Obtain the first and second position coordinates of at least three insertion areas of the uninserted pins, and obtain the first position coordinates. The second position coordinates are Where i ≥ 3;

[0018] A system of linear equations is constructed based on the first and second position coordinates to solve for the scaling, rotation, and translation parameters, wherein the affine transformation matrix parameters include the scaling, rotation, and translation parameters.

[0019] The scaling parameter, rotation parameter, and translation parameter are obtained by solving the linear equation system using the least squares method.

[0020] The linear equation system includes:

[0021]

[0022] in, Let x be the x-coordinate of the insertion area of ​​the i-th uninserted pin in the first image. Let be the ordinate of the insertion area of ​​the i-th uninserted pin in the first image. Let x be the x-coordinate of the insertion area of ​​the i-th uninserted pin in the second image. Let t be the ordinate of the insertion area of ​​the i-th uninserted pin in the second image.x and t y Let a and b be the translation parameters, a and b be the scaling parameters and both be 1, and c and d be the rotation parameters.

[0023] Optionally, multiplying the first matrix corresponding to the first image with the affine transformation matrix to obtain the position-corrected first image includes:

[0024] Perform matrix multiplication using the following formula:

[0025] P′=AP;

[0026] Where P′ is the first image after position correction, A is the first matrix, and P is the affine transformation matrix, wherein the affine transformation matrix is ​​expressed as:

[0027]

[0028] Among them, t x and t y Let a and b be the translation parameters, a and b be the scaling parameters and both be 1, and c and d be the rotation parameters.

[0029] Optionally, the comparison process includes image differential processing.

[0030] Optionally, determining whether the plug-in insertion is abnormal based on the geometric or structural features of the pins in the pin image includes:

[0031] Based on the geometric or structural features of the two furthest pins in the pin image, determine whether the plug-in insertion is abnormal.

[0032] Optionally, determining whether the plug-in insertion is abnormal based on the geometric or structural features of the two furthest pins in the pin image includes:

[0033] Based on binarization processing, the contour information of the two furthest pins in the pin image is obtained;

[0034] The geometric or structural features of the two furthest pins are determined based on the contour information;

[0035] The geometric or structural features of the two pins furthest apart are respectively input into a preset first support vector machine and a second support vector machine for classification processing to obtain the detection result of whether the insertion is abnormal.

[0036] Optionally, the method further includes:

[0037] The first image is processed using a rotating caliper algorithm to determine the two insertion regions that are furthest apart in the first image;

[0038] The two furthest pins in the pin image are determined based on the two furthest insertion areas in the first image.

[0039] According to a second aspect of the embodiments of this application, a processor is provided that employs the method described above.

[0040] According to a third aspect of the embodiments of this application, a detection camera is provided, including the processor described above.

[0041] In this embodiment, by acquiring a first image and a second image, position correction is performed on either the first or second image. Taking the first image as an example, after comparing the position-corrected first image with the uncorrected second image, a pin image containing the pins can be obtained. Since both the first and second images are taken of the insert plate, and the second image is the image after insertion, the pin image is actually an image of the pins inserted into the insertion area of ​​the insert plate. Finally, the geometric or structural features of the pins in the pin image are used to determine whether there is an anomaly, thus realizing the anomaly detection of the pins inserted into the insertion area of ​​the insert plate. The anomaly detection process does not require manual intervention, improving the accuracy and timeliness of the detection results and improving the efficiency of anomaly detection. Attached Figure Description

[0042] Figure 1 This is a flowchart of a method for detecting plug-in insertion anomalies in one embodiment.

[0043] Figure 2 This is a flowchart of position correction in one embodiment.

[0044] Figure 3 This is a schematic diagram of the plug-in insertion system in one embodiment.

[0045] Figure 4 This is a schematic diagram of determining the pin image in one embodiment.

[0046] Figure 5 This is a schematic diagram of image alignment in one embodiment.

[0047] Labeling descriptions: 1. Robot; 2. Gripper; 3. Light source; 4. Pin inspection camera; 5. Solder hole inspection camera; 6. Surface light source; 7. Light source controller; 8. Industrial computer; 9. Irregularly shaped plug-in; 10. PCB board. Detailed Implementation

[0048] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0049] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0050] According to an embodiment of this application, an embodiment of a method for detecting plug-in insertion anomalies is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0051] For ease of understanding, the features involved in this embodiment are described below:

[0052] Plug-in: Refers to a device that needs to be inserted into a circuit board. The structure of a plug-in includes at least a pin for insertion into the circuit board to connect the plug-in to the circuit board. In this embodiment, for ease of understanding, the pin is referred to as a pin. Specifically, plug-ins include, for example, chips.

[0053] Pin: A part of the plug-in, used for insertion into the plug-in board. This embodiment does not specifically limit the shape of the pin. Specifically, a pin is, for example, a chip pin.

[0054] Insertion board: refers to a device that requires the insertion of components, including insertion areas for pins. Specifically, an insertion board is, for example, a PCB board, which has multiple solder holes for inserting chip pins.

[0055] Insertion area: refers to the area opened on or formed on the insert plate, such as a slot or hole.

[0056] Furthermore, this embodiment uses irregularly shaped plug-ins and PCB boards as examples to illustrate the detection method provided in this embodiment. Specifically, irregularly shaped plug-ins refer to plug-ins whose shape is irregular or whose pins are irregular. PCB boards (Printed Circuit Boards) are important electronic components, serving as the support for electronic components and providing electrical connections. Because they are manufactured using electronic printing techniques, they are called "printed" circuit boards.

[0057] In some industrial sectors, it is necessary to connect irregularly shaped components such as chips to PCBs. Due to the large number of pins on the chips, their uneven distribution, and their small size, the insertion process is quite challenging. Even slight errors can lead to pin misalignment or damage, ultimately causing the chip to malfunction. To address this issue, it is essential to regularly inspect the pins inserted into the PCB. This inspection typically includes verifying the correct pin placement and checking for damage.

[0058] Current testing methods mostly rely on manual labor, but manual testing is greatly affected by the subjective nature of workers, and workers are prone to visual fatigue, resulting in poor testing efficiency and accuracy of test results.

[0059] The plug-in insertion anomaly detection method provided in this embodiment is used to detect anomalies in plug-ins inserted into a plug-in board. The plug-in includes several pins, and the plug-in board includes several insertion areas for the pins to be inserted. For example... Figure 1 As shown, the detection methods include:

[0060] S101. Obtain the first image of the pre-insertion plate and the second image of the post-insertion plate.

[0061] The first image contains the insertion area, and the second image contains the insertion area with inserted pins. Specifically, before insertion, the insert plate is photographed to obtain the first image, which contains pixel regions representing the insertion area. After insertion, the insert plate is photographed again. Since the insert has now been inserted into the plate, the pins are located within the insertion area, so the second image contains both the insertion area and the pins located within it. This embodiment does not limit the number or shape of the pins and the insertion area. Preferably, the number of pins is three or more, and the number of insertion areas is greater than the number of pins.

[0062] S102. Perform position correction on the first image or the second image based on the reference image in the first image and the second image.

[0063] In this context, the reference image refers to the image of a reference object. The reference object refers to a structure on the insert board that remains unchanged before and after insertion and is not obscured by the insert or pins in the second image. Preferably, the reference object is the insertion area of ​​the non-insertion region on the insert board; specifically, the reference object is the solder hole in the non-insertion region of the PCB board. Here, the PCB board is the insert board, the solder hole is the insertion area of ​​the insert board, and the non-insertion region refers to the area where the solder hole does not require pin insertion.

[0064] Both the first and second images contain a reference image. Therefore, the position of the first or second image can be corrected based on the reference image, so that the positions of the first and second images correspond.

[0065] The reference body can be a pre-defined structure, such as solder holes in a non-insertion area on a PCB board. By pre-determining the solder holes in the non-insertion area, after obtaining the first and second images, the reference body image corresponding to the solder holes in the non-insertion area can be found in the first and second images using pixel points and / or pixel coordinates. Alternatively, image recognition algorithms or image recognition models can be used to identify the reference body image from the first and second images respectively.

[0066] The reference image can also be determined in real time using algorithms or models. For example, a CNN algorithm or neural network model can be used to identify the first and second images, determine the same region in the first and second images, and identify the image of that region as the reference image.

[0067] Position correction refers to the adjustment of the positions of the first and second images. During the insertion process, the insert plate may translate or rotate horizontally, causing the captured second image and the first image to lie in the same plane but at different positions. This makes it impossible to accurately locate the pin image in the second image based on the first image when identifying the pins later. Therefore, position correction is performed on either the first or second image so that the coordinates of all pixels in the first and second images are the same after correction. It should be noted that the purpose of position correction is to ensure that the coordinates of each insertion area in the first image are consistent with the coordinates of the corresponding insertion area in the second image within the same coordinate system.

[0068] S103. Compare the first image / second image after position correction with the second image / first image without position correction to obtain the pin image.

[0069] After position correction, each insertion area in the first image is positioned identically to its corresponding insertion area in the second image. For example, the insertion panel includes a first insertion area, a second insertion area, and a third insertion area. In the first image, the first insertion area is located in the third column, fourth row; the second insertion area is located in the sixth column, fourth row; and the third insertion area is located in the ninth column, fourth row. In the second image, the first insertion area is also located in the third column, fourth row, and the second and third insertion areas are similarly located. Since the insertion areas in the second image contain pins, after comparison processing, pixels in the first and second images that are in the same position but have different pixel information can be identified. This pixel represents the pin image.

[0070] S104. Determine whether the plug-in insertion is abnormal based on the geometric or structural features of the pins in the pin image.

[0071] The geometric or structural features of the pins can be determined from the pin images, including their position, deformation, and insertion depth, thereby identifying any abnormalities in the plug-in assembly.

[0072] Specifically, geometric features include the shape and perimeter of the stitch outline as reflected in the stitch image, while structural features include the area and center point position of the stitch as reflected in the stitch image.

[0073] Through the above process, by acquiring a first image and a second image, and performing position correction on either the first or second image (taking position correction of the first image as an example), the position-corrected first image is compared with the uncorrected second image to obtain a pin image containing the pins. Since both the first and second images are taken of the insert plate, and the second image is the image after insertion, the pin image is actually an image of the pins inserted into the insertion area of ​​the insert plate. Finally, based on the geometric or structural features of the pins in the pin image, anomalies are determined, thus achieving anomaly detection of pins inserted into the insertion area of ​​the insert plate. The anomaly detection process requires no manual intervention, improving the accuracy and timeliness of the detection results and increasing anomaly detection efficiency.

[0074] In another embodiment of this application, such as Figure 2 As shown, position correction of the first image or the second image is performed based on a reference volume image in the first image and the second image, including:

[0075] S201. Obtain the first position coordinates of the reference image in the first image and the second position coordinates of the reference image in the second image.

[0076] The reference body includes an insertion area at a non-insertion position, the first position coordinates include the coordinates of the center point of the reference body image in the first image, and the second position coordinates include the coordinates of the center point of the reference body image in the second image.

[0077] The first and second images can use the same coordinate system. For example, an xy coordinate system can be established with the x-axis as the horizontal axis and the y-axis as the vertical axis. The left vertex of the first image and the left vertex of the second image can be placed at the origin of the coordinate system, thereby assigning coordinate values ​​to each pixel in the first and second images, so that each pixel in the first image has corresponding coordinates and each pixel in the second image has corresponding coordinates.

[0078] The first image and the second image can also use different coordinate systems. If the first image and the second image use different coordinate systems, then in the subsequent processing, when calculating the affine transformation matrix parameters of the same point on the insert, the position difference between the two coordinate systems can be compensated.

[0079] S202. Calculate the affine transformation matrix parameters based on the first and second position coordinates.

[0080] S203. Construct the affine transformation matrix based on the parameters of the affine transformation matrix.

[0081] S204. Multiply the first matrix corresponding to the first image with the affine transformation matrix to obtain the first image after position correction.

[0082] After obtaining the affine transformation matrix parameters, the first and second images can be aligned according to these parameters. The position of the first image can be corrected to align with the second image, or the position of the second image can be corrected to align with the first image, or both can be corrected simultaneously. This embodiment does not impose specific limitations on this; the aim is simply to ensure that, after aligning the first and second images, the first and second position coordinates of the same point on the insert are identical.

[0083] Based on the above, the affine transformation matrix parameters are first calculated by obtaining the first and second position coordinates. Then, the first and second images are aligned according to the affine transformation matrix parameters to achieve position correction. Using coordinates to correct the positions of the first and second images is convenient to calculate and ensures reliable processing results.

[0084] In another embodiment of this application, the reference image includes an image of at least three insertion areas on the solder plate where no pins are inserted; the first position coordinates are... The second position coordinates are Where i is the number of insertion areas without inserted pins, and i is a positive integer;

[0085] The parameters of the affine transformation matrix are calculated based on the first and second position coordinates, including:

[0086] Obtain the first and second position coordinates of the insertion area of ​​at least three uninserted pins to obtain the first position coordinates. The second position coordinates are Where i ≥ 3;

[0087] A system of linear equations is constructed based on the first and second position coordinates to solve for the scaling, rotation, and translation parameters. The affine transformation matrix parameters include the scaling, rotation, and translation parameters.

[0088] The scaling, rotation, and translation parameters are obtained by solving the linear equation system using the least squares method.

[0089] The system of equations includes:

[0090]

[0091] in, Let x be the x-coordinate of the insertion area of ​​the i-th uninserted pin in the first image. Let be the ordinate of the insertion area of ​​the i-th uninserted pin in the first image. Let x be the x-coordinate of the insertion area of ​​the i-th uninserted pin in the second image. Let t be the ordinate of the insertion area of ​​the i-th uninserted pin in the second image. x and t y is the translation parameter, a and b are the scaling parameters and both are 1, and c and d are the rotation parameters.

[0092] The insertion area where no pins are inserted can be a solder hole in a non-insertion area of ​​the PCB board, or a solder hole where no pins are inserted in this insertion.

[0093] The above methods demonstrate that using a system of equations to calculate the parameters of an affine transformation matrix can help improve the accuracy of such calculations.

[0094] In another embodiment of this application, multiplying the first matrix corresponding to the first image with the affine transformation matrix to obtain the position-corrected first image includes:

[0095] Perform matrix multiplication using the following formula:

[0096] P′=AP;

[0097] Where P′ is the first image after position correction, A is the first matrix, and P is the affine transformation matrix, where the affine transformation matrix is ​​expressed as:

[0098]

[0099] Among them, t x and ty is the translation parameter, a and b are the scaling parameters and both are 1, and c and d are the rotation parameters.

[0100] Based on the above, image alignment using affine transformation algorithms can achieve position correction processing, which helps to improve the accuracy of position correction.

[0101] Taking a PCB board as an example, since the image itself is a matrix, the affine transformation here is equivalent to calculating a matrix (that is, the first matrix) based on the coordinates of the solder holes before and after insertion at three or more non-insertion positions, and then calculating a third offset and rotation matrix (the first image after offset and rotation that can align the solder holes with the second image) with another matrix (the image before insertion, that is, the second matrix).

[0102] In other words, affine transformations between images typically involve calculating an affine transformation matrix. An affine transformation is a linear transformation that includes operations such as scaling, translation, and rotation. In two-dimensional space, an affine transformation is represented by a 2x3 matrix:

[0103]

[0104] a and b are scaling parameters, c and d are rotation parameters, and t... x and t y The scaling parameter is a translation parameter. This scheme does not involve scaling, and the affine transformation process does not involve image scaling. Therefore, the scaling coefficients a = 1 and b = 1. Hence, the scaling parameter is 1. Applying the above matrix to each pixel of an image, i.e., performing matrix multiplication, transforms the coordinates (x1, y1) of each pixel into new coordinates (x2, y2), resulting in the final matrix.

[0105] In another embodiment of this application, the comparison process includes image differential processing.

[0106] Using an image difference algorithm to compare the first and second images helps improve the accuracy of the pin images.

[0107] In another embodiment of this application, determining whether the plug-in insertion is abnormal based on the geometric or structural features of the pins in the pin image includes:

[0108] Based on the geometric or structural features of the two furthest pins in the pin image, determine whether the plug-in insertion is abnormal.

[0109] In one embodiment, when determining whether the plug-in insertion is abnormal, only the geometric or structural features of the two pins furthest apart are considered. Here, the two pins furthest apart refer to the two pins with the greatest straight-line distance.

[0110] When determining the two pins that are furthest apart, the determination can be made based on the two insertion areas that are furthest apart.

[0111] By using the above method, we can determine whether the plug-in insertion is abnormal by using only the two pins that are furthest apart. This reduces the amount of calculation required to determine the abnormality and improves the efficiency of the abnormality determination process.

[0112] In another embodiment of this application, determining whether the plug-in insertion is abnormal based on the geometric or structural features of the two furthest pins in the pin image includes:

[0113] Based on binarization processing, the contour information of the two furthest pins in the pin image is obtained;

[0114] Determine the geometric or structural features of the two furthest pins based on the contour information;

[0115] The geometric or structural features of the two pins furthest apart are input into the preset first and second support vector machines for classification processing to obtain the detection results for whether the insertion is abnormal.

[0116] The first and second support vector machines are trained using different data. Specifically, the first support vector machine is trained using data from the first pin out of the two furthest pins to identify anomalies in the first pin, while the second support vector machine is trained using data from the second pin out of the two furthest pins to identify anomalies in the second pin.

[0117] It should be noted that, due to the camera's shooting position, the first and second pins are usually located on opposite sides of the second image, i.e., on opposite sides of the interposer. During shooting, the first and second pins may be partially obscured by the interposer, and the obstruction may occur in different directions. For example, if the first pin is on the left, the resulting pin image may only show the actual right side of the first pin, and the pin image may only show the actual left side of the second pin. Therefore, using different classification vector machines helps reduce the probability of false positives and improve the accuracy of anomaly detection.

[0118] In another embodiment of this application, the method further includes:

[0119] The first image is processed using a rotating caliper algorithm to determine the two insertion regions that are furthest apart in the first image;

[0120] The two furthest pins in the pin image are determined based on the two furthest insertion areas in the first image.

[0121] Specifically, after obtaining the two insertion areas that are furthest apart, the positions of these two insertion areas can be determined in the second image, thereby identifying the pins located in these two insertion areas in the second image as the two pins that are furthest apart.

[0122] The above methods, which determine the two furthest insertion areas to identify the two furthest pins, help improve the accuracy of pin identification.

[0123] For ease of understanding, such as Figure 3-5 As shown, the method of this embodiment is explained using an irregularly shaped plug-in, a PCB board as the insert board, and a solder hole on the PCB board as the insertion area.

[0124] like Figure 3 As shown, the solder hole inspection camera takes a picture of the PCB board before the plug-in is placed on the PCB board by the gripper, and obtains the solder hole image before insertion (that is, the first image). At the same time or afterward, the pin inspection camera takes a picture of the irregular plug-in on the gripper, and obtains the pin image before insertion. Then, the robot uses the gripper to transport the irregular plug-in to the top of the PCB board and move it down to complete the plug-in insertion.

[0125] After the components are inserted, the solder hole inspection camera takes another picture of the PCB board to obtain the solder hole image after insertion (that is, the second image).

[0126] Industrial irregular component insertion platforms are affected by industrial environment or operation factors such as conveyor belt vibration and robot insertion component contact with PCB boards. This causes PCB board rotation and offset before and after insertion, resulting in misalignment of solder holes in the images before and after insertion. The method in this embodiment uses a template to match the position of non-insertion solder holes (e.g., ...). Figure 4 The four solder holes in the upper left corner (the positions of the non-insertion solder holes) must be positioned so that they are not obscured before or after insertion. Within the defined ROI area, the coordinates of the non-insertion solder holes are precisely located using circle detection. Aligning the solder holes in two PCB images using affine transformation requires calculating an affine transformation matrix that maps the solder hole coordinates in one image to the corresponding coordinates in the other. Assuming image A before insertion and image B after insertion, and their respective solder hole coordinates, the corresponding non-insertion solder hole coordinates in image A will be represented as (x...). A ,y A The coordinates of the corresponding non-insertion position weld hole in image B are represented as (x... B ,y B At least three such corresponding point pairs are needed to calculate the affine transformation matrix.

[0127] Affine transformations can be represented in matrix form as follows:

[0128]

[0129] Where a, b, c, and d control rotation and scaling, and t... x ,t y Control the translation.

[0130] To calculate these parameters, the least squares method is used to fit these point pairs. The specific steps are as follows:

[0131] 1. Collect at least three pairs of corresponding points and Where i = 1, 2, 3.

[0132] 2. Construct a system of linear equations to solve for a, b, c, d, and t. x ,t y :

[0133]

[0134] 3. Write these equations in matrix form:

[0135]

[0136] 4. Solve this overdetermined system of equations using the least squares method to obtain a, b, c, d, t. x ,t y The value of .

[0137] 5. Use the obtained parameters to construct the affine transformation matrix.

[0138] 6. Apply this matrix to all pixels of the uninserted image A to obtain the transformed image C, so that the solder holes are aligned.

[0139] Aligning the images and then performing image differencing can extract the complete pin image. The OTSU method, an adaptive thresholding technique, is used to binarize the differencing image to extract the accurate pin outline.

[0140] Geometric and structural features of the pins, such as contour, perimeter, area, and centroid, are extracted from the contours processed by the image algorithm to effectively describe the shape features of the pins. These features are used as input to a support vector machine (SVM) for training and performing a classification task to determine whether the pin insertion is abnormal.

[0141] Based on the number of solder holes on the PCB board where the irregularly shaped component is inserted, extracting the characteristics of each pin and solder hole after insertion to determine whether there is an insertion abnormality is a cumbersome process with a long computation time. Since the consistency of irregularly shaped component materials is generally high, it is only necessary to determine whether the two pins furthest apart are accurately inserted into the solder holes to determine whether all pins of the entire material are inserted into the solder holes, which greatly improves the efficiency of insertion abnormality detection.

[0142] To identify the two furthest weld holes, the image coordinates of multiple weld holes obtained from the weld hole images before insertion in the first step are used to solve the problem of finding the furthest point pairs. The rotating caliper algorithm is used instead of the brute-force solution (calculating the distance between all point pairs). The time complexity of the rotating caliper algorithm is O(n), where n is the number of points. The time complexity of the brute-force solution is O(n^2), which significantly reduces the time complexity.

[0143] Rotating Caliper Algorithm: First, find the leftmost and rightmost points in the point set; the distance between these two points is the initial farthest distance. Then, select an edge as the initial edge. Using the rotating caliper method, rotate the caliper counter-clockwise, ensuring that each edge of the caliper is tangent to some edge on the convex hull of the point set. During the rotation, continuously update the distance between the farthest point pairs. In each rotation, calculate the distance between the two vertices of the current caliper; if this distance is greater than the previously recorded farthest distance, update the farthest distance. Repeat the rotating caliper until one full rotation is completed, i.e., the caliper returns to its initial state. Output the farthest distance and the corresponding two weld holes.

[0144] After the two furthest pins are accurately inserted into the solder hole, the extracted shape features differ due to camera distortion. Therefore, only the shape features of the two furthest pins of the irregular component are extracted here. Furthermore, in the subsequent process of detecting and judging insertion anomalies, two different classifiers, SVM1 and SVM2, need to be used for these two furthest pins respectively.

[0145] This application also provides a processor that uses the above method to detect plug-in insertion abnormalities.

[0146] This application also provides a detection camera, including the processor described above.

[0147] Preferably, the inspection camera serves as the weld hole inspection camera in the plug-in assembly system, such as... Figure 3 As shown, the insertion and assembly system includes a robot 1, a gripper 2, a light source 3, a pin inspection camera 4, a weld hole inspection camera 5, a surface light source 6, a light source controller 7, and an industrial computer 8. The gripper 2 is controlled by the robot 1, the surface light source 6 and the light source 3 are controlled by the light source controller 7, and the industrial computer 8 controls the weld hole inspection camera 5 and the pin inspection camera 4. The weld hole inspection camera 5 stores the insertion board images acquired before and after insertion separately.

[0148] Robot 1 controls gripper 2 to grip, transport and insert irregularly shaped plug-in 9, and surface light source 6 is used to provide light and support PCB board 10.

[0149] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0150] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0151] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0152] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0153] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0154] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0155] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for detecting plug-in insertion anomalies, characterized in that, The plug-in includes several pins, and the plug-in board includes several insertion areas for inserting the pins. The detection method includes: Acquire a first image of the pre-insertion plate and a second image of the post-insertion plate, wherein the first image contains an image of the insertion area and the second image contains an image of the insertion area in which the pins are inserted; The position of the first image or the second image is corrected based on the reference volume image in the first image and the second image. The first image / second image after position correction is compared with the second image / first image without position correction to obtain the pin image; Based on the geometric or structural features of the pins in the pin image, determine whether the plug-in insertion is abnormal; The step of determining whether the plug-in insertion is abnormal based on the geometric or structural features of the pins in the pin image includes: Based on the geometric or structural features of the two furthest pins in the pin image, determine whether the plug-in insertion is abnormal.

2. The method for detecting plug-in insertion anomalies according to claim 1, characterized in that, The step of correcting the position of the first image or the second image based on a reference volume image in the first image and the second image includes: Obtain the first position coordinates of the reference body image in the first image and the second position coordinates of the reference body image in the second image, wherein the reference body includes an insertion area at a non-insertion position, the first position coordinates include the center point coordinates of the reference body image in the first image, and the second position coordinates include the center point coordinates of the reference body image in the second image; Calculate the affine transformation matrix parameters based on the first and second position coordinates; Construct the affine transformation matrix based on the affine transformation matrix parameters; Multiply the first matrix corresponding to the first image by the affine transformation matrix to obtain the first image after position correction.

3. The method for detecting plug-in insertion anomalies according to claim 2, characterized in that, The reference image includes images of at least three insertion areas on the insert plate where no pins are inserted; the first position coordinates are ( , The second position coordinates are ( , );in, This represents the number of insertion areas where no pins are inserted. It is a positive integer; The calculation of the affine transformation matrix parameters based on the first and second position coordinates includes: Obtain the first and second position coordinates of at least three insertion areas of the uninserted pins, and obtain the first position coordinates ( , The second position coordinates are ( , ),in, ≥3; A system of linear equations is constructed based on the first and second position coordinates to solve for the scaling, rotation, and translation parameters, wherein the affine transformation matrix parameters include the scaling, rotation, and translation parameters. The scaling parameter, rotation parameter, and translation parameter are obtained by solving the linear equation system using the least squares method. The linear equation system includes: =a ； =c d ; in, For the first image The x-coordinate of the insertion area where no pins are inserted. For the first image The ordinate of the insertion area where no pins are inserted. For the second image The x-coordinate of the insertion area where no pins are inserted. For the second image The ordinate of the insertion area where no pins are inserted. and Let a and b be the translation parameters, a and b be the scaling parameters and both be 1, and c and d be the rotation parameters.

4. The method for detecting plug-in insertion anomalies according to claim 3, characterized in that, The step of multiplying the first matrix corresponding to the first image with the affine transformation matrix to obtain the position-corrected first image includes: Perform matrix multiplication using the following formula: ； in, The first image after position correction is shown, where A is the first matrix and P is the affine transformation matrix, wherein the affine transformation matrix is ​​expressed as: ； in, and Let a and b be the translation parameters, a and b be the scaling parameters and both be 1, and c and d be the rotation parameters.

5. The method for detecting plug-in insertion anomalies according to claim 1, characterized in that, The comparison process includes image difference processing.

6. The method for detecting plug-in insertion anomalies according to claim 1, characterized in that, The step of determining whether the plug-in insertion is abnormal based on the geometric or structural features of the two furthest pins in the pin image includes: Based on binarization processing, the contour information of the two furthest pins in the pin image is obtained; The geometric or structural features of the two furthest pins are determined based on the contour information; The geometric or structural features of the two pins furthest apart are respectively input into a preset first support vector machine and a second support vector machine for classification processing to obtain the detection result of whether the insertion is abnormal.

7. The method for detecting plug-in insertion anomalies according to claim 1, characterized in that, The method further includes: The first image is processed using a rotating caliper algorithm to determine the two insertion regions that are furthest apart in the first image; The two furthest pins in the pin image are determined based on the two furthest insertion areas in the first image.

8. A processor, characterized in that, The method described in any one of claims 1-7.

9. A detection camera, characterized in that, Includes the processor as described in claim 8.

Images