Pin defect detection method, device and equipment and computer readable storage medium

Abstract

The embodiment of the application discloses a PIN needle defect detection method, device and equipment and a computer readable storage medium. The method comprises the following steps: obtaining a depth map of a product to be tested by a 3D line laser camera, wherein the product to be tested comprises at least one PIN needle and at least two feature marks; the at least two feature marks are used to establish a reference coordinate system; converting floating-point image data of the depth map into integer image data; fitting a reference surface according to the integer image data corresponding to a reference surface area of the PIN needle and a reference surface area of the feature mark, respectively; establishing a reference coordinate system on the reference surface according to the integer image data and the area of the feature mark; obtaining the height and position degree of each PIN needle in the reference coordinate system according to the area of the PIN needle, wherein the position degree comprises an X-axis position degree and a Y-axis position degree; and obtaining a detection result of whether each PIN needle is normal according to the height and position degree of each PIN needle. The method has small calculation amount and can improve the accuracy and efficiency of PIN needle defect detection.

Description

Technical Field

[0001] This application relates to the field of defect detection technology, specifically to a method, apparatus, device, and computer-readable storage medium for detecting PIN defects. Background Technology

[0002] Pins are a common component in electronic devices used to connect external devices. They are made of metal and have the advantages of reliable communication connections and easy assembly / disassembly. These advantages make them widely used in industries such as printed circuit boards (PCBs), 3C (computer, communication and consumer electronics) and automotive electronics.

[0003] Devices that accept PIN pins have corresponding holes. The PIN pins are inserted into the holes to make connections. With the rapid development of application industries, the quality requirements for PIN pins are getting higher and higher.

[0004] Deviations in the height and position of the pins can lead to problems such as abnormal insertion and communication, resulting in unstable information transmission. Therefore, pin defect detection is particularly important.

[0005] In existing technologies, images of PIN pins are obtained using two sets of 2D cameras. This method is complex and requires integrating the images from both sets of 2D cameras, resulting in low efficiency. Furthermore, fluctuations in the relative shooting positions of the two sets of 2D cameras and the product significantly affect the detection results, leading to inaccurate PIN pin defect detection. Summary of the Invention

[0006] In view of this, embodiments of this application provide a method, apparatus, device, and computer-readable storage medium for detecting PIN pin defects, which can improve the accuracy and efficiency of PIN pin defect detection.

[0007] In a first aspect, embodiments of this application provide a method for detecting defects in PIN pins, including:

[0008] A depth map of the product under test is obtained using a 3D line laser camera. The product under test includes at least one pin and at least two feature marks. The at least two feature marks are used to establish a reference coordinate system.

[0009] Convert the floating-point image data of the depth map to integer image data;

[0010] A reference plane is fitted based on the integer image data corresponding to the reference plane area of ​​the PIN pin and the reference plane area of ​​the feature mark, and a reference coordinate system is established on the reference plane based on the integer image data and the area of ​​the feature mark.

[0011] In the reference coordinate system, the height and position of each PIN pin are obtained according to the region of the PIN pin. The position includes the X-axis position and the Y-axis position. Based on the height and position of each PIN pin, the detection result of whether each PIN pin is normal is obtained.

[0012] Secondly, embodiments of this application also provide a PIN pin defect detection device, comprising:

[0013] The module is used to acquire a depth map of the product under test using a 3D line laser camera. The product under test includes at least one pin and at least two feature marks; the at least two feature marks are used to establish a reference coordinate system.

[0014] The preprocessing module is used to convert the floating-point image data of the depth map into integer image data;

[0015] The fitting module is used to fit the reference plane based on the integer image data corresponding to the reference plane area of ​​the PIN pin and the reference plane area of ​​the feature mark, respectively.

[0016] The construction module is used to establish a reference coordinate system on the reference plane based on the region of integer image data and feature marks;

[0017] The calculation module is used to obtain the height and position of each PIN pin according to the region of the PIN pin in the reference coordinate system. The position includes the X-axis position and the Y-axis position.

[0018] The detection module is used to obtain the detection result of whether each PIN is normal based on the height and position of each PIN.

[0019] Thirdly, embodiments of this application also provide a computer device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of the above method.

[0020] Fourthly, embodiments of this application also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above method.

[0021] Fifthly, embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the above methods.

[0022] Therefore, the embodiments of this application have the following beneficial effects:

[0023] The PIN pin defect detection method provided in this application utilizes a 3D line laser camera to obtain three-dimensional data of the product under test from a single image. The depth value of the depth map can directly reflect the PIN pin height, and the distance between the highest point of the PIN pin and the reference surface (i.e., the distance between a point and a surface) is used to obtain the PIN pin height. The positional accuracy is obtained by using the distance between the PIN pin's coordinates and the X and Y axes of the reference coordinate system. Furthermore, to utilize mature 2D algorithms, this application first converts floating-point data to integer data, simplifying and increasing accuracy. Compared to traditional 2D vision, the method provided in this application only processes one image, resulting in lower computational load. Moreover, a single camera can simultaneously detect positional accuracy and height, improving the accuracy and efficiency of PIN pin defect detection. Attached Figure Description

[0024] Figure 1 This is an application scenario diagram of a PIN pin defect detection method provided in an embodiment of this application;

[0025] Figure 2 This is a schematic diagram of a tooling for acquiring PIN pin images, provided in an embodiment of this application.

[0026] Figure 3 A simplified top view of a product under test provided in an embodiment of this application;

[0027] Figure 4 A flowchart illustrating a PIN pin defect detection method provided in this application embodiment;

[0028] Figure 5 A PIN depth map of a product under test provided in this application embodiment;

[0029] Figure 6 A schematic diagram of a reference surface fitting provided for an embodiment of this application;

[0030] Figure 7 A schematic diagram of a position correction principle provided in an embodiment of this application;

[0031] Figure 8 A schematic diagram of PIN pin calibration provided in this application embodiment;

[0032] Figure 9 A schematic diagram of a PIN pin defect detection device provided in an embodiment of this application;

[0033] Figure 10 This application provides an internal structural diagram of a computer device;

[0034] Figure 11 An internal structural diagram of another computer device provided in this application embodiment;

[0035] Figure 12 This application provides an internal structure diagram of a computer-readable storage medium. Detailed Implementation

[0036] To facilitate understanding of the technical solutions in the embodiments of this application, the technical terms involved in the embodiments of this application will be explained below.

[0037] 3D line laser camera: A three-dimensional camera that captures laser line information projected onto the surface of an object by a laser generator using one or more image sensors, and reconstructs the contour information of the object's surface based on the principle of triangulation. Each laser point contains rich measurement information such as coordinates and brightness.

[0038] Depth map: An image formed by sampling the surface of a target object in the X and Y directions and converting the Z direction information into grayscale or color.

[0039] XY-axis resolution: The actual physical scale (unit: mm) that a 3D camera can resolve along the XY coordinate axes.

[0040] Z-axis resolution / depth resolution: The actual physical scale (unit: mm) that a 3D camera can resolve along the Z-axis.

[0041] Dynamic repeatability: The corresponding measurement item of a piece of material is tested 32 times with pick-up and drop, and the data fluctuation of the corresponding measurement item is observed. The smaller the data fluctuation, the better the dynamic repeatability.

[0042] ROI: Region of Interest. In machine vision, the region to be processed from the image is delineated using shapes such as rectangles, circles, ellipses, and irregular polygons. This region is called the Region of Interest.

[0043] Grayscale thresholding: This method categorizes all brightness values ​​in an image into two classes based on a specified brightness value (i.e., threshold): those within the threshold range and those outside the threshold range.

[0044] Morphological erosion: Mathematical morphology is an image analysis discipline built upon lattice theory and topology, and it forms the fundamental theory of mathematical morphological image processing. Its basic operations include: binary erosion and dilation, binary opening and closing operations, skeleton extraction, limit erosion, hit-and-miss transformation, morphological gradient, Top-hat transformation, grain analysis, watershed transformation, gray-value erosion and dilation, gray-value opening and closing operations, and gray-value morphological gradients. Erosion operations are generally used to remove burrs.

[0045] Median filtering: Median filtering is a nonlinear signal processing technique based on sorting statistics that can effectively suppress noise. The basic principle of median filtering is to replace the value of a point in a digital image or digital sequence with the median value of all points in a neighborhood of that point, so that the surrounding pixel values ​​are close to the true value, thereby eliminating isolated noise points.

[0046] The PIN pin defect detection method provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a communication network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104 or located in the cloud or on other network servers. Terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, etc. Portable wearable devices can include smartwatches, smart bracelets, head-mounted devices, etc. Server 104 can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0047] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the embodiments of this application will be further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0048] First, the inspection fixture for the PIN pin defect detection method provided in the embodiments of this application is introduced.

[0049] See Figure 2 The figure is a schematic diagram of a tooling for acquiring PIN needle images provided in an embodiment of this application.

[0050] This application does not specifically limit the type of product under test; any product including PIN pins is acceptable. For ease of understanding, this application uses a transformer as the product under test, specifically detecting whether the transformer's PIN pins are defective. PIN pin defect detection includes the height and position of each PIN pin, i.e., the corresponding x and y coordinates.

[0051] Figure 2This is merely an exemplary data acquisition fixture, including X and Y axes. A gripper grasps the product under test 10, positioned along the X-axis. A 3D line laser camera 20 is fixedly mounted, and the gripper can move along the Y-axis. The product under test 10 is scanned once within the field of view of the 3D line laser camera 20. The 3D line laser camera 20 transmits the depth map data of the product under test 10 to the memory of an industrial personal computer (IPC) via a network cable. The IPC can analyze the depth map data, set height and position thresholds according to the pin specifications, determine whether the pins are qualified, and finally obtain the test results. The test results can be displayed on the interface or transmitted to the background monitoring system.

[0052] To facilitate understanding, we will first use the PIN pins of a transformer as an example, with reference to the attached diagram. See also... Figure 3 The figure is a schematic diagram of the PIN pins of a product under test provided in an embodiment of this application.

[0053] Figure 3 This is merely a top view of the pins. Taking a transformer with four pins to be tested as an example, denoted by A, B, C, and D respectively, this application does not specifically limit the number of pins. The product under test includes at least one pin to be tested. Furthermore, the positional relationship between the pins is not specifically limited. This application detects defects in each pin. Additionally, the cross-sectional shape of the pins is not specifically limited.

[0054] In addition, feature markers need to be set in order to establish a reference plane during detection. This application does not specifically limit the type of feature markers. They can be set according to user needs. For example, they can be feature points, feature columns, or other distinctive types. Figure 3 The following description uses two feature pillars on the product under test 10 as examples, namely the first feature pillar 11 and the second feature pillar 22. It should be understood that both the first feature pillar 11 and the second feature pillar 22 have a certain height, and for ease of calculation, their heights are the same. This application does not specifically limit the relationship between the height of the first feature pillar 11 and the second feature pillar 22 and the PIN pin.

[0055] This application does not specifically limit the positional relationship between the two positioning posts and the PIN pins to be tested. In order to obtain more accurate data, for example, one possible approach is that the first positioning post is located within the space surrounded by all the PIN pins, and the second positioning post is located outside the space surrounded by all the PIN pins; the PIN pins are the PIN pins of the transformer.

[0056] It should be understood that the images captured by the 3D line laser camera contain a lot of information. This application is only for detecting PIN pins. Therefore, it is sufficient to select a portion of the captured image. For ease of description, this portion can be referred to as the core region. The core region includes four PIN pin regions that need to be measured, two positioning post regions for constructing positional references, and at least two regions for constructing reference surfaces.

[0057] The implementation process of a PIN pin defect detection method provided in this application is described in detail below with reference to the accompanying drawings.

[0058] See Figure 4 This figure is a flowchart of a PIN pin defect detection method provided in an embodiment of this application. The PIN pin defect detection method provided in this embodiment includes:

[0059] S401: Obtain a depth map of the product under test using a 3D line laser camera. The product under test includes at least one pin and at least two feature marks; the at least two feature marks are used to establish a reference coordinate system.

[0060] It should be understood that this application obtains a 3D stereoscopic image of the product under test, not a two-dimensional image obtained by a traditional 2D camera. Using a 3D line laser camera, only one image is needed to determine the height and position of the PIN pins based on the depth map data. For a specific imaging method, please refer to [link to relevant documentation]. Figure 2 The details of the introduction will not be repeated here.

[0061] Calculating the height and position of the PIN requires obtaining the coordinates of both height and position. Position refers to the X and Y coordinates of the PIN, while height corresponds to the Z coordinate. A reference coordinate system is needed for these coordinate values. Furthermore, a 3D stereoscopic image requires a reference plane; otherwise, if the data is not based on a single plane, plane tilt may occur, leading to measurement deviations. Therefore, this application requires establishing a reference plane based on at least two feature marks on the product under test.

[0062] S402: Converts the floating-point image data of the depth map into integer image data.

[0063] Traditionally, the grayscale values ​​of depth map data are converted into floating-point image values ​​(i.e., converted into actual physical height values) for subsequent processing. However, this processing cannot effectively utilize the rich algorithm library in 2D vision, such as shape module matching. Meanwhile, 3D algorithm libraries are limited. This application converts floating-point image data into integer image data. Specifically, the grayscale value in the depth map is divided by the Z-axis resolution and rounded to the nearest integer to obtain the grayscale value at the current position. For example, if the Z-axis resolution is 0.005mm, and the grayscale value at a certain position (x1, y1) is 1.005mm, then the floating-point image data of the depth map is converted to integer image data (201), resulting in a depth map with integer grayscale values. This allows for the use of a rich 2D vision algorithm library for subsequent processing of the depth map. The Z-axis resolution mentioned above is only an example; other resolutions can also be set.

[0064] Since the height and position of the PIN pins ultimately need to be output as actual physical values, subsequent processing requires unit conversion. During preprocessing, the resolution values ​​of the X-axis, Y-axis, and Z-axis and the integer image data can be combined into a single data type for convenient subsequent use.

[0065] Since the X-axis resolution of a 3D line laser camera is a fixed parameter determined by the camera itself, while the Y-axis resolution varies with the scanning speed, inconsistencies between the X-axis and Y-axis resolutions can occur. Typically, Y-axis resolution is sacrificed to increase scanning speed, resulting in lower accuracy for the Y-axis compared to the X-axis resolution. This leads to compression or stretching in the final image. For example, an object that should be circular might appear as an ellipse, hindering subsequent calculations. To address this issue, this application performs image scaling. Based on the ratio between the X-axis and Y-axis resolutions, the original depth map data is scaled. For instance, if the original integer image data is 1024 columns * 500 rows, with an X-axis resolution of 0.015 mm and a Y-axis resolution of 0.03 mm (a 2:1 ratio), the nearest neighbor interpolation algorithm is used to enlarge the image from 1024 columns * 500 rows to 1024 columns * 1000 rows. At this point, both the X and Y resolutions are 0.015 mm, ensuring the final image accurately reflects the shape of the actual object.

[0066] S403: Fit a reference plane based on the integer image data corresponding to the reference plane area of ​​the PIN pin and the reference plane area of ​​the feature mark, and establish a reference coordinate system on the reference plane based on the integer image data and the area of ​​the feature mark.

[0067] Because the pose of the product under test varies, the values ​​at different positions on the reference plane are not completely consistent and the differences are significant. Figure 5The top left and bottom right corners are both reference plane areas, but the significant difference in grayscale values ​​indicates a noticeable tilt in the product. To ensure the accuracy of PIN detection, the area around the PIN and feature marks should be selected as much as possible when fitting the reference plane. Specifically, the area can be selected in all directions (up, down, left, right) of the PIN. For example... Figure 6 As shown, five regions (1-5) can be selected to fit the datum plane, ensuring the accuracy of the fitted datum plane. The above example uses only five regions; the number of regions can be adjusted according to actual needs. For instance, selecting more regions will result in a more accurate fitting result.

[0068] It should be understood that the reference plane can be customized or it can be a solid surface on the product under test.

[0069] S404: In the reference coordinate system, obtain the height and position of each PIN according to the region of the PIN, including the position of the X-axis and the position of the Y-axis; obtain the detection result of whether each PIN is normal based on the height and position of each PIN.

[0070] Before calculating the height and position, preprocessing can be performed to filter out some noise. This is because the pin should be the highest region within the pin region, i.e., the region with the largest z-value. Specifically, morphological erosion can be used to reduce the size of the pin region, then median filtering can be used to remove outlier noise. Next, the maximum height (maxZ) within the pin region is calculated. Then, using a lower limit of maxZ - offset (offset is a preset parameter), a secondary threshold segmentation is performed on the point set of the pin region to extract a new region. This extracted new region more accurately reflects the pin region.

[0071] Specifically, obtaining the height and position of the PIN requires acquiring the final actual physical values, which are then compared with corresponding thresholds to determine if defects exist. Therefore, the height and position obtained from the reference coordinate system need to be converted into actual physical values. The calculated height is multiplied by the Z-axis resolution to obtain the physical height value. Using the PIN's coordinates (X, Y), the distance between the point and line from the Y-axis of the reference coordinate system is calculated as the PIN's X-axis position and the distance between the point and line from the Y-axis of the reference coordinate system are calculated as the PIN's Y-axis position. Finally, these are multiplied by the X-axis and Y-axis resolutions respectively to obtain the physical values ​​of the X-axis and Y-axis positions.

[0072] The PIN pin defect detection method provided in this application utilizes a 3D line laser camera to obtain three-dimensional data of the product under test from a single image. The depth value of the depth map can directly reflect the PIN pin height, and the distance between the highest point of the PIN pin and the reference surface (i.e., the distance between a point and a surface) is used to obtain the PIN pin height. The positional accuracy is obtained by using the distance between the PIN pin's coordinates and the X and Y axes of the reference coordinate system. Furthermore, to utilize mature 2D algorithms, this application first converts floating-point data to integer data, simplifying and increasing accuracy. Compared to traditional 2D vision, the method provided in this application only processes one image, resulting in lower computational load. Moreover, a single camera can simultaneously detect positional accuracy and height, improving the accuracy and efficiency of PIN pin defect detection.

[0073] Because the gripping posture of the product under test may change each time, the position of the product under test in the depth map fluctuates significantly, affecting the position of the test area, such as the positions of PIN pins, reference surfaces, and positioning posts. To quickly correct positional deviations, the method provided in this application embodiment can also obtain the positional deviation before testing, and use the positional deviation to correct the position of the product under test each time it is tested. A detailed description is provided below with reference to the accompanying drawings.

[0074] See Figure 7 This figure is a schematic diagram of a position correction principle provided in an embodiment of this application.

[0075] The method provided in this application embodiment, before fitting the reference surface based on the integer image data corresponding to the reference surface area of ​​the PIN pin and the reference surface area of ​​the feature mark, further includes:

[0076] The positions of the PIN pin area, the feature mark area, the PIN pin reference surface area, and the feature mark reference surface area are corrected according to the pre-obtained position deviations to obtain the corrected PIN pin area, the corrected feature mark area, the corrected PIN pin reference surface area, and the corrected feature mark reference surface area; the position deviations include X-axis position difference, Y-axis position difference, and angle difference.

[0077] This also includes obtaining the positional deviation in advance through the following methods:

[0078] Create a shape template based on the depth map;

[0079] Define the search area, which includes the area of ​​the PIN pins, the area of ​​the feature marks, the reference plane area of ​​the PIN pins, and the reference plane area of ​​the feature marks.

[0080] Use shape templates within the search area to find new locations;

[0081] The positional deviation is obtained based on the new position and the reference position.

[0082] The first step is to create a reference template position in the depth map using a shape template algorithm, such as... Figure 7 The triangle M is placed in the center, and the PIN pin area is set as follows: Figure 7 Circle 33 is placed upright. The search area, or ROI, is defined as including the area of ​​the PIN pins, the area of ​​the feature markers, the reference surface area of ​​the PIN pins, and the reference surface area of ​​the feature markers. The relative position of the ROI to the reference position is fixed.

[0083] The second step, as Figure 7 The tilted triangle N and the circle 44 define the search area, for example... Figure 7 The shape template search is used within the dashed box. After a successful search, a new position is obtained. The affine transformation matrix can be calculated using the reference position and the new position, which is to calculate the position deviation: the X-axis position difference Δx, the Y-axis position difference Δy, and the angle difference Δθ. Using the above position deviations, the corrected PIN area, i.e., circle 44, can be obtained. This area is the actual location of the PIN pin.

[0084] By correcting the above-mentioned positions in this application, the requirements for shooting equipment can be reduced, thereby lowering costs.

[0085] The following section details the process of fitting the reference surface.

[0086] The reference plane is fitted based on the integer image data corresponding to the reference plane area of ​​the PIN pin and the reference plane area of ​​the feature mark, respectively, including:

[0087] The preprocessed data is obtained by removing data with gray values ​​greater than the first threshold from the integer image data corresponding to the reference surface area of ​​the PIN pin and the reference surface area of ​​the feature mark, respectively.

[0088] The data for the updated region is obtained using a morphological erosion algorithm based on the preprocessed data;

[0089] Based on the data in the updated region, a median filtering algorithm is used to remove noise points, resulting in the final fitted data;

[0090] The reference surface is fitted using a fitting plane algorithm based on the fitted data.

[0091] above Figure 6The method involves selecting five regions to fit a reference surface. These regions typically contain noise data, and some regions have significant differences in height values ​​compared to their surroundings. In reality, the values ​​at these locations should be similar. Directly fitting the plane would result in a large fitting error. Therefore, preprocessing is performed first. Invalid points are removed using a set threshold lower limit, and regions that meet the set threshold are extracted. Then, a morphological erosion algorithm is used to narrow down the region range and remove noise-prone areas. The median filtering algorithm is then used on the point set within the region to remove noise points with large numerical fluctuations. Finally, a plane fitting algorithm is used to fit the reference surface.

[0092] The general equation for a plane in a three-dimensional coordinate system is Ax + By + Cz + D = 0. To determine the unknown coefficients, we can transform the equation to: z = -A / C*x, B / C*y, D / C = αx + βy + γ. Substituting all point sets into this equation yields a system of equations. Then, obtaining the least-squares solution for α, β, and γ gives the parameters of the fitted reference plane. Finally, subtracting the reference plane image from the original image yields a new image; the grayscale value of the new image represents the height value based on the reference plane.

[0093] The establishment of the reference coordinate system is described below.

[0094] This application embodiment uses two positioning posts as an example, that is, at least two feature marks include a first positioning post and a second positioning post;

[0095] A reference coordinate system is established on the reference plane based on the region of the integer image data and feature marks, including:

[0096] On the reference plane, based on the integral image data, the first positioning post is used as the origin of the coordinate system. The X-axis is determined by the straight line defined by the first and second positioning posts, and the Y-axis passes through the origin and is perpendicular to the X-axis. For example, the X-axis can be obtained by rotating the straight line defined by the first and second positioning posts counterclockwise by 37.6° around the first positioning post. This is just an example; the X-axis can be rotated by other degrees or not at all, depending on the actual product.

[0097] The first positioning region including the first positioning post and the second positioning region including the second positioning post are respectively divided by a second threshold segmentation to obtain the first positioning post region and the second positioning post region.

[0098] The average Z-value and centroid of the first positioning post area are used as the Z-value and position XY value of the first positioning post, respectively; the average Z-value and centroid of the second positioning post area are used as the Z-value and position XY value of the second positioning post, respectively.

[0099] A reference coordinate system is established based on the Z value and position XY value of the first positioning post, and the Z value and position XY value of the second positioning post.

[0100] Specifically, after establishing the coordinate axes, the key is to accurately obtain the coordinates of the first and second positioning pillars. First, invalid values ​​within the defined positioning pillar ROI are filtered out using a lower threshold, extracting regions that meet the threshold. Then, morphological erosion is used to reduce the region size, removing noise-generating parts. Median filtering is used to remove noise from abnormally fluctuating values. The maximum value (maxZ1) within this region is then calculated. A second threshold segmentation is performed on the region's point set using a lower limit of maxZ - offset1 (offset1 is a set parameter) to extract new regions. This strategy is used because the positioning pillar is actually the highest point in the region, allowing for the extraction of more realistic positioning pillar regions. The region with the largest area that meets the conditions is selected as the positioning pillar region. The average Z-value of this region is then calculated as the Z-value of the positioning pillar, and the centroid (X, Y) of the region is used as the positioning pillar coordinates. After obtaining the coordinates of the two positioning pillars, a reference coordinate system is established based on their coordinates.

[0101] The following describes how to obtain the height and position of the PIN pins.

[0102] In the reference coordinate system, obtain the height and position of each PIN pin according to the region of the PIN pin, including:

[0103] When the area of ​​the PIN pin region is greater than or equal to the preset area threshold, the PIN pin region is determined to be qualified, and the average Z value and centroid of the PIN pin region are used as the height and position of the PIN pin, respectively.

[0104] When the area of ​​the PIN pin region is less than the preset area threshold, the area of ​​the remaining region after subtracting the PIN pin region from the position-corrected PIN pin region is taken as the area difference. The iteration calculation is stopped when the area difference is greater than the preset area threshold. The remaining region corresponding to the maximum area difference in all iterations is taken as the PIN pin calibration region. The average Z value and centroid of the PIN pin calibration region are taken as the height and position of the PIN pin, respectively.

[0105] First, invalid values ​​of the ROI of the pin are filtered using a lower threshold to extract regions that meet the lower threshold. Then, morphological erosion is used to reduce the region size, followed by median filtering to remove outlier noise. The maximum value (maxZ2) within this region is then calculated. A second threshold segmentation is performed on the region's points using a lower threshold of maxZ - offset2 (offset2 is a set parameter) to extract new regions. This strategy is used because the pin is actually the highest point in the region, allowing for a more realistic pin extraction. The region with the largest area that meets the criteria is taken as the pin. However, some pins may have local high points, possibly due to manufacturing variations causing unevenness on the top surface. Figure 8 As shown, for the two PIN pins A and B, the cross position indicates the calculated position degree, which is relatively accurate. However, the cross extraction is abnormal, and the cross of B deviates from the actual PIN position.

[0106] To address the above issues, an area control mechanism is added, setting a preset area threshold. When the area of ​​the selected PIN pin region is greater than or equal to the preset area threshold, it is designated as the PIN pin region. When it is less than the preset area threshold, the ROI of the PIN pin is subtracted from that region, and the iterative calculation is repeated in the remaining region. The iterative calculation stops when the area exceeds the preset area threshold. If the preset area threshold is not met after a certain number of iterations (the number of iterations can be set according to the required calculation accuracy), the remaining region corresponding to the largest area difference among all iterations is used as the PIN pin region. Finally, the average Z-value of this region is calculated as the Z-value of the PIN pin, and the centroid (x, y) of the region is used as the PIN pin position.

[0107] Then, multiplying the Z value by the Z-axis resolution yields the actual physical height value. The PIN pin position (x, y) is calculated by taking the distance from the Y-axis dot line as the X-axis position and the distance from the Y-axis dot line as the Y-axis position, respectively. Finally, these are multiplied by the X-axis resolution and the Y-axis resolution to obtain the physical values ​​of the X-axis and Y-axis positions.

[0108] The PIN pin defect detection method provided in this application embodiment can measure the height and position of the PIN pin, achieving high measurement accuracy, such as a height measurement accuracy of 0.005mm and dynamic repeatability of less than 0.015mm; that is, when repeatedly performing PIN pin defect detection on the product under test, the deviation from the previous one is less than 0.05mm, indicating good dynamic repeatability.

[0109] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps.

[0110] Based on the same inventive concept, this application also provides a PIN pin defect detection device. The solution provided by this device is similar to the solution described in the above method. Therefore, the specific limitations of one or more PIN pin defect detection device embodiments provided below can be found in the limitations of the PIN pin defect detection method above, and will not be repeated here.

[0111] like Figure 9 As shown in the figure, this figure is a schematic diagram of a PIN pin defect detection device provided in an embodiment of this application.

[0112] The PIN pin defect detection device provided in this application includes:

[0113] Module 901 is used to obtain a depth map of the product under test using a 3D line laser camera. The product under test includes at least one pin and at least two feature marks; the at least two feature marks are used to establish a reference coordinate system.

[0114] Preprocessing module 902 is used to convert the floating-point image data of the depth map into integer image data;

[0115] The fitting module 903 is used to fit the reference surface based on the integer image data corresponding to the reference surface area of ​​the PIN pin and the reference surface area of ​​the feature mark, respectively.

[0116] Module 904 is used to establish a reference coordinate system on the reference plane based on the region of the integer image data and feature marks;

[0117] The calculation module 905 is used to obtain the height and position of each PIN pin according to the region of the PIN pin in the reference coordinate system. The position includes the X-axis position and the Y-axis position.

[0118] The detection module 906 is used to obtain the detection result of whether each PIN is normal based on the height and position of each PIN.

[0119] In some embodiments, in fitting a reference plane based on the integer image data corresponding to the reference plane region of the PIN pin and the reference plane region of the feature mark, the fitting module 903 is specifically used for:

[0120] The preprocessed data is obtained by removing data with gray values ​​greater than the first threshold from the integer image data corresponding to the reference surface area of ​​the PIN pin and the reference surface area of ​​the feature mark, respectively.

[0121] The data for the updated region is obtained using a morphological erosion algorithm based on the preprocessed data;

[0122] Based on the data in the updated region, a median filtering algorithm is used to remove noise points, resulting in the final fitted data;

[0123] The reference surface is fitted using a fitting plane algorithm based on the fitted data.

[0124] In some embodiments, in establishing a reference coordinate system on a reference plane based on integer image data and feature marks, the construction module 904 is specifically configured to:

[0125] On the reference plane, based on the integer image data, the first positioning post is used as the origin of the coordinate system, and the straight line determined by the first positioning post and the second positioning post is used to determine the X-axis. The Y-axis passes through the origin and is perpendicular to the X-axis.

[0126] The first positioning region including the first positioning post and the second positioning region including the second positioning post are respectively divided by a second threshold segmentation to obtain the first positioning post region and the second positioning post region.

[0127] The average Z-value and centroid of the first positioning post area are used as the Z-value and position XY value of the first positioning post, respectively; the average Z-value and centroid of the second positioning post area are used as the Z-value and position XY value of the second positioning post, respectively.

[0128] A reference coordinate system is established based on the Z value and position XY value of the first positioning post, and the Z value and position XY value of the second positioning post.

[0129] In some embodiments, the calculation module 905 is specifically used to: obtain the height and position of each PIN pin according to the region of the PIN pin in the reference coordinate system.

[0130] When the area of ​​the PIN pin region is greater than or equal to the preset area threshold, the PIN pin region is determined to be qualified, and the average Z value and centroid of the PIN pin region are used as the height and position of the PIN pin, respectively.

[0131] When the area of ​​the PIN pin region is less than the preset area threshold, the area of ​​the remaining region after subtracting the PIN pin region from the position-corrected PIN pin region is taken as the area difference. The iteration calculation is stopped when the area difference is greater than the preset area threshold. The remaining region corresponding to the maximum area difference in all iterations is taken as the PIN pin calibration region. The average Z value and centroid of the PIN pin calibration region are taken as the height and position of the PIN pin, respectively.

[0132] In some embodiments, the PIN pin defect detection device further includes a correction module, which is used to perform position correction on the area of ​​the PIN pin, the area of ​​the feature mark, the reference surface area of ​​the PIN pin, and the reference surface area of ​​the feature mark according to a pre-obtained position deviation, so as to obtain the area of ​​the PIN pin, the area of ​​the feature mark, the reference surface area of ​​the PIN pin, and the reference surface area of ​​the feature mark after position correction; the position deviation includes X-axis position difference, Y-axis position difference, and angle difference.

[0133] In some embodiments, the PIN pin defect detection device further includes a position deviation module for creating a shape template based on a depth map; setting a search area including the PIN pin region, the feature mark region, the PIN pin reference surface region, and the feature mark reference surface region; using the shape template to search within the search area to obtain a new position; and obtaining a position deviation based on the new position and the reference position.

[0134] In some embodiments, the first positioning post is located within the space surrounded by all the pins, and the second positioning post is located outside the space surrounded by all the pins; the pins are the pins of the transformer.

[0135] Each module in the aforementioned PIN pin defect detection device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0136] In some embodiments, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 10 As shown, the computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs in the non-volatile storage media. The database stores the original images of the product under test. The I / O interfaces are used for information exchange between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements the steps in the aforementioned PIN pin defect detection method.

[0137] In some embodiments, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 11As shown, the computer device includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The input / output interface is used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements the steps in the aforementioned PIN pin defect detection method. The display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen; the input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs or touchpads set on the casing of the computer device, or external keyboards, touchpads or mice, etc.

[0138] Those skilled in the art will understand that Figure 10 or Figure 11 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0139] In some embodiments, a computer device is provided, the computer device including a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the steps in the above method embodiments.

[0140] In some embodiments, such as Figure 12 The diagram shows the internal structure of a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps described in the above-described method embodiments.

[0141] In some embodiments, a computer program product is provided, which includes a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0142] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0143] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0144] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0145] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for detecting defects in PIN pins, characterized in that, The method comprises the following steps: obtaining a depth map of a product to be measured by a 3D line laser camera, the product to be measured comprising at least one PIN needle and at least two feature marks, and the at least two feature marks being used to establish a reference coordinate system; converting floating-point image data of the depth map into integer image data; eliminating data with a gray value greater than a first threshold value from the integer image data corresponding to a reference surface area of the PIN needle and a reference surface area of the feature marks respectively to obtain preprocessed data; using a morphological erosion algorithm to obtain data of an updated area according to the preprocessed data; using a median filter algorithm to filter out noise points to obtain final fitting data according to the data of the updated area; and fitting a reference surface using a fitting plane algorithm according to the fitting data, and establishing a reference coordinate system on the reference surface according to the integer image data and the area of the feature marks; obtaining the height and position degree of each PIN needle in the reference coordinate system according to the area of the PIN needle, specifically including: when the area of the PIN needle is greater than or equal to a preset area threshold, determining that the area of the PIN needle is qualified, and taking the average value of Z values and the center of gravity of the area of the PIN needle as the height and position degree of the PIN needle respectively; when the area of the PIN needle is less than the preset area threshold, taking the area of a remaining area obtained by subtracting the area of the PIN needle from the area of the PIN needle after position correction as an area difference value, until the area difference value is greater than the preset area threshold, stopping iteration calculation, taking the remaining area corresponding to the maximum area difference value in all iteration times as a PIN needle calibration area, and taking the average value of Z values and the center of gravity of the PIN needle calibration area as the height and position degree of the PIN needle respectively, the position degree including X-axis position degree and Y-axis position degree; and obtaining a detection result of whether each PIN needle is normal according to the height and position degree of each PIN needle.

2. The method of claim 1, wherein, Before fitting a reference surface according to the integer image data corresponding to the reference surface area of the PIN needle and the reference surface area of the feature marks, the method further comprises the following steps: positionally correcting the area of the PIN needle, the area of the feature marks, the reference surface area of the PIN needle and the reference surface area of the feature marks according to a previously obtained position deviation, to obtain the area of the PIN needle after position correction, the area of the feature marks after position correction, the reference surface area of the PIN needle after position correction and the reference surface area of the feature marks after position correction; the position deviation including X-axis position difference, Y-axis position difference and angle difference.

3. The method of claim 2, wherein, The method further comprises the following steps of obtaining the position deviation in advance: creating a shape template according to the depth map; setting a search area including the area of the PIN needle, the area of the feature marks, the reference surface area of the PIN needle and the reference surface area of the feature marks; searching for a new position in the search area using the shape template; obtaining the position deviation according to the new position and a reference position.

4. The method of claim 1, wherein, The at least two feature marks include a first positioning column and a second positioning column; The establishing a reference coordinate system on the reference surface according to the integral image data and the region of the feature mark comprises: On the reference surface, according to the integral image data, taking the first positioning column as the coordinate system origin, determining the X axis by the straight line determined by the first positioning column and the second positioning column, and the Y axis passing through the origin and being perpendicular to the X axis; Respectively performing secondary threshold segmentation on a first positioning region including the first positioning column and a second positioning region including the second positioning column to obtain a first positioning column region and a second positioning column region; Taking the Z value average and the gravity center of the first positioning column region as the Z value and the position XY value of the first positioning column respectively, and taking the Z value average and the gravity center of the second positioning column region as the Z value and the position XY value of the second positioning column respectively; Establishing a reference coordinate system according to the Z value and the position XY value of the first positioning column and the Z value and the position XY value of the second positioning column.

5. The method of claim 4, wherein, The first positioning column is located in a space surrounded by all the PIN pins, and the second positioning column is located outside the space surrounded by all the PIN pins; the PIN pins are PIN pins of a transformer.

6. A PIN defect detection apparatus, characterized by, Comprise: An obtaining module is configured to obtain a depth map of a product to be measured by a 3D line laser camera, the product to be measured comprising at least one PIN pin and at least two feature marks; the at least two feature marks are used to establish a reference coordinate system; A preprocessing module is configured to convert floating-point image data of the depth map into integral image data; A fitting module is configured to fit a reference surface according to the integral image data corresponding to a reference surface region of the PIN pin and a reference surface region of the feature mark respectively; to remove data with a gray value greater than a first threshold from the integral image data corresponding to the reference surface region of the PIN pin and the reference surface region of the feature mark respectively to obtain preprocessed data; to obtain data of an updated region using a morphological erosion algorithm according to the preprocessed data; to filter out noise points using a median filter algorithm according to the data of the updated region to obtain final fitting data; and to fit a reference surface using a fitting plane algorithm according to the fitting data; A constructing module is configured to establish a reference coordinate system on the reference surface according to the integral image data and the region of the feature mark. A computing module is configured to obtain a height and a position degree of each PIN in the reference coordinate system according to a region of the PIN, the position degree including an X-axis position degree and a Y-axis position degree; when an area of the region of the PIN is greater than or equal to a preset area threshold, it is determined that the region of the PIN is qualified, and a Z-value average and a barycenter of the region of the PIN are taken as the height and the position degree of the PIN respectively; when the area of the region of the PIN is less than the preset area threshold, an area of a remaining region after the region of the PIN is subtracted from a position-corrected region of the PIN is taken as an area difference value, until the area difference value is greater than the preset area threshold, an iteration is stopped, a remaining region corresponding to a maximum area difference value in all iteration times is taken as a PIN calibration region, and a Z-value average and a barycenter of the PIN calibration region are taken as the height and the position degree of the PIN respectively; A detecting module is configured to obtain a detection result of whether each PIN is normal according to the height and the position degree of each PIN. 7.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is characterized in that, The processor executes the computer program to implement the steps of the method in any one of claims 1 to 5.

8. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method in any one of claims 1 to 5.

Images