Modular pin visual detection method, detection server, medium and product

Abstract

The invention discloses a modularized pin visual detection method, a detection server, a medium and a product, and relates to the field of machine visual detection. The method comprises the following steps: acquiring a standard model of a connector; obtaining a product image through a visual detection module; calculating through an image recognition algorithm to obtain a first actual coordinate of an anchor point in the product image; according to the first actual coordinate and the first standard coordinate of the anchoring point, calculating to obtain an actual deviation angle of the placing position of the connector; establishing a detection coordinate system according to the actual deviation angle and the actual coordinate system; determining a scanning path of the visual detection module; according to the scanning path, obtaining a scanning image of each pin through a visual detection module; calculating a second actual coordinate of each pin according to the detection coordinate system and the scanning image; and performing anomaly detection on each pin to obtain a detection result. According to the technical scheme, the detection precision of the pins is effectively improved, and the problem that the detection precision of the pins is reduced due to deviation of the placement position of the connector is solved.

Description

Technical Field

[0001] The present application relates to the field of machine vision inspection, and in particular to a modular pin visual inspection method, inspection server, medium and product. Background Art

[0002] In the field of modern electronic manufacturing, pins are key components for connecting various electronic components, and their quality and performance stability are crucial. As electronic products develop towards miniaturization, high integration and high performance, the size of pins continues to decrease, but the precision requirements are increasing.

[0003] At present, the inspection of connector pins is usually carried out by collecting image information of the pins through one or more 2D scanning cameras above the pins, and then using computer vision algorithms to process and analyze the image information to obtain the position and height information of each pin, and then determine whether the position and height of the pin are within the error range, and then detect whether the pin is qualified.

[0004] However, since the camera's shooting angle and field of view are pre-set based on the standard placement of the connector, and the camera's imaging principle is based on a fixed optical geometric relationship, when the connector placement deviates, the pin's actual position in the camera coordinate system will not match the preset standard position, resulting in changes in the projection position and posture on the image plane. This change will cause deviations in the extraction and analysis of the pin's feature points in the image based on the preset algorithm, which will in turn cause deviations in the pin's position information and height information calculated through the image, making it easy to misjudge the pin when detecting it, reducing the pin's detection accuracy. Summary of the invention

[0005] The present application provides a modular pin needle visual detection method, detection server, medium and product, which calculates the offset angle of the actual placement position of the connector, establishes a new detection coordinate system based on the offset angle, and then calculates the actual coordinates of each pin needle in the connector based on the new detection coordinate system. Finally, the pin needle is detected by the calculated actual coordinates, thereby improving the accuracy of the calculation of the actual coordinates of the pin needle and thus improving the detection accuracy of the pin needle.

[0006] In a first aspect, the present application provides a pin needle visual detection method, comprising: obtaining a standard model of a connector, the standard model comprising a standard coordinate system, multiple anchor points, a first standard coordinate of each anchor point in the standard coordinate system, and a second standard coordinate of each pin needle in the connector in the standard coordinate system, the standard coordinate system is determined based on the anchor point position relationship between each anchor point in the standard model, the anchor point position relationship is fixed, the x-axis and y-axis of the second standard coordinate represent the position of the pin needle, and the z-axis represents the height of the pin needle; obtaining a product image of the connector through a visual detection module; calculating the first actual coordinates of each anchor point in the product image in the actual coordinate system through an image recognition algorithm, the actual coordinate system is determined based on the anchor point position relationship between each anchor point in the product image; calculating the actual offset angle of the connector placement position based on the first actual coordinates and the first standard coordinates of each anchor point; based on the actual offset angle and the actual coordinate system, A detection coordinate system is established, wherein the detection coordinate system is obtained by rotating the actual coordinate system according to the actual offset angle, and the third actual coordinates of each anchor point in the detection coordinate system match the first standard coordinates of the corresponding anchor point in the standard coordinate system, and the third actual coordinates are obtained by coordinate system transformation of the first actual coordinates according to the actual offset angle; according to the first standard coordinates of each anchor point in the standard model and the second standard coordinates of each pin needle, a scanning path of the visual detection module is determined; according to the scanning path, each pin needle in the connector is scanned in sequence by the visual detection module to obtain a scanned image of each pin needle; based on the detection coordinate system and the scanned image of each pin needle, the second actual coordinates of each pin needle in the connector are calculated; according to the second standard coordinates and the second actual coordinates of each pin needle in the connector, an abnormality detection is performed on each pin needle to obtain a detection result of each pin needle, and the anomaly detection includes position detection and height detection.

[0007] By adopting the above technical solution, according to the coordinates of each anchor point in the standard coordinate system and the coordinates in the actual coordinate system, the offset angle of the connector placement position is obtained by establishing a vector calculation, and then the actual coordinate system is rotated according to the offset angle to obtain the detection coordinate system, and then the actual coordinates of each pin needle in the detection coordinate system are calculated according to the detection coordinate system and the scanned image of each pin needle, so that the calculated actual coordinates can more accurately reflect the actual position and height of the pin needle. Accurate detection results can be obtained by detecting the pin needle through the actual coordinates, which can avoid the reduction of the accuracy of the detection result due to the deviation of the connector placement position, and effectively improve the detection accuracy of the pin needle.

[0008] In combination with some embodiments of the first aspect, in some embodiments, the scanning path of the visual inspection module is determined according to the first standard coordinates of each anchor point and the second standard coordinates of each pin needle, specifically including: according to the second standard coordinates of each pin needle in the connector, calculating through a path planning algorithm a pin needle detection sequence that can shorten the detection time; obtaining the second standard coordinates of the first pin needle detected in the pin needle detection sequence as the initial detection coordinates; according to the first standard coordinates of each anchor point and the initial detection coordinates, calculating the distance between each anchor point and the first pin needle detected in the pin needle detection sequence; selecting the first standard coordinates of the anchor point with the shortest distance to the first pin needle detected in the pin needle detection sequence as the reference coordinates, and the reference coordinates are used to accurately locate the actual position of each pin needle; determining the scanning path of the visual inspection module according to the initial coordinates of the visual inspection module in the standard coordinate system, the reference coordinates, the pin needle detection sequence and the first standard coordinates of each pin needle.

[0009] By adopting the above technical solution, the pin needle detection sequence that can shorten the detection time is calculated through the path planning algorithm, and the pin needles are detected according to the detection sequence, which reduces the total time of detecting each pin needle and improves the detection efficiency of the pin needle. At the same time, by selecting the first standard coordinates of the anchor point with the shortest distance from the first detected pin needle as the reference coordinates, the actual position of each pin needle is located more accurately and efficiently, the accuracy of collecting pin needle images is improved, and the accuracy of calculating the actual coordinates of the pin needle is improved. Since the scanning path is obtained according to the above detection sequence and reference coordinate planning, therefore, according to the scanning path, the image of the pin needle is collected, not only can the image of each pin needle be accurately collected, but also the time of collecting each pin needle image can be reduced, the efficiency of image collection is improved, and the detection efficiency of the pin needle is improved.

[0010] In combination with some embodiments of the first aspect, in some embodiments, the actual offset angle of the connector placement position is calculated based on the first actual coordinates and the first standard coordinates of each anchor point, specifically including: based on the first standard coordinates of each anchor point in the standard model, determining whether the anchor figure formed by each anchor point is a two-dimensional figure; if the anchor figure is a two-dimensional figure, determining the offset angle between the first actual coordinates and the first standard coordinates of each anchor point based on the first actual coordinates and the first standard coordinates of each anchor point; calculating the average value of all the offset angles to obtain the actual offset angle; if the anchor figure is not a two-dimensional figure, obtaining the standard center point coordinates and the actual center point coordinates of the anchor figure in the standard coordinate system; constructing an offset vector pointing from the standard center point coordinates to the actual center point coordinates; and calculating the actual offset angle based on the offset vector.

[0011] The above technical solution is adopted, and different calculation methods are used to calculate the actual offset angle for different anchoring graphics. When the anchoring graphic is a two-dimensional graphic, each anchor point is taken as a separate individual to calculate the offset angle of each anchor point, and then the average value of all offset angles is calculated to obtain the actual offset angle of the connector. This can accurately capture the subtle position changes of each anchor point caused by various factors, and weaken the influence of accidental errors by finding the average value of all offset angles, so that the final offset angle accurately reflects the actual placement state of the connector in the two-dimensional plane, providing a reliable basis for subsequent pin needle detection and improving detection accuracy. When the anchoring graphic is not a two-dimensional graphic, the entire anchoring graphic is taken as a whole to calculate the actual offset angle of the connector. Taking into account that the non-two-dimensional graphic has a complex three-dimensional structure, and the structure is more stable than the two-dimensional graphic, the overall calculation method can better reflect its actual displacement in space, avoiding the information fragmentation and error accumulation problems that may be caused by the isolated analysis of each anchor point in a complex three-dimensional structure, effectively improving the accuracy of calculating the offset angle of the connector placement position, and thus improving the detection accuracy of the pin needle.

[0012] In combination with some embodiments of the first aspect, in some embodiments, obtaining the standard model of the connector specifically includes: determining whether the standard model of the connector exists in the model library; if so, obtaining the standard model corresponding to the connector in the model library; if not, establishing the standard model based on the standard drawing of the connector.

[0013] By adopting the above technical solution, the standard model of the connector is directly obtained through the model library, which avoids repeatedly building the same model, reduces the time for building the model before each detection, and improves the pin detection efficiency.

[0014] In combination with some embodiments of the first aspect, in some embodiments, if the standard model does not exist, then based on the standard drawing of the connector, the standard model is established, specifically including: based on the standard drawing of the connector, generating a 3D model of the connector through a model generation module; obtaining the position information and height information of each pin needle in the 3D model through a point cloud alignment algorithm; generating an initial model based on the position information and the height information of each pin needle; obtaining the standard coordinate system and the anchor point marked by the operator in the initial model, and generating the standard model.

[0015] By adopting the above technical solution, when there is no standard model corresponding to the connector in the model library, the 3D model of the connector is directly generated based on the standard drawing of the connector, and the position information and height information of each pin in the 3D model are automatically obtained through the point cloud registration algorithm, and then the model is automatically established according to the position information and height information of each pin, which saves the time of manual model establishment and improves the efficiency of establishing standard models.

[0016] In combination with some embodiments of the first aspect, in some embodiments, after the step of performing abnormal detection on each pin pin according to the second standard coordinates and the second actual coordinates of each pin pin in the connector to obtain the detection results of each pin pin, the method also includes: when the detection results of all the pin pins of the connector are unqualified, determining the reasons for the unqualifiedness of each pin pin; when the reasons for the unqualifiedness of all the pin pins are the same, obtaining the operating parameters of the production equipment corresponding to the connector; determining whether the operating parameters exceed the normal range; if they exceed the normal range, sending an instruction to stop the equipment operation to the production equipment through the control module; if they do not exceed the normal range, sending a first prompt message to the operator, and the first prompt message is used to prompt the operator that the standard model of the connector may be inaccurate.

[0017] By adopting the above technical solution, when it is detected that all the pins of the connector are unqualified and the reasons for the unqualified are the same, it is determined whether the equipment operation is abnormal during the connector production by obtaining the operating parameters of the production equipment corresponding to the connector. If the equipment is abnormal, the equipment operation is stopped by the control module to prevent the equipment from continuing to produce unqualified connectors, thereby reducing unnecessary increase in production costs and ineffective consumption of production resources. If the equipment is normal, it means that the standard model used to detect the pins of the connector may be inaccurate. At this time, a prompt message is sent to the operator to remind the operator to promptly check whether the standard model is accurate to ensure that the detection process is carried out accurately.

[0018] In combination with some embodiments of the first aspect, in some embodiments, after the step of performing abnormal detection on each pin pin in the connector according to the second standard coordinates and the second actual coordinates of each pin pin to obtain the detection results of each pin pin, the method also includes: determining whether the actual offset angle exceeds a preset offset angle threshold; if so, based on a preset scanning path, obtaining a reference image of a preset reference point next to the connector through the visual inspection module; determining whether the reference image matches a pre-stored standard reference image of the reference point; if not, sending a second prompt message to the operator, and the second prompt message is used to prompt the operator that the fixed position of the visual inspection module is offset.

[0019] By adopting the above technical solution, when the actual offset angle exceeds the preset offset angle threshold, an image of a preset reference point is collected and matched with a pre-stored standard reference image of the reference point to determine whether the actual offset angle exceeds the preset offset angle threshold due to the offset of the fixed position of the visual inspection module. When it is determined that the fixed position of the visual inspection module is offset, a prompt message is immediately sent to the operator to remind the operator to adjust the visual inspection module in time to avoid untimely processing leading to an increase in the offset angle of the visual inspection module, thereby affecting the accuracy of the detection result.

[0020] In a second aspect, an embodiment of the present application provides a detection server, which includes: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the detection server to execute the method described in the first aspect and any possible implementation method of the first aspect.

[0021] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, comprising instructions, which, when executed on a detection server, enable the detection server to execute the method described in the first aspect and any possible implementation of the first aspect.

[0022] In a fourth aspect, the present application provides a computer program product. When the computer program product runs on a detection server, the detection server executes the method described in the first aspect and any possible implementation manner of the first aspect.

[0023] It is understandable that the electronic device provided in the second aspect, the storage medium provided in the third aspect, and the computer program product provided in the fourth aspect are all used to execute the method provided in the present invention. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method, which will not be repeated here.

[0024] One or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages: 1. This application accurately calculates the offset angle of the connector placement position through the standard coordinates and actual coordinates of the anchor point, and then rotates the actual coordinate system according to the offset angle to obtain the detection coordinate system, and then calculates the actual coordinates of each pin in the detection coordinate system according to the detection coordinate system and the scanned image of each pin, so that the calculated actual coordinates can more accurately reflect the actual position and height of the pin, and improve the accuracy of calculating the actual coordinates of the pin. At the same time, the pin is detected according to the calculated actual coordinates of the pin, so that the pin detection result is more accurate, avoiding the reduction of the accuracy of the detection result due to the deviation of the connector placement position, and effectively improving the detection accuracy of the pin.

[0025] 2. This application calculates the pin needle detection order that can minimize the detection time through the path planning algorithm, and detects the pin needles according to the detection order, which reduces the total time of detecting each pin needle and improves the detection efficiency of the pin needle. At the same time, by selecting the first standard coordinates of the anchor point with the shortest distance to the first detected pin needle as the reference coordinates, the actual position of each pin needle is located more accurately and efficiently, effectively improving the accuracy of collecting pin needle images, and then improving the accuracy of calculating the actual coordinates of the pin needle.

[0026] 3. The present application determines the shape of the anchoring figure through the coordinates of the anchoring point, and selects different offset angle calculation methods according to the shape of the anchoring figure to calculate the actual offset angle of the connector, comprehensively considering the characteristics of the two-dimensional figure and the non-two-dimensional figure, so that the calculated actual offset angle is more accurate, thereby improving the accuracy of calculating the actual coordinates of the pin needle. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 This is a schematic diagram of an application scenario of the pin needle visual detection method in an embodiment of the present application; Figure 2 It is a flow chart of a pin needle visual detection method in an embodiment of the present application; Figure 3 It is another flow chart of the pin needle visual detection method in the embodiment of the present application; Figure 4 It is a schematic diagram of an exemplary hardware structure of the detection server in an embodiment of the present application. DETAILED DESCRIPTION

[0028] The terms used in the following embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to be used as limitations to the present application. As used in the specification and appended claims of the present application, the singular expressions "one", "a kind of", "said", "above", "the" and "this" are intended to also include plural expressions, unless there is a clear indication to the contrary in the context. It should also be understood that the term "and / or" used in the present application refers to any or all possible combinations comprising one or more listed items.

[0029] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features, and in the description of the embodiments of the present application, unless otherwise specified, "plurality" means two or more.

[0030] Figure 1 It is a schematic diagram of an application scenario of the pin needle visual detection method in an embodiment of the present application.

[0031] See also Figure 1 In the figure, there is a connector with multiple pins on it. There are two scanning cameras above the pins. There is a certain angle between the two scanning cameras, which are used to collect images of different angles of each pin and transmit the collected images to the detection system.

[0032] Among them, the scanning camera can be but is not limited to a line laser scanning camera, an area array scanning camera, an optical imaging camera and other scanning cameras. The number of scanning cameras can be one or more. The placement of the scanning camera depends on actual needs and is not limited to only the one described above. It is not limited here.

[0033] After receiving the pin needle image transmitted by the scanning camera, the detection system will calculate the actual position and height of the pin needle based on the image, and then perform abnormal detection on the pin needle based on the calculated data. However, since the camera's shooting angle and field of view are pre-set based on the standard placement of the connector, and the camera's imaging principle is based on a fixed optical geometric relationship, when the connector placement deviates, the pin needle's actual position in the camera coordinate system will not match the preset standard position, resulting in changes in the projection position and posture on the image plane. This change will cause deviations in the extraction and analysis of the pin needle feature points in the image based on the preset algorithm, which will in turn cause deviations in the pin needle position information and height information calculated through the image, making it easy to misjudge the pin needle during detection, reducing the pin needle detection accuracy.

[0034] In order to ensure that the placement of the connector is consistent with the standard placement each time, the placement of the connector is usually fixed by making a mold. However, since the mold itself has a certain tolerance during the manufacturing and use process, and the mold will be worn during use, the placement of the connector in the mold will have a slight deviation. Therefore, the use of molds cannot completely avoid the situation where the placement of the connector deviates from the standard placement, thereby affecting the detection accuracy of the pin needle.

[0035] By adopting the pin pin visual detection method in the embodiment of the present application, multiple anchor points are set, and the actual coordinates of each anchor point are compared with the standard coordinates to calculate the offset angle of the connector placement position. A detection coordinate system is then established according to the offset angle. The actual coordinates of the pin pin are calculated by the detection coordinate system. The actual coordinates of the pin pin can be accurately calculated, and the pin pin is detected by the calculated actual coordinates, thereby avoiding the situation where the actual coordinate calculation of the pin pin due to deviation in the placement position, which may cause misjudgment during pin pin detection, and effectively improving the detection accuracy of the pin pin.

[0036] Combine the following Figure 2 To illustrate the method of the embodiment of the present application.

[0037] See also Figure 2 , which is a flow chart of the pin needle visual detection method in an embodiment of the present application.

[0038] S201. Obtain a standard model of a connector.

[0039] Establish a standard model of the connector based on the standard drawing of the connector.

[0040] Among them, the standard drawings record the size specifications, arrangement positions, shape structures and other information of each pin in the connector. The standard model includes a standard coordinate system, multiple anchor points, the first standard coordinates of each anchor point in the standard coordinate system, and the second standard coordinates of each pin in the connector in the standard coordinate system. The standard coordinate system is determined based on the anchor point position relationship between each anchor point in the standard model, and the anchor point position relationship is fixed. The x-axis and y-axis of the second standard coordinate represent the position of the pin, and the z-axis represents the height of the pin.

[0041] Specifically, the standard model is used to detect each pin in the connector. When establishing the standard model, it is necessary to first filter out the information related to the pin from the standard drawing of the connector, including the position of the pin, the height of the pin and other information. Then, based on the filtered pin-related information, use professional modeling software to create the model. When creating the model, it is necessary to ensure that the position and height of each pin are completely consistent with the position and height of the corresponding pin in the standard drawing. After the model is created, the operator is required to set the anchor point and the standard coordinate system on the model. Finally, based on the standard coordinate system set by the operator, calculate the coordinates of each anchor point and the coordinates of each pin vertex, and save the calculated data in the model to obtain the standard model.

[0042] S202: Acquire a product image of the connector through a visual inspection module.

[0043] After executing step S201, a product image of the connector is captured by a scanning camera in the visual inspection module.

[0044] Among them, the number of scanning cameras in the visual inspection module can be two or more, and the scanning camera can be a 2D scanning camera such as a line laser scanning camera, an area array scanning camera, etc. The placement of each scanning camera is determined by the type of scanning camera. For example, if the scanning camera is a line laser scanning camera, the placement of each scanning camera can be parallel, and the laser scanning line of the camera is perpendicular to the connector surface and parallel to the pin to be inspected; if the scanning camera is an area array scanning camera, each scanning camera can be placed at a certain angle.

[0045] Specifically, if the scanning camera is a line laser scanning camera, the connector is scanned line by line through a preset product scanning path to gradually construct the entire product image of the connector; if the scanning camera is an area array scanning camera, the camera's focal length, aperture, shooting angle and other parameters are adjusted so that the lens field of view covers the entire connector, and then the entire product image of the connector is captured.

[0046] S203: Calculate the first actual coordinates of each anchor point in the product image in the actual coordinate system through an image recognition algorithm.

[0047] Based on the anchor points in the standard model, the anchor points in the product image are found through the feature matching algorithm in image recognition, and then the first actual coordinates of each anchor point in the actual coordinate system are calculated through the edge fitting algorithm or the circle fitting algorithm. Among them, the actual coordinate system is determined according to the positional relationship of the anchor points between the anchor points in the product image. When the connector is not offset, the coordinates of each anchor point in the actual coordinate system are consistent with the coordinates in the standard coordinate system.

[0048] Specifically, firstly, the features of each anchor point are obtained based on the relevant information of each anchor point in the standard model, and then the corresponding area in the product image that matches the features of each anchor point is found through the feature matching algorithm. Then, for the corresponding area in the product image that matches the features of each anchor point in the standard model, that is, the anchor points in each product image, their specific contour morphology is further analyzed.

[0049] If the contour of the anchor point is closer to being composed of straight line segments, its first actual coordinate in the actual coordinate system is calculated using the edge fitting algorithm; if the contour of the anchor point presents a more obvious circular feature, its first actual coordinate in the actual coordinate system is calculated using the circle fitting algorithm.

[0050] S204: Calculate the actual offset angle of the connector placement position according to the first actual coordinates and the first standard coordinates of each anchor point.

[0051] A vector is constructed based on the first actual coordinate and the first standard coordinate of each anchor point, and the actual offset angle of the connector placement position is calculated through the vector. The actual offset angle includes the offset angle of the connector placement position relative to each coordinate axis, such as the offset angle relative to the x-axis, the offset angle relative to the y-axis, and the offset angle relative to the z-axis.

[0052] Specifically, first obtain the first actual coordinates and the first standard coordinates of each anchor point. For each anchor point, construct a coordinate difference vector by subtracting the first standard coordinate from its first actual coordinate. For the offset angle of the x-axis, select one or more vectors from multiple coordinate difference vectors, calculate the unit vector projection of these vectors in the x-axis direction, and then calculate the dot product of the corresponding projection vector and the modulus of the projection vector. Substitute the cosine formula of the vector angle to obtain the cosine value of the vector angle, and then use the inverse cosine function to obtain the offset angle of the x-axis. In the same way, calculate the offset angles of the y-axis and z-axis. Finally, sort out and summarize the offset angles of the x-axis, y-axis, and z-axis to obtain the actual offset angle of the connector placement position.

[0053] In some embodiments, multiple groups of different vector combinations can be selected and substituted into the spatial vector angle formula to calculate multiple different actual offset angles. The actual offset angle of the connector placement position is then determined by calculating the average or weighted average of these offset angles, making the calculated actual offset angle more accurate and reliable.

[0054] S205: Establish a detection coordinate system according to the actual offset angle and the actual coordinate system.

[0055] The actual coordinate system is rotated according to the actual offset angle to establish the detection coordinate system.

[0056] Among them, the detection coordinate system is obtained by rotating the actual coordinate system according to the actual offset angle, the third actual coordinates of each anchor point in the detection coordinate system match the first standard coordinates of the corresponding anchor point in the standard coordinate system, and the third actual coordinates are obtained by coordinate system transformation of the first actual coordinates according to the actual offset angle.

[0057] Specifically, first determine the offset angle of the connector placement relative to the x-axis, the offset angle relative to the y-axis, and the offset angle relative to the z-axis according to the actual offset angle, and then rotate the actual coordinate system around the x-axis, y-axis, and z-axis according to the offset angle to obtain the detection coordinate system. In the process of rotating the coordinate system around the x-axis, y-axis, and z-axis, after each rotation of the coordinate system is completed, the coordinates of each anchor point in the rotated coordinate system must be calculated according to the coordinate calculation formula.

[0058] In some embodiments, in order to make the established detection coordinate system more accurate, the third actual coordinates of each anchor point can be matched with the corresponding first standard coordinates, and the third actual coordinates of each anchor point and the first standard coordinates can be compared to check whether the difference between them is within a pre-set reasonable error range. If it is within the reasonable error range, it means that the established detection coordinate system is accurate and reliable; if it is not within the reasonable error range, the detection coordinate system needs to be further adjusted to make the established detection coordinate system accurate and reliable.

[0059] S206: Determine a scanning path of the visual inspection module according to the first standard coordinates of each anchor point and the second standard coordinates of each pin needle.

[0060] After executing step S201, the first standard coordinates of each anchor point and the second standard coordinates of each pin are obtained to determine the scanning path of the visual inspection module.

[0061] Specifically, when the scanning camera is a line laser scanning camera, an anchor point is first selected as the starting point of the scanning path, and the starting point is used as the starting position. According to the distribution law of the pin needles, a scanning path that can pass through each pin needle in sequence is planned in sequence; When the scanning camera is an area array scanning camera, it is necessary to divide the area composed of multiple anchor points into several sub-areas containing pin pins according to the first standard coordinates of each anchor point and the second standard coordinates of each pin pin, and then determine the scanning path of each sub-area according to certain rules.

[0062] S207 . Scan each pin in the connector in sequence through the visual inspection module according to the scanning path to obtain a scan image of each pin.

[0063] The number of scanned images of each pin needle is determined by the number of scanning cameras in the visual inspection module. Generally, the number of scanned images of each pin needle is two or more.

[0064] Specifically, when the scanning camera is a line laser scanning camera, the scanning camera is first moved to the starting position of the scanning path, and then the scanning camera scans each pin in the connector along the scanning path in turn to obtain multiple scanning image fragments, and then through the image stitching and processing algorithm, the image fragments of the same pin obtained at different scanning positions are integrated to finally obtain a complete scanning image of each pin.

[0065] When the scanning camera is an area array scanning camera, the scanning camera is first moved to the starting point of the scanning path so that its field of view can cover the first sub-area to be scanned, and then the image in each sub-area is captured in sequence by the area array scanning camera to obtain the scanned image of the pin needle in each sub-area.

[0066] S208 : Calculate the second actual coordinates of each pin in the connector based on the detection coordinate system and the scanned images of each pin.

[0067] Specifically, the scanning camera is first accurately calibrated to obtain its internal parameters (such as focal length, principal point coordinates, distortion coefficient, etc.) and external parameters (i.e., the rotation matrix and translation vector of the camera relative to the detection coordinate system). Then, for the scanned image of each pin needle, the pixel coordinates of each pin needle in the scanned image are determined by image recognition and feature extraction algorithms. Next, according to the pinhole camera model, the pixel coordinates of each pin needle are converted into normalized coordinates in the camera coordinate system using the acquired camera internal parameters, and then combined with the camera's external parameters (rotation matrix and translation vector), the normalized coordinates in the camera coordinate system are further converted to the detection coordinate system through coordinate transformation to obtain the coordinate information of each pin needle in the detection coordinate system. Finally, the second actual coordinates of each pin needle are calculated by selecting a suitable fitting algorithm (such as the least squares method, etc.) according to the shape characteristics of the pin needle. Among them, the pinhole camera model is an idealized camera model based on the principle of pinhole imaging.

[0068] S209 , performing abnormality detection on each pin pin in the connector according to the second standard coordinates and the second actual coordinates of each pin pin, and obtaining detection results of each pin pin.

[0069] Among them, the detection results include but are not limited to the pin needle qualification status (qualified or unqualified), the pin needle position, the pin needle height error value, the pin needle position status (normal or abnormal) and the pin needle height status (normal or abnormal), etc., which are not limited here.

[0070] Specifically, the position accuracy of each pin is calculated based on the x-axis coordinate and y-axis coordinate in the second standard coordinate and the second actual coordinate of each pin. The position accuracy is a key indicator to measure the degree of deviation of the actual position of the pin in the plane from the standard position. After the position accuracy of each pin is calculated, the position accuracy of each pin is compared with the preset position accuracy tolerance range. If the position accuracy of the pin is within the tolerance range, it means that the position of the pin is normal; if the position accuracy of the pin is not within the tolerance range, it means that the position of the pin is abnormal.

[0071] In addition to comparing the position of the pin pin, it is also necessary to compare the z-axis coordinate of each pin pin in the second standard coordinate with the z-axis coordinate in the second actual coordinate to obtain the height error value of each pin pin, and then determine whether the height error value of each pin pin is within the preset height error range. If the height error value of the pin pin is within the height error range, it means that the height of the pin pin is normal; if the height error value of the pin pin is not within the height error range, it means that the height of the pin pin is abnormal.

[0072] If the position and height of the pin needle are normal, the pin needle is judged to be qualified; if the position or height of the pin needle is abnormal, the pin needle is judged to be unqualified.

[0073] Finally, the inspection results of each pin are obtained by comprehensively considering the qualified status, position degree, height error value, position status and height status of each pin.

[0074] In the embodiment of the present application, the offset angle of the connector placement position is calculated by comparing the actual coordinates of each anchor point with the standard coordinates, and then a detection coordinate system is established according to the offset angle. The actual coordinates of the pin needle are calculated by the detection coordinate system, so the actual coordinates of the pin needle can be accurately calculated, and the pin needle is detected by the calculated actual coordinates, thereby avoiding the situation where the actual coordinate calculation of the pin needle is prone to misjudgment due to deviation in the placement position, and effectively improving the detection accuracy of the pin needle.

[0075] Combine the following Figure 3 To further illustrate the method of the embodiment of the present application.

[0076] See also Figure 3 , is another flow chart of the pin needle visual detection method in an embodiment of the present application.

[0077] S301. Determine whether there is a standard model of the connector in the model library.

[0078] According to the first identification information of the connector, the model library is searched to see whether there is a second identification information of a standard model that matches it. If so, step S302 is executed to obtain a standard model that matches the connector in the model library; if not, step S303 is executed to generate a 3D model of the connector based on the standard drawing of the connector. The first identification information of the connector includes but is not limited to the product model, product version number, etc., and the second identification information of the standard model includes but is not limited to the first identification information of the connector, the industry standard specification number, etc., which are not limited here.

[0079] S302: Obtain a standard model corresponding to the connector in the model library.

[0080] The standard model matching the connector found in step S301 is obtained.

[0081] S303: Based on the standard drawing of the connector, a 3D model of the connector is generated by a model generation module.

[0082] Based on the standard drawings of the connector uploaded by the operator, a 3D model of the connector is generated through the model generation module.

[0083] Specifically, the uploaded standard drawings are first parsed through the model generation module to extract key design information such as component shape, size and position relationship. Then, the 3D model of each component in the connector is constructed through the model generation module, and then the model generation module performs component assembly operations based on the relative position relationship of each component shown in the standard drawings to construct a complete 3D model of the connector.

[0084] S304, obtaining the position information and height information of each pin in the 3D model through a point cloud registration algorithm.

[0085] Specifically, the 3D model is first processed into a point cloud to obtain the original point cloud. Each point in the point cloud carries its coordinate information in three-dimensional space. Then, the point cloud containing each pin is extracted from the original point cloud as the point cloud to be registered, and then the point cloud matching algorithm is used to match the point cloud to be registered with the reference point cloud of the preset pin. After the point cloud registration is completed, the specific position information of each pin on the horizontal plane (i.e., the x and y axis directions) can be obtained by analyzing the coordinate distribution of the registered point cloud in three-dimensional space. As for the height information of the pin, the lowest z-axis coordinate of the pin bottom point cloud after registration and the highest z-axis coordinate corresponding to the pin top point cloud are obtained, and the difference between the two coordinates is calculated to obtain the height information of each pin.

[0086] S305: Generate an initial model based on the position information and height information of each pin.

[0087] Among them, the initial model only contains the structure and information related to the pin.

[0088] Specifically, according to the height information of each pin needle, based on the determined plane position, the construction is carried out along the z-axis direction to shape the shape of each pin needle in the vertical direction, and obtain the three-dimensional model of each pin needle. Then, one of the pin needles is selected as the reference pin needle, and the x-axis and y-axis coordinates of the pin needle in the three-dimensional space are set to 0, making it the origin of the plane coordinates, and constructing the plane reference benchmark of the entire model space. For other pin needles, their positional relationship with the reference pin needle is analyzed one by one, and the offset of other pin needles on the plane relative to the reference pin needle is obtained, thereby obtaining the x-axis and y-axis coordinates of other pin needles in the three-dimensional space, and obtaining the specific position distribution of each pin needle on the plane. Finally, the three-dimensional models of each pin needle are arranged and integrated according to the specific position distribution of each pin needle on the plane, and an initial model containing only the pin needle is obtained.

[0089] S306: Obtain the standard coordinate system and anchor points marked by the operator in the initial model to generate a standard model.

[0090] The standard coordinate system and anchor points marked by the operator in the initial model are obtained, a standard model is generated, and the standard model is stored in the model library.

[0091] Specifically, based on the standard coordinate system and the position information of the anchor points marked by the operator in the initial model, the first standard coordinates of each anchor point in the standard coordinate system are calculated. Then, the x-axis and y-axis coordinates of each pin pin in the standard coordinate system are calculated based on the first standard coordinates of each anchor point and the relative position relationship between each pin pin and the anchor point, and then the z-axis coordinates of each pin pin in the standard coordinate system are obtained based on the height information of each pin pin. Next, the second standard coordinates of each pin pin in the standard coordinate system are obtained by combining the x-axis, y-axis and z-axis coordinates of each pin pin. Finally, the calculated first standard coordinates of each anchor point and the second standard coordinates of each pin pin are saved in the marked initial model to obtain a standard model that can be used to detect each pin pin in the connector.

[0092] S307 . Obtain a product image of the connector through a visual inspection module.

[0093] S308. Calculate the first actual coordinates of each anchor point in the product image in the actual coordinate system through an image recognition algorithm.

[0094] Steps S307, S308 and Figure 2In the illustrated embodiment, steps S202 and S203 are similar, and the descriptions of steps S202 and S203 may be referred to, and will not be repeated here.

[0095] S309: Determine whether the anchoring figure formed by each anchoring point is a two-dimensional figure.

[0096] If the anchor figure formed by each anchor point is a two-dimensional figure, execute step S310 to determine the offset angle between the first actual coordinates and the first standard coordinates of each anchor point; if the anchor figure formed by each anchor point is not a two-dimensional figure, execute step S312 to obtain the standard center point coordinates and actual center point coordinates of the anchor figure in the standard coordinate system.

[0097] Specifically, the first standard coordinates of each anchor point are obtained to determine whether the z-axis coordinates of all anchor points are the same. If they are the same, it is determined that the anchor figure formed by each anchor point is a two-dimensional figure; if they are not the same, it is determined that the anchor figure formed by each anchor point is not a two-dimensional figure.

[0098] S310: Determine an offset angle between a first actual coordinate and a first standard coordinate of each anchor point.

[0099] Specifically, construct the vector from the first standard coordinate of each anchor point to the origin of the coordinates to obtain the standard vector of each anchor point, then construct the vector from the first actual coordinate of each anchor point to the origin of the coordinates to obtain the actual vector of each anchor point. For each anchor point, project its standard vector and actual vector to two-dimensional planes (i.e., xy plane, yz plane, and xz plane) in sequence by projection method to obtain the first standard vector and the first actual vector after projecting to the xy plane, the second standard vector and the second actual vector after projecting to the xy plane, and the third standard vector and the third actual vector after projecting to the xy plane. Then, the vector dot product formula and the modulus length formula are used to calculate the cosine values ​​of the vector angles between the first standard vector and the first actual vector, the second standard vector and the second actual vector, and the third standard vector and the third actual vector, respectively, and then the inverse cosine function is used to obtain the vector angles between the first standard vector and the first actual vector, the second standard vector and the second actual vector, and the third standard vector and the third actual vector.

[0100] Finally, the vector angles between the first standard vector and the first actual vector, the second standard vector and the second actual vector, and the third standard vector and the third actual vector of each anchor point are summarized and sorted to obtain the offset angles of each anchor point relative to the x-axis, y-axis, and z-axis.

[0101] S311. Calculate the average value of all offset angles to obtain the actual offset angle.

[0102] The average value of all offset angles of each anchor point relative to the x-axis, the average value of all offset angles relative to the z-axis, and the average value of all offset angles relative to the y-axis are calculated respectively to obtain the offset angle of the connector position relative to the x-axis, the offset angle relative to the y-axis, and the offset angle relative to the z-axis.

[0103] S312, obtaining the standard center point coordinates and the actual center point coordinates of the anchoring figure in the standard coordinate system.

[0104] Specifically, the first standard coordinates and the first actual coordinates of each anchor point are obtained, and the coordinate means in the x-axis, y-axis and z-axis directions are calculated according to the first standard coordinates of each anchor point, and then the calculated coordinate means in the x-axis, y-axis and z-axis directions are combined to obtain the standard center point coordinates of the anchor figure in the standard coordinate system. Similarly, according to the first actual coordinates of each anchor point, the coordinate means in the x-axis, y-axis and z-axis directions are calculated, and then the calculated coordinate means in the x-axis, y-axis and z-axis directions are combined to obtain the actual center point coordinates of the anchor figure in the actual coordinate system. Among them, the coordinate mean is obtained by accumulating the corresponding coordinate values ​​and dividing them by the number of anchor points.

[0105] S313. Construct an offset vector from the standard center point coordinates to the actual center point coordinates.

[0106] According to the standard center point coordinates and the actual center point coordinates, the difference of the actual center point coordinates relative to the standard center point coordinates in the x-axis, y-axis and z-axis directions is calculated, and according to the calculated coordinate difference, an offset vector pointing from the standard center point coordinates to the actual center point coordinates is constructed.

[0107] S314. Calculate the actual offset angle according to the offset vector.

[0108] According to the offset vector, the offset vector is projected onto a two-dimensional plane by a projection method to calculate the offset angle of the offset vector relative to the x-axis, y-axis and z-axis, or the offset angle of the offset vector relative to the x-axis, y-axis and z-axis is calculated by using the direction cosine method to obtain the offset angle of the connector position relative to the x-axis, the offset angle relative to the y-axis and the offset angle relative to the z-axis.

[0109] S315: Establish a detection coordinate system according to the actual offset angle and the actual coordinate system.

[0110] Step S315 and Figure 2 Step S205 in the illustrated embodiment is similar, and reference may be made to the description in step S205 , which will not be repeated here.

[0111] S316: According to the second standard coordinates of each pin in the connector, a pin detection sequence that can minimize the detection time is calculated by a path planning algorithm.

[0112] Specifically, the second standard coordinate information of each pin in the connector is obtained, and these coordinate data are constructed into a format that can be processed by the path planning algorithm, such as presenting them in the form of a set of coordinate points. Then, a suitable path planning algorithm is selected, such as a greedy algorithm, a dynamic programming algorithm, etc., and the coordinate points of the pins are used as nodes, and the distance between nodes and other related metrics are used as edge weights. Next, the shortest detection time is taken as the optimization goal, and the selected algorithm is used to start the calculation. When the algorithm converges or reaches the preset stop condition, the pin detection sequence that can minimize the detection time can be obtained.

[0113] S317, obtaining the second standard coordinates of the first pin detected in the pin detection sequence as the initial detection coordinates.

[0114] According to the pin needle detection sequence calculated in step S316, the first pin needle to be detected in the detection sequence is determined, and the second standard coordinates of the pin needle are obtained as the initial detection coordinates.

[0115] S318. Calculate the distance between each anchor point and the first pin detected in the pin detection sequence according to the first standard coordinates and the initial detection coordinates of each anchor point.

[0116] The first standard coordinates and initial detection coordinates of each anchor point are obtained, and the distance between each anchor point and the first pin detected in the pin detection sequence is calculated by a three-dimensional space distance calculation formula.

[0117] S319, selecting the first standard coordinates of the anchor point with the shortest distance to the first pin detected in the pin detection sequence as the reference coordinates.

[0118] According to the distances between each anchor point and the first pin detected in the pin detection sequence calculated in step S318, these distance values ​​are compared and analyzed to find the distance with the smallest value, and the first standard coordinates of the anchor point corresponding to the minimum distance are obtained as reference coordinates. The reference coordinates are used to accurately locate the actual position of each pin.

[0119] S320, determining a scanning path of the visual inspection module according to the initial coordinates, reference coordinates, pin detection order, and first standard coordinates of each pin of the visual inspection module in the standard coordinate system.

[0120] Specifically, the initial coordinates of the visual inspection module in the standard coordinate system are obtained, and the initial coordinates are used as the starting point of the scanning path. According to the reference coordinates, combined with the motion characteristics of the visual inspection module (such as moving speed, steering flexibility, etc.), the first segment of the path from the initial coordinates to the reference coordinates is planned. Then, according to the pin needle detection order, the first standard coordinates of each pin needle are used as nodes on the path in turn. According to the coordinate difference between adjacent pin needles and the motion characteristics of the visual inspection module, sub-paths from one pin needle position to the next pin needle position are gradually planned, and these sub-paths are connected in turn to form an intermediate segment path moving between each pin needle. Finally, the path from the initial coordinates to the reference coordinates and the intermediate segment path moving between the pin needles are combined to determine the scanning path of the visual inspection module in the entire detection process.

[0121] S321. Scan each pin in the connector in sequence through a visual inspection module according to the scanning path to obtain a scan image of each pin.

[0122] S322: Calculate the second actual coordinates of each pin in the connector based on the detection coordinate system and the scanned images of each pin.

[0123] S323, performing abnormality detection on each pin pin in the connector according to the second standard coordinates and the second actual coordinates of each pin pin, and obtaining detection results of each pin pin.

[0124] Steps S321-S323 and Figure 2 Steps S207 to S209 in the illustrated embodiment are similar, and the descriptions of steps S207 to S209 may be referred to, and will not be repeated here.

[0125] S324, determine whether all pins in the connector are unqualified.

[0126] According to the detection results of each pin needle, the qualified status of each pin needle is obtained to determine whether the qualified status of each pin needle is unqualified. If so, execute step S325 to determine the reason for the unqualifiedness of each pin needle; if not, execute step S335 to continue to detect the pin needle of the next connector.

[0127] S325, determine the reasons for the failure of each pin.

[0128] According to the detection results of each pin needle, check the position and height of each pin needle. If the position of the pin needle is abnormal, it means that the reason for the failure of the pin needle is abnormal position; if the height of the pin needle is abnormal, it means that the reason for the failure of the pin needle is abnormal height.

[0129] S326, determining whether the reasons for the failure of each pin in the connector are the same.

[0130] According to the reasons for the failure of each pin, determine whether the reasons for the failure of each pin are the same. If so, execute step S327 to obtain the operating parameters of the production equipment corresponding to the connector; if not, execute step S335 to continue to detect the pin of the next connector.

[0131] S327. Obtain operating parameters of the production equipment corresponding to the connector.

[0132] The production equipment for producing the connector is determined by the connector model, and then the operating parameters of the production equipment are obtained through the control system of the production equipment. The operating parameters include part positioning accuracy, assembly force control accuracy, assembly speed, etc.

[0133] S328. Determine whether the operating parameters are beyond the normal range.

[0134] The acquired operating parameters of the production equipment are compared with the parameter range of the operating parameters under the preset normal operating conditions to determine whether any operating parameter of the production equipment exceeds the corresponding parameter range. If so, execute step S329 to send an instruction to stop the equipment operation to the production equipment through the control module; if not, execute step S330 to send a first prompt message to the operator.

[0135] S329. Send a command to stop the operation of the equipment to the production equipment through the control module.

[0136] When it is determined that the operating parameters of the production equipment are beyond the normal range, an instruction to stop the operation of the equipment is sent to the production equipment through the control module to stop the operation of the production equipment.

[0137] S330: Send a first prompt message to the operator.

[0138] When it is determined that the operating parameters of the production equipment do not exceed the normal range, a first prompt message is sent to the operator, wherein the first prompt message is used to prompt the operator that the standard model of the connector may be inaccurate.

[0139] Specifically, the first prompt information is pushed to the operator through various channels, such as by using the display screen of the equipment operation panel to pop up a striking pop-up window in a prominent position, and the pop-up window presents the first prompt information in detail in clear and concise text. If the enterprise has a mobile application for production management, the first prompt information can also be pushed to the operator's mobile device such as a mobile phone or tablet computer, so that the operator can receive and view the prompt information while on the move or at a remote location.

[0140] S331. Determine whether the actual offset angle exceeds a preset offset angle threshold.

[0141] Based on the actual offset angle, determine whether the actual offset angle exceeds the preset offset angle threshold. If it exceeds, execute step S332 to obtain a reference image of the preset reference point next to the connector through the visual inspection module; if it does not exceed, execute step S335 to continue inspecting the pin of the next connector.

[0142] Specifically, the offset angle of the connector placement position relative to the x-axis, the offset angle relative to the y-axis, and the offset angle relative to the z-axis in the actual offset angle are obtained, and the offset angles of the x-axis, y-axis, and z-axis are compared with the preset offset angle thresholds of the x-axis, y-axis, and z-axis, respectively. If the offset angles of the x-axis, y-axis, and z-axis do not exceed the threshold, it is determined that the actual offset angle does not exceed the preset offset angle threshold; if the offset angle of the x-axis exceeds the threshold, or the offset angle of the y-axis exceeds the threshold, or the offset angle of the z-axis exceeds the threshold, it is determined that the actual offset angle exceeds the preset offset angle threshold.

[0143] S332. Acquire a reference image of a preset reference point next to the connector through a visual inspection module.

[0144] Based on the preset scanning path, a reference image of a preset reference point next to the connector is acquired by the visual inspection module.

[0145] S333: Determine whether the reference image matches a pre-stored standard reference image of the reference point.

[0146] Determine whether the reference image matches the standard reference image of the pre-stored reference point. If so, execute step S335 to continue detecting the pin of the next connector; if not, execute step S334 to send a second prompt message to the operator.

[0147] Specifically, the reference image and the standard reference image are first preprocessed, including size normalization and denoising.

[0148] Then, extract the key features respectively. In the shape feature extraction, use the contour extraction algorithm to calculate the perimeter, area, circularity, rectangularity and other parameters of the reference points in the reference image and the standard reference image respectively, and compare these shape feature parameters of the reference points in the two images; for the feature point features, use the scale-invariant feature transform (SIFT) algorithm or the speeded up robust feature (SURF) algorithm to extract the feature points and corresponding feature descriptors of the reference points in the reference image and the standard reference image respectively, and then judge the matching degree of the reference image and the standard reference image by matching the number of feature points, the distance between feature points and other indicators. Among them, the feature descriptor is a quantitative representation of the local features in the image (or other data types), which is essentially a vector.

[0149] Finally, a reasonable matching threshold is set according to the extracted features, such as the sum of the absolute values ​​of the difference in shape feature parameters of the reference points in the reference image and the standard reference image is less than a certain value, the number of feature point matches accounts for a certain percentage, and the average distance error of feature point matches is within a reasonable range. If the comparison results of all features meet the corresponding matching threshold requirements, the reference image is determined to match the standard reference image, otherwise, the reference image is determined to not match the standard reference image.

[0150] S334: Send a second prompt message to the operator.

[0151] When the reference image does not match the standard reference image of the pre-stored reference point, a second prompt message is sent to the operator, wherein the second prompt message is used to prompt the operator that the fixed position of the visual detection module is offset.

[0152] S335, detect the pin of the next connector.

[0153] Determine whether the connector to be detected is of the same type as the connector currently being detected. If so, execute step S307 ; if not, execute step S301 .

[0154] In the embodiment of the present application, the standard model of the connector is obtained through the model library, which improves the efficiency of obtaining the standard model; according to the anchoring figure composed of the anchor points, different calculation methods are executed to calculate the offset angle of the connector placement position, which improves the accuracy of calculating the offset angle; the detection coordinate system is established through the offset angle to calculate the actual coordinates of each pin needle, which improves the accuracy of calculating the actual coordinates of the pin needle; before planning the scanning path, the pin needle detection sequence that can shorten the detection time is calculated, the speed of scanning the pin needle is improved, thereby improving the detection efficiency of the pin needle, and the anchor point closest to the first pin needle detected in the pin needle detection sequence is used as the reference coordinate for calculating the actual coordinates of the pin needle, so that the actual position of each pin needle is located more accurately and efficiently, and the accuracy of collecting pin needle images is improved, thereby improving the accuracy of calculating the actual coordinates of the pin needle; when the operating parameters of the production equipment are detected to be abnormal, the operation of the production equipment is stopped quickly, avoiding the continued production of unqualified connectors, reducing unnecessary increase in production costs and ineffective consumption of production resources; when it is detected that the fixed position of the camera in the visual inspection module is offset, the operator is immediately notified to adjust to avoid the camera offset affecting the detection accuracy of the pin needle. The overall detection accuracy and efficiency of the pin needles are improved, and the problem of reduced pin needle detection accuracy due to deviation in connector placement or camera offset is avoided.

[0155] The above describes the pin needle visual detection method in the embodiment of the present application. The following describes the detection server in the embodiment of the present application in detail in combination with the above-mentioned pin needle visual detection method.

[0156] See also Figure 4 , is a schematic diagram of an exemplary hardware structure of the detection server in an embodiment of the present application.

[0157] In some embodiments, the detection server 400 is a computer device, which can be a terminal device. The computer device includes a processor 401, a memory 402, a communication module 403, an input device 404 and an output device 405 connected by a system bus. Among them, the processor 401 of the computer device is used to provide computing and control capabilities. The memory 402 of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database is used to store data. The communication module 403 of the computer device is used to send prompt information to the operator and send control instructions to the control system of the production equipment. The input device 404 of the computer device is used to obtain the standard drawings uploaded by the operator and the anchor points and standard coordinate systems marked on the initial model. The output device 405 of the computer device is used to display various information, including standard models, test results, etc. When the computer program is executed by the processor 401, the pin needle visual detection method in the embodiment of the present application is implemented.

[0158] Those skilled in the art will understand that Figure 4 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0159] In some embodiments of the present application, a computer-readable storage medium is provided, including instructions, which, when executed on the detection server 400, can enable the detection server 400 to execute the pin visual detection method in the embodiments of the present application.

[0160] In some embodiments of the present application, a computer program product is also provided. When the computer program product is run on the detection server 400, the detection server 400 executes the pin needle visual detection method in the embodiments of the present application.

[0161] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

[0162] As used in the above embodiments, the term "when..." may be interpreted to mean "if..." or "after..." or "in response to determining..." or "in response to detecting...", depending on the context. Similarly, the phrases "upon determining..." or "if (the stated condition or event) is detected" may be interpreted to mean "if determining..." or "in response to determining..." or "upon detecting (the stated condition or event)" or "in response to detecting (the stated condition or event)", depending on the context.

[0163] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk), etc.

[0164] Those skilled in the art can understand that to implement all or part of the processes in the above-mentioned embodiments, the processes can be completed by computer programs to instruct related hardware, and the programs can be stored in computer-readable storage media. When the programs are executed, they can include the processes of the above-mentioned method embodiments. The aforementioned storage media include: ROM or random access memory RAM, magnetic disk or optical disk and other media that can store program codes.

Claims

1. A pin visual detection method, characterized in that: include: Acquire a standard model of the connector; the standard model includes a standard coordinate system, multiple anchor points, a first standard coordinate of each anchor point in the standard coordinate system, and a second standard coordinate of each pin in the connector in the standard coordinate system; the standard coordinate system is determined based on the anchor point position relationship between each anchor point in the standard model; the anchor point position relationship is fixed; the x-axis and y-axis of the second standard coordinate represent the position of the pin, and the z-axis represents the height of the pin; Acquire a product image of the connector through a visual inspection module; Calculating first actual coordinates of each anchor point in the product image in an actual coordinate system by an image recognition algorithm; the actual coordinate system is determined according to the positional relationship of the anchor points between the anchor points in the product image; Calculating an actual offset angle of the connector placement position according to the first actual coordinates and the first standard coordinates of each anchor point; Establishing a detection coordinate system according to the actual offset angle and the actual coordinate system; The detection coordinate system is obtained by rotating the actual coordinate system according to the actual offset angle; the third actual coordinates of each anchor point in the detection coordinate system match the first standard coordinates of the corresponding anchor point in the standard coordinate system; the third actual coordinates are obtained by performing coordinate system conversion on the first actual coordinates according to the actual offset angle; Determining a scanning path of the visual inspection module according to the first standard coordinates of each anchor point and the second standard coordinates of each pin needle; According to the scanning path, the visual inspection module sequentially scans each pin in the connector to obtain a scanned image of each pin; Calculating the second actual coordinates of each pin in the connector based on the detection coordinate system and the scanned images of each pin; According to the second standard coordinates and the second actual coordinates of each pin pin in the connector, each pin pin is subjected to abnormality detection to obtain a detection result of each pin pin; the abnormality detection includes position detection and height detection.

2. The method according to claim 1, characterized in that Determining the scanning path of the visual inspection module according to the first standard coordinates of each anchor point and the second standard coordinates of each pin needle specifically includes: According to the second standard coordinates of each pin in the connector, a pin detection sequence that can minimize the detection time is calculated by a path planning algorithm; Acquire the second standard coordinates of the first pin detected in the pin detection sequence as the initial detection coordinates; Calculate the distance between each anchor point and the first pin detected in the pin detection sequence according to the first standard coordinates and the initial detection coordinates of each anchor point; The first standard coordinates of the anchor point with the shortest distance from the first pin detected in the pin detection sequence are selected as reference coordinates; the reference coordinates are used to accurately locate the actual position of each pin; The scanning path of the visual inspection module is determined according to the initial coordinates of the visual inspection module in the standard coordinate system, the reference coordinates, the pin needle detection sequence, and the first standard coordinates of each pin needle.

3. The method according to claim 1, characterized in that The calculating the actual offset angle of the connector placement position according to the first actual coordinates and the first standard coordinates of each anchor point specifically includes: Based on the first standard coordinates of each anchor point in the standard model, determining whether the anchor figure formed by each anchor point is a two-dimensional figure; If the anchor figure is a two-dimensional figure, determining an offset angle between the first actual coordinates and the first standard coordinates of each anchor point based on the first actual coordinates and the first standard coordinates of each anchor point; Calculate the average value of all the offset angles to obtain the actual offset angle; If the anchor figure is not a two-dimensional figure, obtaining the standard center point coordinates and the actual center point coordinates of the anchor figure in the standard coordinate system; Constructing an offset vector from the standard center point coordinates to the actual center point coordinates; The actual offset angle is calculated based on the offset vector.

4. The method according to claim 1, characterized in that The standard model of obtaining the connector specifically includes: Determine whether a standard model of the connector exists in the model library; If so, obtaining the standard model corresponding to the connector in the model library; If it does not exist, the standard model is established based on the standard drawing of the connector.

5. The method according to claim 4, characterized in that If the standard model does not exist, the standard model is established based on the standard drawing of the connector, specifically including: Based on the standard drawing of the connector, a 3D model of the connector is generated by a model generation module; Acquire the position information and height information of each pin in the 3D model through a point cloud registration algorithm; Generate an initial model based on the position information and the height information of each pin needle; The standard coordinate system and the anchor points marked by the operator in the initial model are obtained to generate the standard model.

6. The method according to claim 1, characterized in that After the step of performing abnormality detection on each pin pin in the connector according to the second standard coordinates and the second actual coordinates of each pin pin to obtain the detection result of each pin pin, the method further includes: When the detection results of all the pins of the connector are unqualified, determining the reasons for the unqualifiedness of each pin; When the reasons for the failure of all pins are the same, obtaining the operating parameters of the production equipment corresponding to the connector; determining whether the operating parameter is outside a normal range; If it exceeds the normal range, a command to stop the operation of the equipment is sent to the production equipment through the control module; If it does not exceed the normal range, a first prompt message is sent to the operator; the first prompt message is used to prompt the operator that the standard model of the connector may be inaccurate.

7. The method according to claim 1, characterized in that After the step of performing abnormality detection on each pin pin in the connector according to the second standard coordinates and the second actual coordinates of each pin pin to obtain the detection result of each pin pin, the method further includes: Determining whether the actual offset angle exceeds a preset offset angle threshold; If so, based on the preset scanning path, obtaining a reference image of a preset reference point next to the connector through the visual inspection module; determining whether the reference image matches a pre-stored standard reference image of the reference point; If there is no match, a second prompt message is sent to the operator; the second prompt message is used to prompt the operator that the fixed position of the visual inspection module is offset.

8. A detection server, characterized in that: include: one or more processors and memory; The memory is coupled to the one or more processors, and the memory is used to store computer program codes, where the computer program codes include computer instructions. The one or more processors call the computer instructions to enable the detection server to execute the method according to any one of claims 1 to 7.

9. A computer-readable storage medium storing computer instructions, characterized in that: When the computer instructions are executed on the detection server, the detection server is caused to execute the method according to any one of claims 1 to 7.

10. A computer program product, characterized in that When the computer program product is run on a detection server, the detection server is caused to execute the method according to any one of claims 1 to 7.

Images