Modular pin vision inspection method, inspection server, medium and product

Abstract

A modular pin needle visual detection method, a detection server, a medium and a product relate to the field of machine vision detection. The method comprises: acquiring a standard model of a connector; acquiring a product image through a visual detection module; calculating a first actual coordinate of an anchor point in the product image through an image recognition algorithm; calculating an actual offset angle of the placement position of the connector according to the first actual coordinate and the first standard coordinate of the anchor point; establishing a detection coordinate system according to the actual offset angle and the actual coordinate system; determining a scanning path of the visual detection module; acquiring scanning images of each pin needle through the visual detection module according to the scanning path; calculating second actual coordinates of each pin needle according to the detection coordinate system and the scanning images; and performing abnormal detection on each pin needle to obtain a detection result. The technical scheme of the present application effectively improves the detection accuracy of the pin needle and avoids the problem of reduced detection accuracy of the pin needle caused by the deviation of the placement position of the connector.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of machine vision detection, and particularly relates to a modular pin visual detection method, a detection server, a medium and a product. BACKGROUND

[0002] In the field of modern electronic manufacturing, pin is a key component for connecting various electronic components, and the stability of its quality and performance is crucial. With the development of electronic products towards miniaturization, high integration and high performance, the size of pin is continuously reduced, and the precision requirement is increasingly improved.

[0003] Currently, the detection of pin of a connector is usually performed by one or more 2D scanning cameras above the pin to collect image information of the pin, and then computer vision algorithms are used to process and analyze the image information to obtain position information and height information of each pin, and then it is judged whether the position and height of the pin are within the error range, and then whether the pin is qualified is detected.

[0004] However, since the shooting angle and the field of view of the camera are preset based on the standard placement position of the connector, and the imaging principle of the camera is based on fixed optical geometry, when the placement position of the connector deviates, the actual position of the pin in the camera coordinate system will not match the preset standard position, resulting in changes in the projection position and posture on the image plane. This change will cause deviation in the extraction and analysis of pin feature points in the image based on the preset algorithm, and thus the pin position information and height information calculated from the image will be deviated, which will easily cause misjudgment when detecting the pin, and reduce the detection accuracy of the pin. SUMMARY

[0005] The present application provides a modular pin visual detection method, a detection server, a medium and a product, which calculates the offset angle of the actual placement position of the connector, establishes a new detection coordinate system according to the offset angle, calculates the actual coordinates of each pin in the connector according to the new detection coordinate system, and finally detects the pin through the calculated actual coordinates, thereby improving the accuracy of the calculation of the actual coordinates of the pin and the detection accuracy of the pin.

[0006] In a first aspect, the present application provides a pin visual detection method, comprising: obtaining a standard model of a connector, the standard model comprising a standard coordinate system, a plurality of anchor points, first standard coordinates of each anchor point in the standard coordinate system, and second standard coordinates of each pin in the connector in the standard coordinate system, the standard coordinate system being determined based on an anchor point position relationship between each anchor point in the standard model, the anchor point position relationship being fixed, the x-axis and y-axis of the second standard coordinates representing the position of the pin, and the z-axis representing the height of the pin; obtaining a product image of the connector through a visual detection module; calculating first actual coordinates of each anchor point in an actual coordinate system in the product image through an image recognition algorithm, the actual coordinate system being determined according to the anchor point position relationship between each anchor point in the product image; calculating an actual offset angle of the placement position of the connector according to the first actual coordinates and the first standard coordinates of each anchor point; establishing a detection coordinate system according to the actual offset angle and the actual coordinate system, the detection coordinate system being obtained by rotating the actual coordinate system according to the actual offset angle, the third actual coordinates of each anchor point in the detection coordinate system matching the first standard coordinates of the corresponding anchor point in the standard coordinate system, the third actual coordinates being obtained by coordinate system conversion of the first actual coordinates according to the actual offset angle; determining a scanning path of the visual detection module according to the first standard coordinates of each anchor point and the second standard coordinates of each pin in the standard model; sequentially scanning each pin in the connector through the visual detection module according to the scanning path to obtain a scanning image of each pin; calculating second actual coordinates of each pin in the connector based on the detection coordinate system and the scanning image of each pin; and performing abnormal detection on each pin according to the second standard coordinates and the second actual coordinates of each pin in the connector to obtain a detection result of each pin, the abnormal detection including position detection and height detection.

[0007] According to the above technical solution, the offset angle of the placement position of the connector is calculated by establishing a vector based on the coordinates of each anchor point in the standard coordinate system and the coordinates in the actual coordinate system, and then the detection coordinate system is obtained by rotating the actual coordinate system according to the offset angle, and further the actual coordinates of each pin in the detection coordinate system are calculated based on the detection coordinate system and the scanning image of each pin, so that the calculated actual coordinates can more accurately reflect the actual position and height of the pin. The pin is detected by the actual coordinates, and an accurate detection result can be obtained, which can avoid the reduction of the accuracy of the detection result caused by the deviation of the placement position of the connector, and effectively improves the detection precision of the pin.

[0008] In some embodiments of the first aspect, in some embodiments, the scanning path of the visual detection module is determined according to the first standard coordinates of the anchor points and the second standard coordinates of the pin needles, specifically comprising: according to the second standard coordinates of the pin needles in the connector, a pin needle detection sequence that can make the detection time shortest is calculated by a path planning algorithm; the second standard coordinates of the pin needle detected first in the pin needle detection sequence are obtained as initial detection coordinates; according to the first standard coordinates of the anchor points and the initial detection coordinates, the distances between the anchor points and the pin needle detected first in the pin needle detection sequence are calculated; the first standard coordinates of the anchor point with the shortest distance to the pin needle detected first in the pin needle detection sequence are selected as reference coordinates, which are used to accurately locate the actual positions of the pin needles; and according to the initial coordinates of the visual detection module in the standard coordinate system, the reference coordinates, the pin needle detection sequence and the first standard coordinates of the pin needles, the scanning path of the visual detection module is determined.

[0009] By using the above technical solution, the pin needle detection sequence that can make the detection time shortest is calculated by a path planning algorithm, and the pin needles are detected according to the detection sequence, which reduces the total time for detecting the pin needles and improves the detection efficiency of the pin needles. At the same time, by selecting the first standard coordinates of the anchor point with the shortest distance to the pin needle detected first as reference coordinates, the actual positions of the pin needles are more accurately and efficiently located, the accuracy of collecting pin needle images is improved, and the accuracy of calculating the actual coordinates of the pin needles is improved. Since the scanning path is planned according to the above detection sequence and reference coordinates, the pin needles are collected according to the scanning path, which not only accurately collects the images of the pin needles, but also reduces the time for collecting the images of the pin needles, improves the efficiency of image collection, and further improves the detection efficiency of the pin needles.

[0010] In some embodiments of the first aspect, in some embodiments, the actual offset angle of the connector placement position is calculated according to the first actual coordinates and the first standard coordinates of each anchor point, specifically comprising: determining whether an anchor pattern formed by each anchor point is a two-dimensional pattern based on the first standard coordinates of each anchor point in the standard model; if the anchor pattern is a two-dimensional pattern, determining the offset angle between the first actual coordinates and the first standard coordinates of each anchor point based on the first actual coordinates and the first standard coordinates of each anchor point; calculating the average of all the offset angles to obtain the actual offset angle; if the anchor pattern is not a two-dimensional pattern, obtaining standard center point coordinates and actual center point coordinates of the anchor pattern in the standard coordinate system; constructing an offset vector from the standard center point coordinates to the actual center point coordinates; and calculating the actual offset angle according to the offset vector.

[0011] According to the above technical solution, different calculation methods are used to calculate the actual offset angle for different anchor patterns. When the anchor pattern is a two-dimensional pattern, the offset angle of each anchor point is calculated as an individual, and the average of all the offset angles is calculated to obtain the actual offset angle of the connector, which can accurately capture the subtle position changes of each anchor point caused by various factors, and weaken the influence of accidental errors by calculating the average of all the offset angles, so that the final offset angle accurately reflects the real placement state of the connector in the two-dimensional plane, providing a reliable basis for subsequent pin detection and improving the detection accuracy. When the anchor pattern is not a two-dimensional pattern, the actual offset angle of the connector is calculated as a whole, which takes into account the complex three-dimensional structure of the non-two-dimensional pattern and the more stable structure compared to the two-dimensional pattern. The overall calculation method can better reflect the real displacement of the connector in space, avoiding information fragmentation and error accumulation problems caused by analyzing each anchor point in isolation in a complex three-dimensional structure, effectively improving the accuracy of calculating the offset angle of the connector placement position, and further improving the detection accuracy of the pin.

[0012] In some embodiments of the first aspect, in some embodiments, the standard model of the connector is obtained, specifically comprising: determining whether the standard model of the connector exists in the model library; if it exists, obtaining the standard model corresponding to the connector in the model library; if it does not exist, establishing the standard model based on the standard drawing of the connector.

[0013] According to the above technical solution, the standard model of the connector is directly obtained from the model library, avoiding repeated establishment of the same model multiple times, reducing the time for establishing the model before each detection, and improving the efficiency of pin detection.

[0014] In some embodiments of the first aspect, when the standard model does not exist, the standard model is established based on the standard drawing of the connector, specifically comprising: generating a 3D model of the connector based on the standard drawing of the connector by a model generation module; obtaining position information and height information of each pin needle in the 3D model by a point cloud registration algorithm; generating an initial model based on the position information and the height information of each pin needle; obtaining the standard coordinate system and the anchor point labeled in the initial model by an operator to generate the standard model.

[0015] With the above technical solution, when the standard model corresponding to the connector does not exist in the model library, a 3D model of the connector is directly generated based on the standard drawing of the connector, and the position information and the height information of each pin needle in the 3D model are automatically obtained by a point cloud registration algorithm, and then a model is automatically established according to the position information and the height information of each pin needle, thereby saving the time for manually establishing a model and improving the efficiency of establishing a standard model.

[0016] In some embodiments of the first aspect, after the step of detecting the abnormality of each pin needle according to the second standard coordinate and the second actual coordinate of each pin needle in the connector to obtain the detection result of each pin needle, the method further comprises: when the detection results of all pin needles of the connector are unqualified, determining the unqualified reasons of each pin needle; when the unqualified reasons of all pin needles are the same, obtaining the running parameters of the production equipment corresponding to the connector; determining whether the running parameters are out of the normal range; if the running parameters are out of the normal range, sending a command to stop the operation of the production equipment by a control module; if the running parameters are not out of the normal range, sending a first prompt information to the operator, the first prompt information being used to prompt the operator that the standard model of the connector may be inaccurate.

[0017] With the above technical solution, when it is detected that all pin needles of the connector are unqualified and the unqualified reasons are the same, whether the equipment is abnormal during the production of the connector is determined by obtaining the running parameters of the production equipment corresponding to the connector, if the equipment is abnormal, the operation of the equipment is stopped by a control module to avoid the equipment to continue producing unqualified connectors, thereby reducing unnecessary increase of production cost and invalid consumption of production resources; if the equipment is normal, it indicates that the standard model for detecting the pin needles of the connector may be inaccurate, at this time, a prompt information is sent to the operator to remind the operator to timely detect whether the standard model is accurate, thereby ensuring that the detection process is accurately performed.

[0018] In some embodiments of the first aspect, after the step of performing anomaly detection on each pin according to the second standard coordinates and the second actual coordinates of the pins in the connector to obtain a detection result of each pin, the method further comprises: determining whether the actual offset angle exceeds a preset offset angle threshold; if yes, acquiring a reference image of a preset reference point beside the connector by the vision detection module based on a preset scanning path; determining whether the reference image matches a standard reference image of the reference point stored in advance; and if not, sending a second prompt information to the operator, the second prompt information being used to prompt the operator that the fixed position of the vision detection module is offset.

[0019] With the above technical solution, when the actual offset angle exceeds the preset offset angle threshold, the image of the preset reference point is acquired, and the acquired image is matched with the standard reference image of the reference point stored in advance to determine whether the actual offset angle exceeding the preset offset angle threshold is caused by the offset of the fixed position of the vision detection module. When it is determined that the fixed position of the vision detection module is offset, prompt information is immediately sent to the operator to remind the operator to timely adjust the vision detection module, so as to avoid the offset angle of the vision detection module being increased due to untimely processing, thereby affecting the accuracy of the detection result.

[0020] In the second aspect, the embodiments of the present application provide a detection server, comprising: one or more processors and a memory; the memory is coupled with the one or more processors, and the memory is used to store computer program codes, the computer program codes comprising computer instructions, and the one or more processors invoke the computer instructions to enable the detection server to perform the method described in the first aspect and any possible implementation manner of the first aspect.

[0021] In the third aspect, the embodiments of the present application provide a computer readable storage medium comprising instructions, when the instructions run on the detection server, enable the detection server to perform the method described in the first aspect and any possible implementation manner of the first aspect.

[0022] In the fourth aspect, the present application provides a computer program product, when the computer program product runs on the detection server, enables the detection server to perform the method described in the first aspect and any possible implementation manner of the first aspect.

[0023] It can be understood that the electronic device provided in the second aspect, the storage medium provided in the third aspect and the computer program product provided in the fourth aspect are all used to perform the method provided by the present application. Therefore, the beneficial effects that can be achieved thereby can refer to the beneficial effects in the corresponding method, which will not be described here again.

[0024] The one or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages:

[0025] 1、The present application calculates the offset angle of the connector placement position by the standard coordinates and actual coordinates of the anchor point, and then rotates the actual coordinate system according to the offset angle to obtain a detection coordinate system, and then calculates the actual coordinates of each pin needle in the detection coordinate system according to the detection coordinate system and the scan image of each pin needle, so that the calculated actual coordinates can more accurately reflect the actual position and height of the pin needle, and the accuracy of calculating the actual coordinates of the pin needle is improved. At the same time, the pin needle is detected according to the calculated actual coordinates of the pin needle, so that the detection result of the pin needle is more accurate, and the accuracy of the detection result caused by the deviation of the connector placement position is avoided, and the detection precision of the pin needle is effectively improved.

[0026] 2、The present application calculates the pin needle detection sequence that can make the detection time shortest through the path planning algorithm, and detects the pin needle according to the detection sequence, which reduces the total time of detecting each pin needle and improves the detection efficiency of the pin needle. At the same time, the first standard coordinate of the anchor point with the shortest distance from the first detected pin needle is selected as the reference coordinate, so that the actual position of each pin needle is more accurate and efficient, the accuracy of collecting the pin needle image is effectively improved, and the accuracy of calculating the actual coordinates of the pin needle is improved.

[0027] 3、The present application determines the shape of the anchor pattern through the coordinates of the anchor point, calculates the actual offset angle of the connector according to different offset angle calculation methods selected according to the shape of the anchor pattern, and comprehensively considers the characteristics of two-dimensional patterns and non-two-dimensional patterns, so that the calculated actual offset angle is more accurate, and the accuracy of calculating the actual coordinates of the pin needle is improved. BRIEF DESCRIPTION OF DRAWINGS

[0028] Figure 1 is an application scene schematic diagram of the pin needle visual detection method in the embodiments of the present application;

[0029] Figure 2 is a flowchart of the pin needle visual detection method in the embodiments of the present application;

[0030] Figure 3 is another flowchart of the pin needle visual detection method in the embodiments of the present application;

[0031] Figure 4 is an exemplary hardware structure schematic diagram of the detection server in the embodiments of the present application. DETAILED DESCRIPTION

[0032] The terminology used in the following embodiments of the present application is for the purpose of describing particular embodiments only and is not intended to be limiting of the present application. As used in the description of the embodiments of the application and the appended claims, the singular forms "a", "an" and "the" are intended to include both singular and plural forms, unless the context clearly indicates otherwise. It will be further understood that the terms "and / or", as used in the present application, refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0033] Hereinafter, the terms "first", "second" are only for the purpose of description, and cannot be understood as implying or suggesting relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include one or more of the features, and in the description of the embodiments of the present application, the meaning of "multiple" is two or more, unless otherwise specified.

[0034] Figure 1 is a schematic diagram of an application scenario of the pin visual detection method in the embodiments of the present application.

[0035] Please refer to Figure 1 , there is a connector on the connector, the connector has a plurality of pin, above the pin has two scanning cameras, the two scanning cameras are at a certain angle, for collecting the images of the pin at different angles, and transmitting the collected images to the detection system.

[0036] Among them, the scanning camera can be but not limited to a line laser scanning camera, a plane array scanning camera, an optical imaging camera and the like, the number of scanning cameras can be one or more, and the placement position of the scanning camera is determined according to the actual demand, which is not limited to only one of the above, which is not limited here.

[0037] After receiving the pin image transmitted by the scanning camera, the detection system will calculate the actual position and height of the pin according to the image, and then detect the abnormality of the pin according to the calculated data. However, since the shooting angle and the field of view range of the camera are pre-set based on the standard placement position of the connector, and the imaging principle of the camera is based on fixed optical geometry, when the placement position of the connector deviates, the actual position of the pin in the camera coordinate system will not match the pre-set standard position, resulting in the change of the projection position and posture on the image plane. This change will cause deviation in the extraction and analysis of the pin feature points in the image based on the pre-set algorithm, and further cause deviation in the pin position information and height information calculated from the image, so that false judgment is easy to occur when detecting the pin, and the detection accuracy of the pin is reduced.

[0038] In order to ensure that the placement position of the connector is consistent with the standard placement position each time, the placement position of the connector is usually fixed by making a mold, but due to the existence of certain tolerances in the manufacture and use of the mold itself, and the wear of the mold during use, the placement position of the connector in the mold will have a slight deviation. Therefore, using the mold cannot completely avoid the deviation of the placement position of the connector from the standard placement position, thereby affecting the detection accuracy of the pin needle.

[0039] The pin needle visual detection method in the embodiment of the application sets multiple anchor points, compares the actual coordinates of each anchor point with the standard coordinates to calculate the offset angle of the placement position of the connector, and then establishes a detection coordinate system according to the offset angle, so that the actual coordinates of the pin needle can be accurately calculated by calculating the actual coordinates of the pin needle through the detection coordinate system. The pin needle is detected through the actual coordinates obtained by calculation, which avoids the error in the calculation of the actual coordinates of the pin needle due to the deviation of the placement position, and further avoids the misjudgment of the pin needle during detection, thereby effectively improving the detection accuracy of the pin needle.

[0040] The method of the embodiment of the application will be described below in combination with Figure 2 .

[0041] Please refer to Figure 2 , which is a flowchart of the pin needle visual detection method in the embodiment of the application.

[0042] S201, a standard model of a connector is obtained.

[0043] According to the standard drawing of the connector, the standard model of the connector is established.

[0044] The standard drawing records the size specifications, arrangement positions, shapes and structures of each pin needle in the connector. The standard model includes a standard coordinate system, multiple anchor points, first standard coordinates of each anchor point in the standard coordinate system, and second standard coordinates of each pin needle in the standard coordinate system. The standard coordinate system is determined based on the anchor point position relationship between the anchor points in the standard model, and the anchor point position relationship is fixed. The x-axis and y-axis of the second standard coordinates represent the position of the pin needle, and the z-axis represents the height of the pin needle.

[0045] Specifically, the standard model is used to detect the pins in the connector. When establishing the standard model, the pin-related information needs to be screened from the standard drawing of the connector, including the position of the pin, the height of the pin, and the like. Then, according to the screened pin-related information, a professional modeling software is used to create the model, and when creating the model, the position and height of each pin need to be ensured to be completely consistent with the position and height of the corresponding pin in the standard drawing. After the model is created, the anchor points and the standard coordinate system need to be set by the operator on the model. Finally, according to the standard coordinate system set by the operator, the coordinates of each anchor point and the coordinates of the vertex of each pin are calculated, and the calculated data is saved in the model to obtain the standard model.

[0046] S202, acquiring a product image of the connector through a visual detection module.

[0047] After the step of S201 is executed, the product image of the connector is collected by the scanning camera in the visual detection module.

[0048] The number of scanning cameras in the visual detection module can be two or more, and the scanning cameras can be 2D scanning cameras such as line laser scanning cameras and area array scanning cameras. The placement positions of the scanning cameras are determined according to the types of the scanning cameras. For example, if the scanning camera is a line laser scanning camera, the placement positions of the scanning cameras can be parallel, and the laser scanning line of the camera is perpendicular to the surface of the connector and parallel to the pin to be detected. If the scanning camera is an area array scanning camera, the scanning cameras can be placed at a certain angle.

[0049] Specifically, if the scanning camera is a line laser scanning camera, the connector is scanned row by row through a pre-set product scanning path to gradually build the entire product image of the connector. If the scanning camera is an area array scanning camera, the focal length, aperture, and shooting angle of the camera are adjusted to make the lens field of view cover the entire connector, and then the entire product image of the connector is obtained by shooting.

[0050] S203, calculating first actual coordinates of each anchor point in the actual coordinate system in the product image through an image recognition algorithm.

[0051] Based on the anchor points in the standard model, the anchor points in the product image are found through a feature matching algorithm in image recognition, and the first actual coordinates of each anchor point in the actual coordinate system are calculated through a side line fitting algorithm or a circle fitting algorithm. The actual coordinate system is determined according to the anchor point position relationship between the anchor points in the product image. When the connector does not deviate, the coordinates of each anchor point in the actual coordinate system are consistent with the coordinates in the standard coordinate system.

[0052] Specifically, first, the features of each anchor point are obtained according to the related information of each anchor point in the standard model, and then the corresponding regions in the product image that match the features of each anchor point are found through a feature matching algorithm. Then, for the corresponding regions in the product image that match the features of each anchor point in the standard model, that is, the anchor points in each product image that are determined, the specific contour shape is further analyzed.

[0053] If the contour of the anchor point tends to be composed of straight line segments, the first actual coordinates of the anchor point in the actual coordinate system are calculated through a side line fitting algorithm; if the contour of the anchor point presents a relatively obvious circular feature, the first actual coordinates of the anchor point in the actual coordinate system are calculated through a circle fitting algorithm.

[0054] S204, according to the first actual coordinates and the first standard coordinates of each anchor point, the actual offset angle of the connector placement position is calculated.

[0055] According to the first actual coordinates and the first standard coordinates of each anchor point, a vector is constructed, and the actual offset angle of the connector placement position is calculated through the vector. The actual offset angle includes the offset angle of the connector placement position relative to each coordinate axis, such as the offset angle relative to the x-axis, the offset angle relative to the y-axis, and the offset angle relative to the z-axis.

[0056] Specifically, first, the first actual coordinates and the first standard coordinates of each anchor point are obtained, and for each anchor point, a coordinate difference vector is constructed by subtracting the first standard coordinates from the first actual coordinates. For the offset angle of the x-axis, one or more vectors are selected from the plurality of coordinate difference vectors, the unit vector projection of these vectors in the x-axis direction is calculated, then the dot product of the corresponding projection vectors and the modulus of the projection vectors are calculated, the cosine value of the vector included angle is obtained by substituting the cosine formula of the vector included angle, and the inverse cosine function is used to obtain the offset angle of the x-axis. In the same way, the offset angles of the y-axis and the z-axis are calculated. Finally, the offset angles of the x-axis, the y-axis and the z-axis are sorted and summarized to obtain the actual offset angle of the connector placement position.

[0057] In some embodiments, a plurality of different vector combinations can be selected, and these vectors are substituted into the spatial vector included angle formula to calculate a plurality of different actual offset angles, and then the average or weighted average of these offset angles is calculated to determine the actual offset angle of the connector placement position, so that the calculated actual offset angle is more accurate and reliable.

[0058] S205, according to the actual offset angle and the actual coordinate system, a detection coordinate system is established.

[0059] The detection coordinate system is established by rotating the actual coordinate system according to the actual offset angle.

[0060] The detection coordinate system is obtained by rotating the actual coordinate system according to the actual offset angle. The third actual coordinate of each anchor point in the detection coordinate system matches the first standard coordinate of the corresponding anchor point in the standard coordinate system. The third actual coordinate is obtained by performing coordinate system transformation on the first actual coordinate according to the actual offset angle.

[0061] Specifically, firstly, the offset angles of the connector placement relative to the x-axis, y-axis, and z-axis are determined based on the actual offset angles. Then, the actual coordinate system is rotated around the x-axis, y-axis, and z-axis respectively to obtain the detection coordinate system. During the rotation of the coordinate system around the x-axis, y-axis, and z-axis, after each rotation, the coordinates of each anchor point in the rotated coordinate system are calculated using the coordinate calculation formula.

[0062] In some embodiments, to make the established detection coordinate system more accurate, the third actual coordinates of each anchor point can be matched with its corresponding first standard coordinates. The difference between the third actual coordinates and the first standard coordinates of each anchor point is compared to see if it is within a pre-set reasonable error range. If it is within the reasonable error range, the established detection coordinate system is accurate and reliable; if it is not within the reasonable error range, the detection coordinate system needs to be further adjusted to ensure that the established detection coordinate system is accurate and reliable.

[0063] S206. Determine the scanning path of the visual inspection module based on the first standard coordinates of each anchor point and the second standard coordinates of each pin.

[0064] After completing step S201, the first standard coordinates of each anchor point and the second standard coordinates of each pin are obtained to determine the scanning path of the visual inspection module.

[0065] Specifically, when the scanning camera is a line laser scanning camera, first select an anchor point as the starting point of the scanning path. Using this starting point as the starting position, according to the distribution pattern of the pins, plan a scanning path that can pass through each pin in sequence.

[0066] When the scanning camera is an area scan camera, it is necessary to divide the area composed of multiple anchor points into several sub-regions containing pins based on the first standard coordinates of each anchor point and the second standard coordinates of each pin, and then determine the scanning path of each sub-region according to certain rules.

[0067] S207. According to the scanning path, the visual inspection module sequentially scans each pin in the connector to obtain a scanned image of each pin.

[0068] The number of scanning images of each pin needle is determined by the number of scanning cameras in the visual detection module, and in general, the number of scanning images of each pin needle is two or more.

[0069] Specifically, when the scanning camera is a line laser scanning camera, the scanning camera is first moved to the starting position of the scanning path, and then the scanning camera is sequentially scanned along the scanning path to obtain a plurality of scanning image segments. Through image stitching and processing algorithms, the image segments of the same pin needle obtained at different scanning positions are integrated, and finally the complete scanning image of each pin needle is obtained.

[0070] When the scanning camera is a face array scanning camera, the scanning camera is first moved to the starting position of the scanning path, so that the field of view range can cover the first sub-region to be scanned. Then, according to the scanning sequence, the face array scanning camera sequentially captures the images in each sub-region to obtain the scanning images of the pin needles in each sub-region.

[0071] S208, based on the detection coordinate system and the scanning images of each pin needle, calculating the second actual coordinates of each pin needle in the connector.

[0072] Specifically, the scanning camera is first accurately calibrated to obtain its internal parameters (such as focal length, principal point coordinates, distortion coefficient, etc.) and external parameters (i.e. the rotation matrix and translation vector of the camera relative to the detection coordinate system). Then, for each pin needle scanning image, the pixel coordinates of each pin needle in the scanning image are determined through image recognition and feature extraction algorithms. Next, according to the pinhole camera model, the pixel coordinates of each pin needle are converted to normalized coordinates in the camera coordinate system using the obtained camera internal parameters, and then combined with the camera external parameters (rotation matrix and translation vector) to further convert the normalized coordinates in the camera coordinate system to the detection coordinate system to obtain the coordinate information of each pin needle in the detection coordinate system. Finally, the second actual coordinates of each pin needle are calculated by selecting a suitable fitting algorithm (such as least squares method, etc.) according to the shape characteristics of the pin needle. The pinhole camera model is an idealized camera model based on the pinhole imaging principle.

[0073] S209, according to the second standard coordinates and the second actual coordinates of each pin needle in the connector, performing anomaly detection on each pin needle to obtain the detection result of each pin needle.

[0074] The detection result includes, but is not limited to, pin needle pass / fail condition (pass or fail), pin needle position degree, pin needle height error value, pin needle position condition (normal or abnormal), and pin needle height condition (normal or abnormal), and the like, which are not limited herein.

[0075] Specifically, the position degree of each pin needle is calculated according to the x-axis coordinate and the y-axis coordinate in the second standard coordinate and the second actual coordinate of each pin needle. The position degree is a key index for measuring the deviation degree of the actual position of the pin needle in the plane relative to the standard position. After the position degree of each pin needle is calculated, the position degree of each pin needle is compared with the pre-set position degree tolerance range. If the pin needle position degree is within the tolerance range, it indicates that the position of the pin needle is normal. If the pin needle position degree is not within the tolerance range, it indicates that the position of the pin needle is abnormal.

[0076] In addition to comparing the position degree of the pin needle, the z-axis coordinate in the second standard coordinate of each pin needle is compared with the z-axis coordinate in the second actual coordinate to obtain the height error value of each pin needle. Then, it is determined whether the height error value of each pin needle is within the pre-set height error range. If the height error value of the pin needle is within the height error range, it indicates that the height of the pin needle is normal. If the height error value of the pin needle is not within the height error range, it indicates that the height of the pin needle is abnormal.

[0077] If the position and the height of the pin needle are normal, it is determined that the pin needle is qualified. If the position or the height of the pin needle is abnormal, it is determined that the pin needle is unqualified.

[0078] Finally, the detection result of each pin needle is obtained by comprehensively considering the pass / fail condition, the position degree, the height error value, the position condition, and the height condition of each pin needle.

[0079] In the embodiment of the present application, the offset angle of the connector placement position is calculated by comparing the actual coordinate and the standard coordinate of each anchor point. Then, the detection coordinate system is established according to the offset angle. The actual coordinate of the pin needle is calculated through the detection coordinate system, so that the actual coordinate of the pin needle can be accurately calculated. The pin needle is detected through the calculated actual coordinate, which avoids the error in the calculation of the actual coordinate of the pin needle due to the deviation of the placement position, and further avoids the misjudgment in the detection of the pin needle, thereby effectively improving the detection precision of the pin needle.

[0080] The method of the embodiment of the present application will be further described below. Figure 3

[0081] Please refer to Figure 3 , which is another flowchart of the pin needle visual detection method in the embodiment of the present application.​

[0082] S301, determine whether a standard model of the connector exists in the model library.

[0083] According to the first identification information of the connector, it is determined whether the second identification information of the standard model exists in the model library. If it exists, the step S302 is executed to obtain the standard model of the connector in the model library. If it does not exist, the step S303 is executed to generate the 3D model of the connector based on the standard drawing of the connector. The first identification information of the connector includes but is not limited to product model number, product version number, etc. The second identification information of the standard model includes but is not limited to the first identification information of the connector, industry standard specification number, etc. Herein, no limitation is made.

[0084] S302, obtaining the standard model corresponding to the connector in the model library.

[0085] The standard model matched with the connector in the step S301 is obtained.

[0086] S303, generating the 3D model of the connector based on the standard drawing of the connector through the model generation module.

[0087] Based on the standard drawing of the connector uploaded by the operator, the 3D model of the connector is generated through the model generation module.

[0088] Specifically, first, the model generation module is used to analyze the uploaded standard drawing and extract key design information such as part shape, size, and positional relationship. Then, the model generation module is used to construct the 3D model of each part in the connector, and the model generation module is used to assemble the parts according to the relative positional relationship of each part shown in the standard drawing to construct the complete 3D model of the connector.

[0089] S304, obtaining the positional information and height information of each pin in the 3D model through the point cloud registration algorithm.

[0090] Specifically, first, the 3D model is point clouded to obtain the original point cloud, and each point in the point cloud has its coordinate information in the three-dimensional space. Then, the point cloud containing each pin in the original point cloud is extracted as the to-be-registered point cloud, and the point cloud matching algorithm is used to match the to-be-registered point cloud with the preset reference point cloud of the pin. After the point cloud registration is completed, the coordinate distribution of the registered point cloud in the three-dimensional space is analyzed to obtain the specific positional information of each pin in the horizontal plane (i.e. x, y axis direction). As for the height information of the pin, the lowest z-axis coordinate of the bottom point cloud of the pin after registration and the highest z-axis coordinate corresponding to the top point cloud of the pin are obtained, and the difference between the two coordinates is calculated to obtain the height information of each pin.

[0091] S305, generate an initial model based on the position information and height information of each pin.

[0092] The initial model only contains the structure and information related to the pin.

[0093] Specifically, according to the height information of each pin, the morphology of each pin in the vertical direction is shaped based on the determined plane position along the z-axis direction, and the three-dimensional model of each pin is obtained. Then, one of the pins is selected as a reference pin, and the x-axis and y-axis coordinates of the pin in the three-dimensional space are set to 0, so that it becomes the plane coordinate origin, and the plane reference datum of the entire model space is constructed. For other pins, their positional relationship with the reference pin is analyzed one by one to obtain the offset of other pins relative to the reference pin in the plane, and thus the x-axis and y-axis coordinates of other pins in the three-dimensional space are obtained, and the specific position distribution of each pin in the plane is obtained. Finally, the three-dimensional models of each pin are placed and integrated according to the specific position distribution of each pin in the plane, and the initial model containing only the pin is obtained.

[0094] S306, obtain the standard coordinate system and anchor points marked by the operator in the initial model, and generate a standard model.

[0095] Obtain the standard coordinate system and anchor points marked by the operator in the initial model, generate a standard model, and store the standard model in the model library.

[0096] Specifically, based on the position information of the standard coordinate system and anchor points marked by the operator in the initial model, the first standard coordinates of each anchor point in the standard coordinate system are calculated. Then, according to the first standard coordinates of each anchor point and the relative positional relationship between each pin and the anchor point, the x-axis and y-axis coordinates of each pin in the standard coordinate system are calculated, and then according to the height information of each pin, the z-axis coordinates of each pin in the standard coordinate system are obtained. Next, the x-axis, y-axis and z-axis coordinates of each pin are combined to obtain the second standard coordinates of each pin in the standard coordinate system. Finally, the first standard coordinates of each anchor point and the second standard coordinates of each pin are saved in the marked initial model to obtain a standard model that can be used to detect each pin in the connector.

[0097] S307, obtain the product image of the connector through a visual detection module.

[0098] S308, calculate the first actual coordinates of each anchor point in the actual coordinate system in the product image through an image recognition algorithm.

[0099] Steps S307, S308 are similar to steps S202, S203 in the embodiment shown in FIG. 2, and the description in steps S202, S203 can be referred to, and thus will not be repeated here. Figure 2 Steps S202, S203 in the embodiment shown in FIG. 2 are similar to steps S202, S203 in the embodiment shown in FIG. 2, and the description in steps S202, S203 can be referred to, and thus will not be repeated here.

[0100] S309, determine whether the anchor pattern formed by the anchor points is a two-dimensional pattern.

[0101] If the anchor pattern formed by the anchor points is a two-dimensional pattern, the step S310 is executed to determine the offset angle between the first actual coordinates of the anchor points and the first standard coordinates. If the anchor pattern formed by the anchor points is not a two-dimensional pattern, the step S312 is executed to obtain the standard center point coordinates and the actual center point coordinates of the anchor pattern in the standard coordinate system.

[0102] Specifically, the first standard coordinates of the anchor points are obtained, and it is determined whether the z-axis coordinates of all the anchor points are the same. If the z-axis coordinates of all the anchor points are the same, it is determined that the anchor pattern formed by the anchor points is a two-dimensional pattern. If the z-axis coordinates of all the anchor points are not the same, it is determined that the anchor pattern formed by the anchor points is not a two-dimensional pattern.

[0103] S310, determine the offset angle between the first actual coordinates of the anchor points and the first standard coordinates.

[0104] Specifically, the vector from the first standard coordinates of each anchor point to the coordinate origin is constructed to obtain the standard vector of each anchor point, and the vector from the first actual coordinates of each anchor point to the coordinate origin is constructed to obtain the actual vector of each anchor point. For each anchor point, the standard vector and the actual vector are sequentially projected onto a two-dimensional plane (i.e., the xy plane, the yz plane, and the xz plane) by the projection method to obtain the first standard vector and the first actual vector after projection onto the xy plane, the second standard vector and the second actual vector after projection onto the xy plane, and the third standard vector and the third actual vector after projection onto the xy plane. Then, the vector cosine values between the first standard vector and the first actual vector, the second standard vector and the second actual vector, and the third standard vector and the third actual vector are calculated by the vector dot product formula and the modulus length formula, respectively, and the vector angles between the first standard vector and the first actual vector, the second standard vector and the second actual vector, and the third standard vector and the third actual vector are obtained by the inverse cosine function.

[0105] Finally, the vector angles between the first standard vector and the first actual vector, the second standard vector and the second actual vector, and the third standard vector and the third actual vector of each anchor point are summarized to obtain the offset angles of each anchor point relative to the x-axis, the y-axis, and the z-axis.

[0106] S311, calculate the average value of all the offset angles to obtain the actual offset angle.

[0107] Calculate the average of all offset angles of each anchor point relative to the x-axis, the average of all offset angles relative to the z-axis, and the average of all offset angles relative to the y-axis, respectively, to obtain the offset angle of the connector placement position relative to the x-axis, the offset angle relative to the y-axis, and the offset angle relative to the z-axis.

[0108] S312, obtain the standard center point coordinates and the actual center point coordinates of the anchor pattern in the standard coordinate system.

[0109] Specifically, the first standard coordinates and the first actual coordinates of each anchor point are obtained, the coordinate mean values in the x-axis, y-axis and z-axis directions are calculated according to the first standard coordinates of each anchor point, and then the calculated coordinate mean values in the x-axis, y-axis and z-axis directions are combined to obtain the standard center point coordinates of the anchor pattern in the standard coordinate system. Similarly, the coordinate mean values in the x-axis, y-axis and z-axis directions are calculated according to the first actual coordinates of each anchor point, and then the calculated coordinate mean values in the x-axis, y-axis and z-axis directions are combined to obtain the actual center point coordinates of the anchor pattern in the actual coordinate system. The coordinate mean value is obtained by accumulating the corresponding coordinate value and dividing by the number of anchor points.

[0110] S313, construct an offset vector from the standard center point coordinates to the actual center point coordinates.

[0111] According to the standard center point coordinates and the actual center point coordinates, the coordinate differences of the actual center point coordinates relative to the standard center point coordinates in the x-axis, y-axis and z-axis directions are calculated, and an offset vector from the standard center point coordinates to the actual center point coordinates is constructed according to the calculated coordinate differences.

[0112] S314, calculate the actual offset angle according to the offset vector.

[0113] According to the offset vector, the offset angle of the offset vector relative to the x-axis, y-axis and z-axis is calculated by projecting the offset vector to a two-dimensional plane by projection method, or the offset angle of the offset vector relative to the x-axis, y-axis and z-axis is calculated by using direction cosine method, to obtain the offset angle of the connector placement position relative to the x-axis, the offset angle relative to the y-axis, and the offset angle relative to the z-axis.

[0114] S315, establish a detection coordinate system according to the actual offset angle and the actual coordinate system.

[0115] Step S315 is similar to step S205 in the embodiment shown in Figure 2 The step S205 in the embodiment shown in

[0116] S316, according to the second standard coordinates of each pin in the connector, the pin detection sequence that can make the detection time shortest is calculated through a path planning algorithm.

[0117] Specifically, the second standard coordinate information of each pin in the connector is acquired, and the coordinate data is constructed into a format that can be processed by a path planning algorithm, such as a form of a set of coordinate points. Then, a suitable path planning algorithm is selected, for example, a greedy algorithm, a dynamic programming algorithm, etc., and the coordinate points of the pins are taken as nodes and the distance between the nodes and other related measurements are taken as the weight of the edges. Then, the shortest detection time is taken as the optimization goal, and the selected algorithm is used to start the calculation, and when the algorithm converges or reaches the preset stopping condition, the pin detection sequence that can make the detection time shortest can be obtained.

[0118] S317, the second standard coordinate of the first detected pin in the pin detection sequence is acquired as the initial detection coordinate.

[0119] According to the pin detection sequence calculated in step S316, the first detected pin in the detection sequence is determined, and the second standard coordinate of the pin is acquired as the initial detection coordinate.

[0120] S318, according to the first standard coordinates of each anchor point and the initial detection coordinate, the distance between each anchor point and the first detected pin in the pin detection sequence is calculated.

[0121] The first standard coordinates of each anchor point and the initial detection coordinate are acquired, and the distance between each anchor point and the first detected pin in the pin detection sequence is calculated through a three-dimensional space distance calculation formula.

[0122] S319, the first standard coordinate of the anchor point with the shortest distance to the first detected pin in the pin detection sequence is selected as the reference coordinate.

[0123] According to the distance between each anchor point and the first detected pin in the pin detection sequence calculated in step S318, the distance values are compared and analyzed, and the anchor point with the smallest distance is found, and the first standard coordinate of the anchor point is acquired as the reference coordinate. The reference coordinate is used to accurately locate the actual position of each pin.

[0124] S320, according to the initial coordinate of the visual detection module in the standard coordinate system, the reference coordinate, the pin detection sequence, and the first standard coordinates of each pin, the scanning path of the visual detection module is determined.

[0125] Specifically, the initial coordinates of the visual detection module in the standard coordinate system are acquired, the initial coordinates are taken as the starting point of the scanning path, and a first path from the initial coordinates to the reference coordinates is planned according to the reference coordinates and in combination with the motion characteristics (such as moving speed, turning flexibility, etc.) of the visual detection module. Then, according to the pin detection sequence, the first standard coordinates of each pin are sequentially taken as nodes on the path, and sub-paths from one pin position to the next pin position are gradually planned according to the coordinate differences between adjacent pins and in combination with the motion characteristics of the visual detection module, and the sub-paths are sequentially connected to form intermediate paths for moving between pins. Finally, the path from the initial coordinates to the reference coordinates and the intermediate paths for moving between pins are comprehensively determined to form the scanning path of the visual detection module in the entire detection process.

[0126] S321, according to the scanning path, each pin in the connector is sequentially scanned by the visual detection module to obtain a scanning image of each pin.

[0127] S322, based on the detection coordinate system and the scanning images of each pin, second actual coordinates of each pin in the connector are calculated.

[0128] S323, according to the second standard coordinates and the second actual coordinates of each pin in the connector, an abnormality detection is performed on each pin to obtain a detection result of each pin.

[0129] Steps S321-S323 are similar to steps S207-S209 in the embodiment shown in Figure 2 The steps S207-S209 in the embodiment shown in

[0130] S324, it is determined whether each pin in the connector is unqualified.

[0131] According to the detection results of each pin, the qualification of each pin is acquired, and it is determined whether the qualification of each pin is unqualified. If yes, the step of S325 is executed to determine the unqualified reasons of each pin; if no, the step of S335 is executed to continue detecting the pins of the next connector.

[0132] S325, the unqualified reasons of each pin are determined.

[0133] According to the detection results of each pin, the position and height of each pin are checked. If the position of the pin is abnormal, it indicates that the unqualified reason of the pin is position abnormality; if the height of the pin is abnormal, it indicates that the unqualified reason of the pin is height abnormality.

[0134] S326, determine whether the unqualified reasons of the pins in the connector are all the same.

[0135] According to the unqualified reasons of the pins, it is determined whether the unqualified reasons of the pins are all the same. If yes, the step S327 is executed to obtain the operation parameters of the production equipment corresponding to the connector. If no, the step S335 is executed to continue to detect the pins of the next connector.

[0136] S327, obtain the operation parameters of the production equipment corresponding to the connector.

[0137] The production equipment for producing the connector is determined through the model of the connector, and the operation parameters of the production equipment are obtained through the control system of the production equipment. The operation parameters include part positioning accuracy, assembly force control accuracy, assembly speed, etc.

[0138] S328, determine whether the operation parameters are out of the normal range.

[0139] The obtained operation parameters of the production equipment are compared with the parameter range of the operation parameters under the normal operation condition to determine whether any operation parameter of the production equipment is out of the corresponding parameter range. If yes, the step S329 is executed to send a command to stop the operation of the equipment to the production equipment through the control module. If no, the step S330 is executed to send a first prompt information to the operator.

[0140] S329, send a command to stop the operation of the equipment to the production equipment through the control module.

[0141] When it is determined that the operation parameters of the production equipment are out of the normal range, a command to stop the operation of the equipment is sent to the production equipment through the control module to stop the operation of the production equipment.

[0142] S330, send a first prompt information to the operator.

[0143] When it is determined that the operation parameters of the production equipment are not out of the normal range, a first prompt information is sent to the operator. The first prompt information is used to prompt the operator that the standard model of the connector may be inaccurate.

[0144] Specifically, the first prompt information is pushed to the operator through various channels, such as popping up a prominent pop-up window in a conspicuous position of the display screen of the equipment operation panel, and presenting the first prompt information in clear and concise words in the pop-up window. If the enterprise has a mobile application for production management, the first prompt information can also be pushed to the mobile devices such as mobile phones or tablet computers of the operator, so that the operator can receive and view the prompt information in a mobile state or at a remote location.

[0145] S331, determining whether the actual offset angle exceeds a preset offset angle threshold.

[0146] According to the actual offset angle, it is determined whether the actual offset angle exceeds a preset offset angle threshold. If it exceeds, the step of S332 is executed, and the reference image of the preset reference point beside the connector is acquired by the visual detection module. If it does not exceed, the step of S335 is executed, and the pin needle of the next connector is continuously detected.

[0147] Specifically, the offset angle of the connector placement position relative to the x-axis, the offset angle relative to the y-axis, and the offset angle relative to the z-axis in the actual offset angle are acquired. The offset angles of the x-axis, the y-axis, and the z-axis are compared with the preset offset angle thresholds of the x-axis, the y-axis, and the z-axis respectively. If the offset angles of the x-axis, the y-axis, and the z-axis do not exceed the thresholds, it is determined that the actual offset angle does not exceed the preset offset angle threshold. If the offset angle of the x-axis exceeds the threshold, or the offset angle of the y-axis exceeds the threshold, or the offset angle of the z-axis exceeds the threshold, it is determined that the actual offset angle exceeds the preset offset angle threshold.

[0148] S332, acquiring the reference image of the preset reference point beside the connector by the visual detection module.

[0149] Based on the preset scanning path, the reference image of the preset reference point beside the connector is acquired by the visual detection module.

[0150] S333, determining whether the reference image matches the standard reference image of the preset reference point.

[0151] It is determined whether the reference image matches the standard reference image of the preset reference point. If it matches, the step of S335 is executed, and the pin needle of the next connector is continuously detected. If it does not match, the step of S334 is executed, and the second prompt information is sent to the operator.

[0152] Specifically, the reference image and the standard reference image are first image preprocessed, including size normalization and denoising processing.

[0153] Then, the key features are extracted respectively. In the shape feature extraction, a contour extraction algorithm is used to calculate the perimeter, area, circularity, rectangularity and other parameters of the reference points in the reference image and the standard reference image respectively, and the shape feature parameters of the reference points in the two images are compared. For the feature point features, the scale-invariant feature transform (SIFT) algorithm or the speeded up robust features (SURF) algorithm is used to extract the feature points and the corresponding feature descriptors of the reference points in the reference image and the standard reference image, and then the matching degree of the reference image and the standard reference image is judged by matching the number of feature points, the distance between feature points and other indicators. The feature descriptor is a quantitative representation form of local features in an image (or other data types), which is essentially a vector.

[0154] Finally, a reasonable matching threshold is set according to the extracted features, such as the sum of the absolute values of the shape feature parameter differences of the reference points in the reference image and the standard reference image is less than a certain value, the matching number of feature points accounts for a certain percentage, and the average distance error of the matching feature points is within a reasonable range. If the comparison results of all features meet the corresponding matching threshold requirements, it is determined that the reference image and the standard reference image match, otherwise, it is determined that the reference image and the standard reference image do not match.

[0155] S334, sending a second prompt information to the operator.

[0156] When the reference image does not match the standard reference image of the reference point stored in advance, a second prompt information is sent to the operator. The second prompt information is used to prompt the operator that the fixed position of the visual detection module has deviated.

[0157] S335, detecting the pin needle of the next connector.

[0158] It is determined whether the connector to be detected and the currently detected connector are of the same type. If the connector to be detected and the currently detected connector are of the same type, the step of S307 is executed; if the connector to be detected and the currently detected connector are not of the same type, the step of S301 is executed.

[0159] In the embodiments of the present application, the standard model of the connector is obtained through the model library, thereby improving the efficiency of obtaining the standard model; the offset angle of the placement position of the connector is calculated in different calculation modes according to the anchor pattern composed of the anchor points, thereby improving the accuracy of calculating the offset angle; the detection coordinate system is established through the offset angle for calculating the actual coordinates of the pins, thereby improving the accuracy of calculating the actual coordinates of the pins; before planning the scanning path, the pin detection sequence that can make the detection time shortest is calculated, thereby improving the scanning speed of the pins, improving the detection efficiency of the pins, taking the anchor point closest to the pin detected first in the pin detection sequence as the reference coordinate for calculating the actual coordinates of the pins, making the positioning of the actual positions of the pins more accurate and efficient, improving the accuracy of collecting the pin images, and further improving the accuracy of calculating the actual coordinates of the pins; when the running parameters of the production equipment are detected to be abnormal, the running of the production equipment is stopped immediately, thereby avoiding the continuous production of unqualified connectors, reducing the unnecessary increase of production cost and the invalid consumption of production resources; when the fixed position of the camera in the visual detection module is detected to be offset, the operator is informed to adjust immediately, thereby avoiding the influence of the camera offset on the detection accuracy of the pins. The overall detection accuracy and efficiency of the pins are improved, and the problem of reduced detection accuracy of the pins caused by the deviation of the placement position of the connector or the camera offset is avoided.

[0160] The pin visual detection method in the embodiments of the present application is described above, and the detection server in the embodiments of the present application will be described in detail below in combination with the above pin visual detection method.

[0161] Please refer to Figure 4 , which is an example of a hardware structure of the detection server in the embodiments of the present application.

[0162] In some embodiments, the detection server 400 is a computer device, which can be a terminal device. The computer device comprises a processor 401, a memory 402, a communication module 403, an input device 404 and an output device 405 connected through a system bus. The processor 401 of the computer device is configured to provide computing and control capabilities. The memory 402 of the computer device comprises a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for running the operating system and the computer program in the non-volatile storage medium. The database is configured to store data. The communication module 403 of the computer device is configured to send prompt information to an operator and send control instructions to a control system of a production device. The input device 404 of the computer device is configured to obtain a standard drawing uploaded by the operator and an anchor point and a standard coordinate system marked on an initial model. The output device 405 of the computer device is configured to display various information, including a standard model, a detection result, etc. The computer program is executed by the processor 401 to implement the pin visual detection method in the embodiments of the present application.

[0163] Those skilled in the art can understand that, Figure 4 The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can comprise more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0164] In some embodiments of the present application, a computer readable storage medium is provided, comprising instructions which, when executed on the detection server 400, cause the detection server 400 to perform the pin visual detection method in the embodiments of the present application.

[0165] In some embodiments of the present application, a computer program product is also provided, which, when executed on the detection server 400, causes the detection server 400 to perform the pin visual detection method in the embodiments of the present application.

[0166] The above-described embodiments are only used to illustrate the technical solutions of the present application, but not limit the present application; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements to some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

[0167] In the above embodiments, the term "when" can be interpreted as meaning "if" or "after" or "in response to determining" or "in response to detecting" depending on the context. Similarly, the phrase "upon determining" or "if detecting (the stated condition or event)" can be interpreted as meaning "if determining" or "in response to determining" or "upon detecting (the stated condition or event)" or "in response to detecting (the stated condition or event)" depending on the context.

[0168] In the above embodiments, all or part of the methods can be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the methods can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer can be a general purpose computer, a special purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another computer readable storage medium, for example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line) or wireless (such as infrared, wireless, microwave, etc.) manner. The computer readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. integrated with one or more available media. The available media can be magnetic media (such as floppy disk, hard disk, magnetic tape), optical media (such as DVD), or semiconductor media (such as solid state disk), etc.

[0169] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by a computer program to instruct the relevant hardware to complete, and the program can be stored in a computer readable storage medium, and the program can include the processes of the above-mentioned method embodiments when executed. The aforementioned storage medium includes ROM or random access memory (RAM), magnetic disk or optical disk, and various media that can store program codes.

Claims

1. A visual inspection method for pins, characterized in that, include: Obtain a standard model of the connector; the standard model includes a standard coordinate system, multiple anchor points, a first standard coordinate of each anchor point in the standard coordinate system, and a second standard coordinate of each pin in the connector in the standard coordinate system; the standard coordinate system is determined based on the positional relationship between the anchor points in the standard model; the positional relationship of the anchor points is fixed; the x-axis and y-axis of the second standard coordinate system represent the position of the pin, and the z-axis represents the height of the pin; The product image of the connector is obtained through a visual inspection module; The first actual coordinates of each anchor point in the product image in the actual coordinate system are calculated using an image recognition algorithm; the actual coordinate system is determined based on the positional relationship between the anchor points in the product image. Calculate the actual offset angle of the connector placement position based on the first actual coordinates and the first standard coordinates of each anchor point; Establish a detection coordinate system based on the actual offset angle and the actual coordinate system; The detection coordinate system is obtained by rotating the actual coordinate system according to the actual offset angle; the third actual coordinate of each anchor point in the detection coordinate system matches the first standard coordinate of the corresponding anchor point in the standard coordinate system; the third actual coordinate is obtained by performing a coordinate system transformation on the first actual coordinate according to the actual offset angle. The scanning path of the vision inspection module is determined based on the first standard coordinates of each anchor point and the second standard coordinates of each pin, including: calculating the pin detection sequence that minimizes the detection time using a path planning algorithm based on the second standard coordinates of each pin in the connector; obtaining the second standard coordinates of the first pin detected in the pin detection sequence as the initial detection coordinates; calculating the distance between each anchor point and the first pin detected in the pin detection sequence based on the first standard coordinates of each anchor point and the initial detection coordinates; selecting the first standard coordinates of the anchor point with the shortest distance to the first pin detected in the pin detection sequence as the reference coordinates; the reference coordinates are used to accurately locate the actual position of each pin; and determining the scanning path of the vision inspection module based on the initial coordinates of the vision inspection module in the standard coordinate system, the reference coordinates, the pin detection sequence, and the first standard coordinates of each pin. According to the scanning path, the visual inspection module sequentially scans each pin in the connector to obtain a scanned image of each pin; Based on the detection coordinate system and the scanned images of each pin, the second actual coordinates of each pin in the connector are calculated; Based on the second standard coordinates and the second actual coordinates of each pin in the connector, anomaly detection is performed on each pin to obtain the detection results of each pin; the anomaly detection includes position detection and height detection.

2. The method according to claim 1, characterized in that, The step of calculating the actual offset angle of the connector placement position based on the first actual coordinates and the first standard coordinates of each anchor point specifically includes: Based on the first standard coordinates of each anchor point in the standard model, determine whether the anchoring pattern formed by each anchor point is a two-dimensional pattern; If the anchoring graphic is a two-dimensional graphic, then the offset angle between the first actual coordinates and the first standard coordinates of each anchoring point is determined based on the first actual coordinates and the first standard coordinates of each anchoring point. Calculate the average of all the aforementioned offset angles to obtain the actual offset angle; If the anchoring graphic is not a two-dimensional graphic, obtain the standard center point coordinates and the actual center point coordinates of the anchoring graphic in the standard coordinate system; Construct an offset vector pointing from the standard center point coordinates to the actual center point coordinates; The actual offset angle is calculated based on the offset vector.

3. The method according to claim 1, characterized in that, The standard model for obtaining the connector specifically includes: Determine if a standard model for a connector exists in the model library; If it exists, then obtain the standard model corresponding to the connector from the model library; If it does not exist, then a standard model shall be established based on the standard drawings of the connector.

4. The method according to claim 3, characterized in that, If the aforementioned does not exist, then based on the standard drawings of the connector, a standard model is established, specifically including: Based on the standard drawings of the connector, a 3D model of the connector is generated through the model generation module; The position and height information of each pin in the 3D model are obtained by point cloud registration algorithm; An initial model is generated based on the position and height information of each pin. Obtain the standard coordinate system and anchor points marked by the operator in the initial model, and generate the standard model.

5. The method according to claim 1, characterized in that, After the step of performing anomaly detection on each pin based on the second standard coordinate and the second actual coordinate of each pin in the connector, and obtaining the detection result of each pin, the method further includes: When the test results for all pins of the connector are unqualified, determine the reason for the unqualification of each pin; When all the reasons for non-compliance of the pins are the same, obtain the operating parameters of the production equipment corresponding to the connector; Determine whether the operating parameters are outside the normal range; If the operation exceeds the normal range, the control module sends a command to stop the equipment operation to the production equipment. If the information is within the normal range, a first prompt message is sent to the operator; the first prompt message is used to indicate to the operator that the standard model of the connector may be inaccurate.

6. The method according to claim 1, characterized in that, After the step of performing anomaly detection on each pin based on the second standard coordinate and the second actual coordinate of each pin in the connector, and obtaining the detection result of each pin, the method further includes: Determine whether the actual offset angle exceeds a preset offset angle threshold; If so, then based on the preset scanning path, the visual detection module obtains a reference image of the preset reference point next to the connector; Determine whether the reference image matches a pre-stored standard reference image of the reference point; If there is a mismatch, a second prompt message is sent to the operator; the second prompt message is used to indicate to the operator that the fixed position of the visual detection module has shifted.

7. A detection server, characterized in that, include: One or more processors and memory; The memory is coupled to the one or more processors, the memory being used to store computer program code, the computer program code including computer instructions, the one or more processors invoking the computer instructions to cause the detection server to perform the method as described in any one of claims 1-6.

8. A computer-readable storage medium storing computer instructions, characterized in that, When the computer instructions are executed on the detection server, the detection server performs the method as described in any one of claims 1-6.

9. A computer program product, characterized in that, When the computer program product is run on the detection server, the detection server performs the method as described in any one of claims 1-6.

Images