Connector pin detection method and system

By acquiring connector images on a transparent plate and constructing point clouds and depth maps, and using techniques such as normal vectors and Gaussian curvature, efficient and automatic detection of connector pins is achieved, solving the problem of insufficient recognition capability in existing technologies and improving detection efficiency and accuracy.

CN118761999BActive Publication Date: 2024-11-12SHENZHEN SANWORD AUTOMATION EQUIP
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411229526.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-03
Publication Date
2024-11-12
Estimated Expiration
2044-09-03

AI Technical Summary

Technical Problem

In existing technologies, the detection efficiency of connector pins is low, especially in identifying pin bending and scratches. Manual inspection is inefficient, and image stitching methods are complex and have limited recognition capabilities.

Method used

By placing the connector on a transparent plate, collecting image data and preprocessing it, constructing the planar equation of the connector point cloud, extracting normal vectors to generate a depth map, locating the pin position, calculating gray values, dividing regular shape regions, using Gaussian curvature and binarized gray values ​​to identify scratches, and line curvature to identify bends, thus achieving automatic detection.

Benefits of technology

It improves the ability to identify pin scratches and bends, accurately locates abnormal parts, and enhances detection efficiency and accuracy.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118761999B_ABST
    Figure CN118761999B_ABST
Patent Text Reader

Abstract

The present invention relates to the field of machine vision technology, and discloses a connector pin detection method and system, the method comprising: detecting a connector point cloud in a preprocessed image, constructing a plane equation of the connector point cloud, extracting a normal vector of the connector from the plane equation, generating a depth map of the connector using the normal vector, locating the pin position in the connector using the depth map, and calculating the gray value of the connector in the preprocessed image using the normal vector; calculating the Gaussian curvature in a regular shape area using the depth map, and using the Gaussian curvature and the binary gray value to determine whether there is an abnormal scratch in the pin position, and obtaining a scratch determination result; extracting the pin line in the pin position using the binary gray value, performing an initial line cutting on the pin line, and obtaining an initial cut line, and using the line curvature to determine whether there is an abnormal bend in the pin line, and obtaining a bend determination result. The present invention can improve the ability to identify scratches and bends of the pin.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of machine vision technology, and in particular to a connector pin detection method and system. Background Art

[0002] The process of connector pin detection refers to the process of using image processing technology to identify the position of the pin in the image and whether there are any abnormalities in the pin. The abnormalities of the pin mainly include bending of the pin, which makes it unable to enter the socket on the socket, and scratches on the surface of the pin make the protective coating on the surface of the pin ineffective.

[0003] At present, most factories still use manual visual observation to detect whether the pins are bent or scratched. The efficiency of manual detection is obviously not high enough. Furthermore, some research programs apply image recognition technology to pin identification. By collecting connector images and matching the connector images with standard template images, the position of the pins can be identified. However, due to the shooting angle problem, a photo taken at one shooting angle cannot accurately determine the position of the pin, which leads to the use of multi-angle image stitching to achieve pin identification. However, when stitching images, it is necessary to consider the differences in texture and clarity of images taken by cameras at different angles, which makes the image registration before image stitching more complicated. In addition, this method can only identify the position of the pin in the image, while the identification of scratches and bends is relatively vague, because the light of the captured image is insufficient. Therefore, the connector pin detection is not capable of identifying scratches and bends on the pins. Summary of the invention

[0004] In order to solve the above problems, the present invention provides a connector pin detection method and system, which can improve the connector pin detection ability to identify scratches and bends of the pins.

[0005] In a first aspect, the present invention provides a connector pin detection method, comprising:

[0006] After placing the connector on the transparent plate, collecting image data of the connector on the transparent plate, and performing image preprocessing on the image data to obtain a preprocessed image;

[0007] Detecting a connector point cloud in the preprocessed image, constructing a plane equation of the connector point cloud, extracting a normal vector of the connector from the plane equation, generating a depth map of the connector using the normal vector, locating a pin position in the connector using the depth map, and calculating a grayscale value of the connector in the preprocessed image using the normal vector;

[0008] Divide the regular shape area in the pin position, calculate the Gaussian curvature in the regular shape area using the depth map, binarize the gray value to obtain a binary gray value, and use the Gaussian curvature and the binary gray value to determine whether there is an abnormal scratch in the pin position to obtain a scratch determination result;

[0009] Extracting the pin line in the pin position by using the binary grayscale value, performing initial line cutting on the pin line to obtain an initial cutting line, calculating the line curvature of the initial cutting line, and using the line curvature to determine whether the pin line has abnormal bending to obtain a bending determination result;

[0010] The pin position, the scratch identification result and the bend identification result are used as the automatic detection result of the pin of the connector.

[0011] In a possible implementation manner of the first aspect, extracting the normal vector of the connector from the plane equation includes:

[0012] The plane equation is converted into a matrix equation using the following formula:

[0013] ;

[0014] ;

[0015] ;

[0016] ;

[0017] ;

[0018] in, represents the matrix equation, represents the normal vector of the connector, Representation equation , represents the plane equation, , , Represents the 3D coordinates of the i-th k-nearest neighbor point cloud in the k-nearest neighbor point cloud of the connector point cloud, Represents the total number of k-nearest neighbor point clouds of the connector point cloud;

[0019] The normal vector of the connector is extracted from the matrix equation by singular value decomposition method.

[0020] In a possible implementation manner of the first aspect, generating a depth map of the connector by using the normal vector includes:

[0021] According to the normal vector, the depth gradient of the connector is calculated using the following formula:

[0022] ;

[0023] ;

[0024] in, represents the depth gradient, represents the depth gradient in a three-dimensional image, represents a normal vector of the connector;

[0025] Based on the depth gradient, the depth of the connector is calculated using the following formula:

[0026] ;

[0027] ;

[0028] ;

[0029] ;

[0030] in, Indicates the depth of the connector, represents the depth gradient, and x and y represent independent variables;

[0031] A depth map of the connector is generated using the depth of the connector.

[0032] In a possible implementation manner of the first aspect, locating a pin position in the connector by using the depth map includes:

[0033] identifying highly consistent connector regions in the depth map;

[0034] Determining whether the number of the connector areas is greater than a preset number;

[0035] When the number of the connector areas is greater than the preset number, detecting whether the boundaries between each of the connector areas in the connector areas are connected;

[0036] When the boundaries between each connector area in the connector area are not connected, obtaining the area bottom surface of the connector area;

[0037] Calculate the center point-border distance between the center point on the bottom surface of the region and the border;

[0038] When the variance of the center point-boundary distance is less than a preset variance, the bottom surface of the area and the connector area are used as the pin positions.

[0039] In a possible implementation manner of the first aspect, calculating the grayscale value of the connector in the preprocessed image by using the normal vector includes:

[0040] Acquire a pin image and a non-pin image of the connector in the pre-processed image using the pin position;

[0041] respectively collecting the pin reflectivity and the non-pin reflectivity of the pin image and the non-pin image;

[0042] Based on the pin reflectivity and the non-pin reflectivity, the grayscale value of the connector in the pre-processed image is calculated using the following formula:

[0043] ;

[0044] in, Represents the gray value, Indicates the grayscale value corresponding to the pin image in the preprocessed image, Indicates the grayscale value corresponding to the non-pin image in the preprocessed image, represents the light intensity on the pin image, represents the light intensity on the non-pin image, Indicates the pin reflectivity, Represents the non-pin reflectivity, It represents the direction vector of the light reflected from the pin position to the camera after the light source emitted by the camera hits the pin position. It represents the direction vector of the light reflected from the non-pin position to the camera after the light source emitted by the camera hits the non-pin position. represents the normal vector at the pin position, Represents the normal vector at non-pin locations.

[0045] In a possible implementation manner of the first aspect, calculating the Gaussian curvature in the regular shape area by using the depth map includes:

[0046] The basic form of the surface in the regular shape area in the depth map is constructed using the following formula:

[0047] ;

[0048] ;

[0049] in, Represents the first basic form of the surface in the basic form of the surface, Represents the second basic form of the surface in the basic form of the surface, Represents the first-order partial derivative of the grayscale value at the coordinate point (x, y) in the depth map on the x-axis, Represents the first-order partial derivative of the grayscale value at the coordinate point (x, y) in the depth map on the y-axis, Represents the second-order partial derivative of the grayscale value at the coordinate point (x, y) in the depth map on the x-axis, Represents the second-order partial derivative of the grayscale value at the coordinate point (x, y) in the depth map on the x-axis and y-axis, Represents the second-order partial derivative of the grayscale value at the coordinate point (x, y) in the depth map on the y-axis;

[0050] Based on the basic form of the surface, the Gaussian curvature within the regular shape area is calculated using the following formula:

[0051] ;

[0052] in, represents the Gaussian curvature, Represents the first-order partial derivative of the grayscale value at the coordinate point (x, y) in the depth map on the x-axis, Represents the first-order partial derivative of the grayscale value at the coordinate point (x, y) in the depth map on the y-axis, Represents the second-order partial derivative of the grayscale value at the coordinate point (x, y) in the depth map on the x-axis, Represents the second-order partial derivative of the grayscale value at the coordinate point (x, y) in the depth map on the x-axis and y-axis, Represents the second-order partial derivative of the grayscale value at the coordinate point (x, y) in the depth map on the y-axis.

[0053] In a possible implementation of the first aspect, the using the Gaussian curvature and the binary grayscale value to determine whether there is an abnormal scratch in the pin position to obtain a scratch determination result includes:

[0054] Using the binary grayscale value to identify whether there is a similar scratch area in the pin position;

[0055] When the similar scratch area exists in the pin insertion position, checking whether the size of the similar scratch area is consistent with that of the pin insertion position;

[0056] When the size of the similar scratch area is inconsistent with that of the pin insertion position, selecting the minimum curvature value of the Gaussian curvature of the non-similar scratch area in the pin insertion position except the similar scratch area;

[0057] Determining whether the Gaussian curvature of the scratch-like region is less than the minimum curvature value;

[0058] When the Gaussian curvature of the scratch-like region is less than the minimum curvature value, it is determined that an abnormal scratch exists in the pin position, and a first scratch identification result is obtained;

[0059] When the Gaussian curvature of the scratch-like region is not less than the minimum curvature value, it is determined that no abnormal scratch exists in the pin position, and a second scratch discrimination result is obtained.

[0060] In a possible implementation manner of the first aspect, extracting the pin lines in the pin positions by using the binary grayscale values ​​includes:

[0061] Extracting a similar pin region in the pin position by using the binary grayscale value;

[0062] Obtaining a pin normal vector on the pin-like area;

[0063] Based on the pin normal vector, the divergence on the similar pin area is calculated using the following formula:

[0064] ;

[0065] in, represents the divergence over the pin-like region, represents the pin normal vector;

[0066] When the divergence is less than a preset divergence, determining whether the length of the pin-like region is a maximum length;

[0067] When the length of the pin-like region is the maximum length, the pin-like region is used as the pin line in the pin position.

[0068] In a possible implementation of the first aspect, the step of using the line curvature to determine whether the pin line is abnormally bent to obtain a bending determination result includes:

[0069] Determining whether a curvature standard deviation of the line curvature is greater than a preset standard deviation;

[0070] When the curvature standard deviation of the line curvature is not greater than the preset standard deviation, doubling the line length of the initial cutting line corresponding to the line curvature to obtain a doubled length line;

[0071] The curvature of the doubled length line is used to determine whether the pin line has abnormal bending, and a bending determination result is obtained.

[0072] In a second aspect, the present invention provides a connector pin detection system, the system comprising:

[0073] An image preprocessing module, used for collecting image data of the connector on the transparent board after placing the connector on the transparent board, and performing image preprocessing on the image data to obtain a preprocessed image;

[0074] A grayscale calculation module, used to detect a connector point cloud in the preprocessed image, construct a plane equation of the connector point cloud, extract a normal vector of the connector from the plane equation, generate a depth map of the connector using the normal vector, locate a pin position in the connector using the depth map, and calculate a grayscale value of the connector in the preprocessed image using the normal vector;

[0075] a scratch identification module, used to divide the regular shape area in the pin position, calculate the Gaussian curvature in the regular shape area by using the depth map, binarize the gray value to obtain a binary gray value, and use the Gaussian curvature and the binary gray value to determine whether there is an abnormal scratch in the pin position to obtain a scratch identification result;

[0076] A bending determination module is used to extract the pin line in the pin position by using the binary grayscale value, perform initial line cutting on the pin line to obtain an initial cutting line, calculate the line curvature of the initial cutting line, and use the line curvature to determine whether the pin line has abnormal bending to obtain a bending determination result;

[0077] The result determination module is used to use the pin position, the scratch determination result and the bend determination result as the automatic detection result of the pin of the connector.

[0078] Compared with the prior art, the technical principle and beneficial effects of this solution are:

[0079] The embodiment of the present invention places the connector on a transparent plate to photograph the bottom surface of the connector through the transparent plate. The embodiment of the present invention detects the connector point cloud in the preprocessed image to use the connector point cloud to detect the depth of different parts of the connector in the preprocessed image. Furthermore, the embodiment of the present invention extracts the normal vector of the connector from the plane equation to pave the way for subsequently determining the depth map of the connector using the normal vector. Furthermore, the embodiment of the present invention generates the depth map of the connector by using the normal vector to observe the depth of different parts of the connector from the depth map. Furthermore, the embodiment of the present invention locates the pin position in the connector by using the depth map to determine the base area of ​​the pin on the connector joint and the area where the pin itself is located to prevent other modules of the connector from being confused with the pin. Furthermore, the embodiment of the present invention calculates the grayscale image of the connector in the preprocessed image by using the normal vector. The degree value is used to separate the scratch from the background area when a scratch appears on the connector. The embodiment of the present invention divides the regular shape area in the pin position to divide the area belonging to the whole in the pin position into a whole and separate the area not belonging to the whole. Further, the embodiment of the present invention calculates the Gaussian curvature in the regular shape area by using the depth map to determine whether there is a concave scratch in the regular shape area. Further, the embodiment of the present invention determines whether there is an abnormal scratch in the pin position by using the Gaussian curvature and the binary grayscale value, so as to use the dual data of the binary image and the curvature to identify the scratch. Further, the embodiment of the present invention determines whether there is an abnormal bend in the pin line by using the line curvature, so as to calculate the curvature from the smaller segmentation line, and determine which specific line segments have a larger curvature by gradually increasing the length of the segmentation line, so as to accurately locate the bending part. Therefore, a connector pin detection method and system proposed in the embodiment of the present invention can improve the ability of connector pin detection to identify scratches and bends of the pin. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0081] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0082] Figure 1A schematic diagram of a flow chart of a connector pin detection method provided by an embodiment of the present invention;

[0083] Figure 2 A schematic diagram of a module of a connector pin detection system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0084] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0085] An embodiment of the present invention provides a connector pin detection method, and the execution subject of the connector pin detection method includes but is not limited to at least one of the electronic devices such as a server and a terminal that can be configured to execute the method provided by the embodiment of the present invention. In other words, the connector pin detection method can be executed by software or hardware installed in a terminal device or a server device, and the software can be a blockchain platform. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, etc. The server can be an independent server, or it can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and big data and artificial intelligence platforms.

[0086] See also Figure 1 FIG. 1 is a flow chart of a connector pin detection method provided by an embodiment of the present invention. Figure 1 The connector pin detection methods described in include:

[0087] S1. After placing a connector on a transparent plate, collecting image data of the connector on the transparent plate, and performing image preprocessing on the image data to obtain a preprocessed image.

[0088] In the embodiment of the present invention, the connector is placed on a transparent plate so as to photograph the bottom surface of the connector through the transparent plate.

[0089] The connector refers to an electrical connector, which is composed of a connector and pins on the connector. For example, the charger head is the connector, and the metal sheet on the charger head is the pin.

[0090] Furthermore, in an embodiment of the present invention, the image data refers to multi-angle images under strong light illumination, including images from the front, back, side and bottom vision.

[0091] Optionally, the process of performing image preprocessing on the image data to obtain a preprocessed image refers to a process of removing noise and background areas in the image.

[0092] S2. Detect the connector point cloud in the preprocessed image, construct a plane equation of the connector point cloud, extract the normal vector of the connector from the plane equation, generate a depth map of the connector using the normal vector, locate the pin position in the connector using the depth map, and calculate the grayscale value of the connector in the preprocessed image using the normal vector.

[0093] The embodiment of the present invention detects the connector point cloud in the pre-processed image, so as to use the connector point cloud to detect the depth of different parts of the connector in the pre-processed image.

[0094] The connector point cloud refers to a three-dimensional point cloud.

[0095] Optionally, the process of detecting the connector point cloud in the pre-processed image may achieve mapping between three-dimensional point cloud coordinates and two-dimensional image coordinate data through camera intrinsic parameters.

[0096] Optionally, the process of constructing the plane equation of the connector point cloud is performed by formula Construct, where , , Represents the 3D coordinates of the i-th k-nearest neighbor point cloud in the k-nearest neighbor point cloud of the connector point cloud, Represents the normal vector of the connector.

[0097] Furthermore, the embodiment of the present invention extracts the normal vector of the connector from the plane equation to pave the way for subsequently determining the depth map of the connector using the normal vector.

[0098] In one embodiment of the present invention, extracting the normal vector of the connector from the plane equation includes: converting the plane equation into a matrix equation using the following formula:

[0099] ;

[0100] ;

[0101] ;

[0102] ;

[0103] ;

[0104] in, represents the matrix equation, represents the normal vector of the connector, Representation equation , represents the plane equation, , , Represents the 3D coordinates of the i-th k-nearest neighbor point cloud in the k-nearest neighbor point cloud of the connector point cloud, Represents the total number of k-nearest neighbor point clouds of the connector point cloud;

[0105] The normal vector of the connector is extracted from the matrix equation by singular value decomposition method.

[0106] Furthermore, the embodiment of the present invention generates a depth map of the connector by using the normal vector, so as to obtain the depths of different parts of the connector by observing from the depth map.

[0107] In one embodiment of the present invention, the generating the depth map of the connector by using the normal vector includes: calculating the depth gradient of the connector by using the following formula according to the normal vector:

[0108] ;

[0109] ;

[0110] in, represents the depth gradient, represents the depth gradient in a three-dimensional image, represents a normal vector of the connector;

[0111] Based on the depth gradient, the depth of the connector is calculated using the following formula:

[0112] ;

[0113] ;

[0114] ;

[0115] ;

[0116] in, Indicates the depth of the connector, represents the depth gradient, and x and y represent independent variables;

[0117] A depth map of the connector is generated using the depth of the connector.

[0118] Furthermore, the embodiment of the present invention locates the position of the pin in the connector by utilizing the depth map to determine the base area of ​​the pin on the connector joint and the area where the pin itself is located, so as to prevent modules at other heights of the connector from being mixed with the pin.

[0119] The pin position includes the bottom surface of the area and the pin area.

[0120] In one embodiment of the present invention, the use of the depth map to locate the pin position in the connector includes: identifying highly consistent connector areas in the depth map; determining whether the number of the connector areas is greater than a preset number; when the number of the connector areas is greater than the preset number, detecting whether the boundaries between each connector area in the connector area are connected; when the boundaries between each connector area in the connector area are not connected, obtaining the bottom surface of the connector area; calculating the center point-boundary distance between the center point on the bottom surface of the area and the boundary; when the variance of the center point-boundary distance is less than a preset variance, using the bottom surface of the area and the connector area as the pin position.

[0121] Among them, the connector area is generally pins of the same height, and the preset number is 2. Generally, the number of pins on a connector joint is at least 2. It should be noted that the pins are generally distributed on the joint surface like independent columns. Therefore, the boundaries between each connector area in the connector area are not connected. Furthermore, the bottom surface of the area refers to the area where the connector area is connected to the joint, that is, the bottom surface of the connector area (column). The preset variance is used to ensure that the discrete degree of the center point-boundary distance is small, so that when the variance of the center point-boundary distance is less than the preset variance, it indicates that the bottom surface of the area is a regular circular area or a bottom surface (usually a line) area formed when the metal sheet is erected.

[0122] Furthermore, the embodiment of the present invention calculates the grayscale value of the connector in the preprocessed image by using the normal vector, so that when scratches appear on the connector, the scratches can be separated from the background area.

[0123] In one embodiment of the present invention, the calculating the grayscale value of the connector in the preprocessed image by using the normal vector includes: obtaining a pin image and a non-pin image of the connector in the preprocessed image by using the pin position; respectively collecting the pin reflectivity and the non-pin reflectivity of the pin image and the non-pin image; and calculating the grayscale value of the connector in the preprocessed image by using the following formula based on the pin reflectivity and the non-pin reflectivity:

[0124]

[0125] in, Represents the gray value, Indicates the grayscale value corresponding to the pin image in the preprocessed image, Indicates the grayscale value corresponding to the non-pin image in the preprocessed image, represents the light intensity on the pin image, represents the light intensity on the non-pin image, Indicates the pin reflectivity, Represents the non-pin reflectivity, It represents the direction vector of the light reflected from the pin position to the camera after the light source emitted by the camera hits the pin position. It represents the direction vector of the light reflected from the non-pin position to the camera after the light source emitted by the camera hits the non-pin position. represents the normal vector at the pin position, Represents the normal vector at non-pin locations.

[0126] The pin image refers to the image of the position corresponding to the pin position.

[0127] S3. Divide the regular shape area in the pin position, use the depth map to calculate the Gaussian curvature in the regular shape area, binarize the grayscale value to obtain a binary grayscale value, and use the Gaussian curvature and the binary grayscale value to determine whether there is an abnormal scratch in the pin position to obtain a scratch determination result.

[0128] The embodiment of the present invention divides the pin position into regular-shaped areas, so as to divide the areas belonging to an integrated whole in the pin position into a whole, and separate the areas not belonging to an integrated whole.

[0129] The regularly shaped area includes the pin base area and the pin area in the pin position. For example, when the pin is a cylindrical pin, the base at the bottom of the area in the pin position is a protruding circular ring.

[0130] Furthermore, the embodiment of the present invention calculates the Gaussian curvature in the regular shape region by using the depth map to determine whether there is a concave scratch in the regular shape region.

[0131] In one embodiment of the present invention, the calculating the Gaussian curvature in the regular shape region by using the depth map includes: constructing a basic form of the surface in the regular shape region in the depth map by using the following formula:

[0132] ;

[0133] ;

[0134] in, Represents the first basic form of the surface in the basic form of the surface, Represents the second basic form of the surface in the basic form of the surface, Represents the first-order partial derivative of the grayscale value at the coordinate point (x, y) in the depth map on the x-axis, Represents the first-order partial derivative of the grayscale value at the coordinate point (x, y) in the depth map on the y-axis, Represents the second-order partial derivative of the grayscale value at the coordinate point (x, y) in the depth map on the x-axis, Represents the second-order partial derivative of the grayscale value at the coordinate point (x, y) in the depth map on the x-axis and y-axis, Represents the second-order partial derivative of the grayscale value at the coordinate point (x, y) in the depth map on the y-axis;

[0135] Based on the basic form of the surface, the Gaussian curvature within the regular shape area is calculated using the following formula:

[0136] ;

[0137] in, represents the Gaussian curvature, Represents the first-order partial derivative of the grayscale value at the coordinate point (x, y) in the depth map on the x-axis, Represents the first-order partial derivative of the grayscale value at the coordinate point (x, y) in the depth map on the y-axis, Represents the second-order partial derivative of the grayscale value at the coordinate point (x, y) in the depth map on the x-axis, Represents the second-order partial derivative of the grayscale value at the coordinate point (x, y) in the depth map on the x-axis and y-axis, Represents the second-order partial derivative of the grayscale value at the coordinate point (x, y) in the depth map on the y-axis.

[0138] Optionally, the process of binarizing the grayscale value to obtain the binarized grayscale value includes mean filtering, image difference and binarization.

[0139] Furthermore, the embodiment of the present invention uses the Gaussian curvature and the binary grayscale value to determine whether there is an abnormal scratch in the pin position, so as to identify the scratch by using the dual data of the binary image and the curvature.

[0140] In one embodiment of the present invention, the method of using the Gaussian curvature and the binary grayscale value to discriminate whether there is an abnormal scratch in the pin position to obtain a scratch discrimination result includes: using the binary grayscale value to identify whether there is a similar scratch area in the pin position; when the similar scratch area exists in the pin position, checking whether the size of the similar scratch area is consistent with that of the pin position; when the size of the similar scratch area is inconsistent with that of the pin position, selecting the minimum curvature value of the Gaussian curvature of the non-similar scratch area in the pin position except the similar scratch area; judging whether the Gaussian curvature of the similar scratch area is less than the minimum curvature value; when the Gaussian curvature of the similar scratch area is less than the minimum curvature value, judging that there is an abnormal scratch in the pin position to obtain a first scratch discrimination result; when the Gaussian curvature of the similar scratch area is not less than the minimum curvature value, judging that there is no abnormal scratch in the pin position to obtain a second scratch discrimination result.

[0141] Among them, the similar scratch area refers to an area whose binary grayscale value is obviously different from the binary grayscale value of the background area. It should be noted that if there is a scratch in the pin position, the existing scratch is smaller than the size of the pin position. Therefore, if there is a scratch in the pin position, the similar scratch area is inconsistent in size with the pin position.

[0142] S4. Using the binary grayscale value to extract the pin line in the pin position, performing initial line cutting on the pin line to obtain an initial cutting line, calculating the line curvature of the initial cutting line, and using the line curvature to determine whether the pin line has abnormal bending to obtain a bending determination result.

[0143] In the embodiment of the present invention, the pin line refers to the pins excluding the bottom surface of the area in the pin position.

[0144] In one embodiment of the present invention, the method of extracting the pin lines in the pin position by using the binary grayscale value includes: extracting a similar pin area in the pin position by using the binary grayscale value; obtaining a pin normal vector on the similar pin area; and calculating the divergence on the similar pin area based on the pin normal vector by using the following formula:

[0145] ;

[0146] in, represents the divergence over the pin-like region, represents the pin normal vector;

[0147] When the divergence is less than a preset divergence, it is determined whether the length of the pin-like region is the maximum length; when the length of the pin-like region is the maximum length, the pin-like region is used as the pin line in the pin position.

[0148] The pin-like region refers to a region whose grayscale value is obviously different from the grayscale value of the joint, and the preset divergence refers to a divergence with a value of 0.

[0149] Optionally, the process of performing initial cutting on the pin line to obtain the initial cut line refers to the process of dividing the pin line according to very small unit intervals, which paves the way for the subsequent calculation of the curvature of each divided line. This is because calculating the curvature from a smaller divided line can determine which specific line segments have a larger curvature by gradually increasing the length of the divided line, thereby accurately locating the bending part.

[0150] The line curvature refers to a value indicating the degree of curvature of the curve at a certain point.

[0151] Furthermore, the embodiment of the present invention utilizes the line curvature to determine whether the pin line has abnormal bending, so as to calculate the curvature starting from a smaller dividing line, and determine which specific line segments have a larger curvature by gradually increasing the length of the dividing line, thereby accurately locating the bending part.

[0152] In one embodiment of the present invention, the use of the line curvature to determine whether the pin line has abnormal bending to obtain a bending determination result includes: determining whether the curvature standard deviation of the line curvature is greater than a preset standard deviation; when the curvature standard deviation of the line curvature is not greater than the preset standard deviation, doubling the line length of the initial cutting line corresponding to the line curvature to obtain a doubled length line; using the curvature of the doubled length line to determine whether the pin line has abnormal bending to obtain a bending determination result.

[0153] The preset standard deviation is mainly used to determine the degree of discreteness of the line curvature.

[0154] Optionally, the process of doubling the line length of the initial cutting line corresponding to the line curvature is a process of combining every two adjacent initial cutting lines into one.

[0155] S5. Using the pin position, the scratch identification result and the bend identification result as the automatic detection result of the pins of the connector.

[0156] It can be seen that the embodiment of the present invention places the connector on a transparent plate to photograph the bottom surface of the connector through the transparent plate. The embodiment of the present invention detects the connector point cloud in the preprocessed image to use the connector point cloud to detect the depth of different parts of the connector in the preprocessed image. Further, the embodiment of the present invention extracts the normal vector of the connector from the plane equation to pave the way for subsequently determining the depth map of the connector using the normal vector. Further, the embodiment of the present invention generates a depth map of the connector using the normal vector to observe the depth of different parts of the connector from the depth map. Further, the embodiment of the present invention locates the position of the pin in the connector by using the depth map to determine the base area of ​​the pin on the connector joint and the area where the pin itself is located to prevent modules of other heights of the connector from being mixed with the pin. Further, the embodiment of the present invention calculates the connector in the preprocessed image by using the normal vector. The grayscale value of the device is used to separate the scratch from the background area when a scratch appears on the connector. The embodiment of the present invention divides the regular shape area in the pin position to divide the area belonging to the whole in the pin position into a whole and separate the area not belonging to the whole. Further, the embodiment of the present invention calculates the Gaussian curvature in the regular shape area by using the depth map to determine whether there is a concave scratch in the regular shape area. Further, the embodiment of the present invention determines whether there is an abnormal scratch in the pin position by using the Gaussian curvature and the binary grayscale value, so as to use the dual data of the binary image and the curvature to identify the scratch. Further, the embodiment of the present invention determines whether there is an abnormal bend in the pin line by using the line curvature, so as to calculate the curvature from the smaller segmentation line, and determine which specific line segments have a larger curvature by gradually increasing the length of the segmentation line, so as to accurately locate the bending part. Therefore, a connector pin detection method and system proposed in the embodiment of the present invention can improve the recognition ability of the connector pin detection for the scratches and bends of the pin.

[0157] like Figure 2 The figure shows a functional module diagram of the connector pin detection system of the present invention.

[0158] The connector pin detection system 200 of the present invention can be installed in an electronic device. According to the functions to be implemented, the connector pin detection system can include an image preprocessing module 201, a grayscale calculation module 202, a scratch discrimination module 203, a bend discrimination module 204 and a result determination module 205. The module of the present invention can also be called a unit, which refers to a series of computer program segments that can be executed by an electronic device processor and can complete fixed functions, which are stored in the memory of the electronic device.

[0159] In the embodiment of the present invention, the functions of each module / unit are as follows:

[0160] The image preprocessing module 201 is used to collect image data of the connector on the transparent board after placing the connector on the transparent board, and perform image preprocessing on the image data to obtain a preprocessed image;

[0161] The grayscale calculation module 202 is used to detect the connector point cloud in the preprocessed image, construct the plane equation of the connector point cloud, extract the normal vector of the connector from the plane equation, generate a depth map of the connector using the normal vector, locate the pin position in the connector using the depth map, and calculate the grayscale value of the connector in the preprocessed image using the normal vector;

[0162] The scratch identification module 203 is used to divide the regular shape area in the pin position, calculate the Gaussian curvature in the regular shape area using the depth map, binarize the gray value to obtain a binary gray value, and use the Gaussian curvature and the binary gray value to determine whether there is an abnormal scratch in the pin position to obtain a scratch identification result;

[0163] The bending determination module 204 is used to extract the pin line in the pin position by using the binary grayscale value, perform initial line cutting on the pin line to obtain an initial cutting line, calculate the line curvature of the initial cutting line, and use the line curvature to determine whether the pin line has abnormal bending to obtain a bending determination result;

[0164] The result determination module 205 is used to use the pin position, the scratch identification result and the bend identification result as the automatic detection result of the pin of the connector.

[0165] In detail, each module in the connector pin detection system 200 in the embodiment of the present invention is used in the same manner as described above. Figure 1 The connector pin detection method described in the present invention has the same technical means and can produce the same technical effect, so it will not be repeated here.

[0166] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional modules.

[0167] It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0168] Therefore, no matter from which point of view, the embodiments should be regarded as illustrative and non-restrictive, and the scope of the present invention is limited by the appended claims rather than the above description, so it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention. Any attached figure mark in the claims should not be regarded as limiting the claims involved.

[0169] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0170] The above description is only a specific embodiment of the present invention, so that those skilled in the art can understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but should conform to the widest scope consistent with the principles and novel features invented herein.

Claims

1. A connector pin detection method, characterized in that: The method comprises: After placing the connector on the transparent plate, collecting image data of the connector on the transparent plate, and performing image preprocessing on the image data to obtain a preprocessed image; Detecting a connector point cloud in the preprocessed image, constructing a plane equation of the connector point cloud, extracting a normal vector of the connector from the plane equation, generating a depth map of the connector using the normal vector, locating a pin position in the connector using the depth map, and calculating a grayscale value of the connector in the preprocessed image using the normal vector, wherein generating a depth map of the connector using the normal vector includes: According to the normal vector, the depth gradient of the connector is calculated using the following formula: ; ; in, represents the depth gradient, represents the depth gradient in a three-dimensional image, represents a normal vector of the connector; Based on the depth gradient, the depth of the connector is calculated using the following formula: ; ; ; ; in, Indicates the depth of the connector, represents the depth gradient, and x and y represent independent variables; generating a depth map of the connector using the depth of the connector; Divide the regular shape area in the pin position, calculate the Gaussian curvature in the regular shape area by using the depth map, binarize the gray value to obtain a binary gray value, and use the Gaussian curvature and the binary gray value to determine whether there is an abnormal scratch in the pin position to obtain a scratch determination result, wherein the calculation of the Gaussian curvature in the regular shape area by using the depth map includes: The basic form of the surface in the regular shape area in the depth map is constructed using the following formula: ; ; in, Represents the first basic form of the surface in the basic form of the surface, Represents the second basic form of the surface in the basic form of the surface, Represents the first-order partial derivative of the grayscale value at the coordinate point (x, y) in the depth map on the x-axis, Represents the first-order partial derivative of the grayscale value at the coordinate point (x, y) in the depth map on the y-axis, Represents the second-order partial derivative of the grayscale value at the coordinate point (x, y) in the depth map on the x-axis, Represents the second-order partial derivative of the grayscale value at the coordinate point (x, y) in the depth map on the x-axis and y-axis, Represents the second-order partial derivative of the grayscale value at the coordinate point (x, y) in the depth map on the y-axis; Based on the basic form of the surface, the Gaussian curvature within the regular shape area is calculated using the following formula: ; in, represents the Gaussian curvature, Represents the first-order partial derivative of the grayscale value at the coordinate point (x, y) in the depth map on the x-axis, Represents the first-order partial derivative of the grayscale value at the coordinate point (x, y) in the depth map on the y-axis, Represents the second-order partial derivative of the grayscale value at the coordinate point (x, y) in the depth map on the x-axis, Represents the second-order partial derivative of the grayscale value at the coordinate point (x, y) in the depth map on the x-axis and y-axis, Represents the second-order partial derivative of the grayscale value at the coordinate point (x, y) in the depth map on the y-axis; Extracting the pin line in the pin position by using the binary grayscale value, performing initial line cutting on the pin line to obtain an initial cutting line, calculating the line curvature of the initial cutting line, and using the line curvature to determine whether the pin line has abnormal bending to obtain a bending determination result; The pin position, the scratch identification result and the bend identification result are used as the automatic detection result of the pin of the connector.

2. The method according to claim 1, characterized in that The extracting the normal vector of the connector from the plane equation comprises: The plane equation is converted into a matrix equation using the following formula: ; ; ; ; ; in, represents the matrix equation, represents the normal vector of the connector, Representing equations , represents the plane equation, , , Represents the 3D coordinates of the i-th k-nearest neighbor point cloud in the k-nearest neighbor point cloud of the connector point cloud, Represents the total number of k-nearest neighbor point clouds of the connector point cloud; The normal vector of the connector is extracted from the matrix equation by singular value decomposition method.

3. The method according to claim 1, characterized in that The method of locating the pin position in the connector by using the depth map includes: identifying highly consistent connector regions in the depth map; Determining whether the number of the connector areas is greater than a preset number; When the number of the connector areas is greater than the preset number, detecting whether the boundaries between each of the connector areas in the connector areas are connected; When the boundaries between each connector area in the connector area are not connected, obtaining the area bottom surface of the connector area; Calculate the center point-border distance between the center point on the bottom surface of the region and the border; When the variance of the center point-boundary distance is less than a preset variance, the bottom surface of the area and the connector area are used as the pin positions.

4. The method according to claim 1, characterized in that The step of calculating the grayscale value of the connector in the preprocessed image by using the normal vector includes: Acquire a pin image and a non-pin image of the connector in the pre-processed image using the pin position; respectively collecting the pin reflectivity and the non-pin reflectivity of the pin image and the non-pin image; Based on the pin reflectivity and the non-pin reflectivity, the grayscale value of the connector in the pre-processed image is calculated using the following formula: ; in, Represents the gray value, Indicates the grayscale value corresponding to the pin image in the preprocessed image, Indicates the grayscale value corresponding to the non-pin image in the preprocessed image, represents the light intensity on the pin image, represents the light intensity on the non-pin image, Indicates the pin reflectivity, Represents the non-pin reflectivity, It represents the direction vector of the light reflected from the pin position to the camera after the light source emitted by the camera hits the pin position. It represents the direction vector of the light reflected from the non-pin position to the camera after the light source emitted by the camera hits the non-pin position. represents the normal vector at the pin position, Represents the normal vector at non-pin locations.

5. The method according to claim 1, characterized in that: The method of using the Gaussian curvature and the binary grayscale value to determine whether there is an abnormal scratch in the pin position and obtaining a scratch determination result includes: Using the binary grayscale value to identify whether there is a similar scratch area in the pin position; When the similar scratch area exists in the pin insertion position, checking whether the size of the similar scratch area is consistent with that of the pin insertion position; When the size of the similar scratch area is inconsistent with that of the pin insertion position, selecting the minimum curvature value of the Gaussian curvature of the non-similar scratch area in the pin insertion position except the similar scratch area; Determining whether the Gaussian curvature of the scratch-like region is less than the minimum curvature value; When the Gaussian curvature of the scratch-like region is less than the minimum curvature value, it is determined that an abnormal scratch exists in the pin position, and a first scratch identification result is obtained; When the Gaussian curvature of the scratch-like region is not less than the minimum curvature value, it is determined that no abnormal scratch exists in the pin position, and a second scratch discrimination result is obtained.

6. The method according to claim 1, characterized in that The step of extracting the pin lines in the pin positions by using the binary grayscale values ​​comprises: Extracting a similar pin region in the pin position by using the binary grayscale value; Obtaining a pin normal vector on the pin-like area; Based on the pin normal vector, the divergence on the similar pin area is calculated using the following formula: ; in, represents the divergence on the pin-like region, represents the pin normal vector; When the divergence is less than a preset divergence, determining whether the length of the pin-like region is a maximum length; When the length of the pin-like region is the maximum length, the pin-like region is used as the pin line in the pin position.

7. The method according to claim 1, characterized in that The method of using the line curvature to determine whether the pin line is abnormally bent and obtaining a bending determination result includes: Determining whether a curvature standard deviation of the line curvature is greater than a preset standard deviation; When the curvature standard deviation of the line curvature is not greater than the preset standard deviation, doubling the line length of the initial cutting line corresponding to the line curvature to obtain a doubled length line; The curvature of the doubled length line is used to determine whether the pin line has abnormal bending, and a bending determination result is obtained.

8. A connector pin detection system, characterized in that: The system comprises: An image preprocessing module, used for collecting image data of the connector on the transparent board after placing the connector on the transparent board, and performing image preprocessing on the image data to obtain a preprocessed image; A grayscale calculation module is used to detect the connector point cloud in the preprocessed image, construct a plane equation of the connector point cloud, extract a normal vector of the connector from the plane equation, generate a depth map of the connector using the normal vector, locate the pin position in the connector using the depth map, and calculate the grayscale value of the connector in the preprocessed image using the normal vector, wherein generating the depth map of the connector using the normal vector includes: According to the normal vector, the depth gradient of the connector is calculated using the following formula: ; ; in, represents the depth gradient, represents the depth gradient in a three-dimensional image, represents a normal vector of the connector; Based on the depth gradient, the depth of the connector is calculated using the following formula: ; ; ; ; in, Indicates the depth of the connector, represents the depth gradient, and x and y represent independent variables; generating a depth map of the connector using the depth of the connector; A scratch identification module is used to divide the regular shape area in the pin position, calculate the Gaussian curvature in the regular shape area by using the depth map, binarize the gray value to obtain the binary gray value, and use the Gaussian curvature and the binary gray value to determine whether there is an abnormal scratch in the pin position to obtain a scratch identification result, wherein the calculation of the Gaussian curvature in the regular shape area by using the depth map includes: The basic form of the surface in the regular shape area in the depth map is constructed using the following formula: ; ; in, Represents the first basic form of the surface in the basic form of the surface, Represents the second basic form of the surface in the basic form of the surface, Represents the first-order partial derivative of the grayscale value at the coordinate point (x, y) in the depth map on the x-axis, Represents the first-order partial derivative of the grayscale value at the coordinate point (x, y) in the depth map on the y-axis, Represents the second-order partial derivative of the grayscale value at the coordinate point (x, y) in the depth map on the x-axis, Represents the second-order partial derivative of the grayscale value at the coordinate point (x, y) in the depth map on the x-axis and y-axis, Represents the second-order partial derivative of the grayscale value at the coordinate point (x, y) in the depth map on the y-axis; Based on the basic form of the surface, the Gaussian curvature within the regular shape area is calculated using the following formula: ; in, represents the Gaussian curvature, Represents the first-order partial derivative of the grayscale value at the coordinate point (x, y) in the depth map on the x-axis, Represents the first-order partial derivative of the grayscale value at the coordinate point (x, y) in the depth map on the y-axis, Represents the second-order partial derivative of the grayscale value at the coordinate point (x, y) in the depth map on the x-axis, Represents the second-order partial derivative of the grayscale value at the coordinate point (x, y) in the depth map on the x-axis and y-axis, Represents the second-order partial derivative of the grayscale value at the coordinate point (x, y) in the depth map on the y-axis; A bending determination module is used to extract the pin line in the pin position by using the binary grayscale value, perform initial line cutting on the pin line to obtain an initial cutting line, calculate the line curvature of the initial cutting line, and use the line curvature to determine whether the pin line has abnormal bending to obtain a bending determination result; The result determination module is used to use the pin position, the scratch determination result and the bend determination result as the automatic detection result of the pin of the connector.