Connector pin defect intelligent detection method and system based on machine vision

Abstract

The invention provides a connector pin defect intelligent detection method and system based on machine vision, and relates to the technical field of machine vision, and the method comprises the steps: obtaining a multi-view image under a multi-light-source illumination condition, calculating three-dimensional shape data, extracting a pin region, building a correlation edge, and combining the characteristics of a surface normal vector, a boundary contour and the like to obtain a pin defect detection result; and optimizing node features based on a graph neural network, carrying out pin instance extraction and defect classification, and iteratively updating a defect label through a defect propagation probability. According to the invention, various defects of the connector pins can be accurately identified, the detection precision and efficiency are improved, and the omission ratio and the false detection ratio are reduced.

Description

Technical Field

[0001] This invention relates to the field of machine vision technology, and in particular to a machine vision-based intelligent detection method and system for connector pin defects. Background Technology

[0002] Electronic connectors are a crucial component of electronic devices, and their quality directly impacts the reliability and lifespan of these devices. Connector pins are key components responsible for transmitting electrical signals, and defects such as deformation, breakage, and misalignment may occur during the manufacturing process.

[0003] As electronic products become smaller and more integrated, connector pins are becoming smaller and closer together, which places higher demands on pin defect detection technology.

[0004] Currently, connector pin defect detection mainly relies on manual visual inspection and traditional machine vision technology. Manual inspection is inefficient and cannot meet the needs of large-scale production. Traditional machine vision technology is mainly based on two-dimensional image analysis, usually using images acquired under single light source illumination for processing. Existing technologies still have shortcomings, such as the inability of the acquired images to fully reflect the three-dimensional geometric features of the pins, the tendency to miss defects at certain angles, the lack of consideration for the spatial correlation between pins, the difficulty in effectively identifying complex defects caused by the interaction of adjacent pins, and insufficient sensitivity to minor defects on the pin surface, especially in high-density connectors where minute pin bends or surface scratches are often difficult to detect accurately. Summary of the Invention

[0005] This invention provides a machine vision-based intelligent detection method and system for connector pin defects, which can at least solve some of the problems existing in the prior art.

[0006] A first aspect of this invention provides a machine vision-based intelligent detection method for connector pin defects, comprising: A multi-view image sequence of the connector to be tested under multi-source illumination conditions is acquired. The three-dimensional topography data is obtained by solving the pixel grayscale values ​​of the same spatial position under different illumination angles. Edge pixels with significant surface curvature changes are extracted from the three-dimensional topography data and grouped according to common characteristics to obtain candidate pin regions. After spatial position verification of the candidate pin regions, pin regions that meet the preset design specifications are retained. Node identifiers are assigned to each pin region and associated edges are established between spatially adjacent and posture-coordinated pin regions. The surface normal vector and boundary contour corresponding to the pin region are used as node features. The node features are propagated to neighboring nodes along the associated edge and the neighbor propagated features are aggregated to obtain optimized node features. Based on the optimized node features, a set of pin instances is determined. The surface morphology features of each pin instance in the set of pin instances are extracted from the 3D morphology data and concatenated with the optimized node features to obtain a high-dimensional feature representation. After performing multi-scale convolution operation on the high-dimensional feature representation, the initial detection result is obtained by decoding. Calculate the defect propagation probability between each pair of spatially adjacent pin instances in the pin instance set, and iteratively update the defect type label of each pin in the preliminary detection result based on the defect propagation probability to obtain the defect detection result.

[0007] In one alternative implementation, A multi-view image sequence of the connector to be tested under multi-source illumination conditions is acquired. Three-dimensional topography data is obtained by calculating the pixel grayscale values ​​of the same spatial location under different illumination angles. Edge pixels with significant surface curvature changes are extracted from the three-dimensional topography data and grouped according to common characteristics to obtain candidate pin regions, including: Multiple light sources are lit sequentially according to a preset time sequence. When each light source is lit, the camera captures an image of the connector to be detected. The number of images of the connector to be detected is the same as the number of light sources, and they are combined to obtain a multi-view image sequence. For each pixel position in the multi-view image sequence, extract multiple pixel gray values ​​corresponding to the pixel position when different light sources are lit. Substitute the multiple pixel gray values ​​and the incident angle of the corresponding light source into the preset photometric stereo constraint equation for solving, and obtain the surface normal vector and reflectivity of the spatial point corresponding to each pixel position. Traverse all pixel positions to obtain three-dimensional topography data. The surface curvature value is obtained by calculating the gradient of the surface normal vector of each spatial point in the three-dimensional topography data in the horizontal and vertical directions. Spatial points with surface curvature values ​​greater than a curvature threshold are marked as candidate edge pixels. The spatial distance between any two candidate edge pixels is calculated. Candidate edge pixels with spatial distances less than a preset connectivity distance are grouped into the same connectivity group. For each connectivity group, the spatial coordinates of all candidate edge pixels in the connectivity group are extracted and convex hull calculation is performed to obtain the minimum closed contour surrounding the connectivity group. The minimum closed contour is used as the boundary contour, and candidate pin regions are determined based on the boundary contour.

[0008] In one alternative implementation, After spatial position verification of the candidate pin regions, pin regions that conform to the preset design specifications are retained. Each pin region is assigned a node identifier, and association edges are established between spatially adjacent and attitude-coordinated pin regions, including: The centroids corresponding to the three-dimensional coordinates of all spatial points within the candidate pin region are calculated to obtain the center coordinates of the candidate pin region. Principal component decomposition is performed on the boundary contour of the candidate pin region to obtain the principal direction vector, which is used as the attitude direction of the candidate pin region. The center coordinates are projected onto the horizontal reference plane of the spatial coordinate system to obtain the projected coordinates. The candidate pin regions are then arranged in a grid according to the row and column positions of the projected coordinates to obtain a two-dimensional array distribution. For each grid position in the two-dimensional array distribution, when the number of candidate pin regions in the grid position is greater than one, the candidate pin region with the closest boundary contour area to the standard pin area and the highest boundary contour integrity is retained, and other candidate pin regions in the grid position are removed to obtain the redundant pin region. The absolute value of the difference between the center coordinate distance of the redundant pin region corresponding to the adjacent grid positions in the two-dimensional array distribution and the preset pin design spacing is calculated as the center distance deviation. The redundant pin regions with a center distance deviation less than the position verification threshold are retained as pin regions that meet the design specifications. For each pin region, a node identifier is generated according to the row index and column index in the two-dimensional array distribution. The inner product of the attitude direction vector of the pin region and the corresponding pin region of the adjacent grid position is calculated as the attitude cooperation metric. An association edge is established between adjacent pin regions whose attitude cooperation metric is greater than the cooperation threshold.

[0009] In one alternative implementation, Using the surface normal vector and boundary contour corresponding to the pin region as node features, the node features are propagated to neighboring nodes along the associated edges, and the neighbor propagation features are aggregated to obtain optimized node features, including: The surface normal vectors of all spatial points within the boundary contour corresponding to the stitch area are obtained and statistical analysis is performed to obtain the mean vector and variance value of the surface normal vectors. The mean vector and variance value are then concatenated to form the surface normal vector feature. The boundary contour of the stitch region is sampled to obtain a sequence of key points. The spatial distance and angle change between adjacent key points in the sequence of key points are calculated and spliced ​​to obtain the boundary contour features. The surface normal vector features and the boundary contour features are spliced ​​to obtain the node features corresponding to the stitch region. For each pin region, the node identifiers of all pin regions connected to the current pin region through the associated edge are found according to the node identifier of the pin region to obtain the neighboring node identifiers. The node features of the pin regions corresponding to all neighboring node identifiers are extracted to obtain the neighboring node feature set. The weight value is determined based on the associated edge, and the node features in the neighboring node feature set are weighted and summed according to the weight value to obtain the neighborhood propagation feature. Calculate the feature similarity between the node features and the neighborhood propagation features, determine the adaptive weight coefficient based on the feature similarity, and perform feature fusion on the node features and neighborhood propagation features corresponding to the pin region according to the adaptive weight coefficient to obtain optimized node features.

[0010] In one alternative implementation, Based on the optimized node features, a set of pin instances is determined. The surface morphology features of each pin instance in the set are extracted from the 3D morphology data and concatenated with the optimized node features to obtain a high-dimensional feature representation. After performing multi-scale convolution operations on the high-dimensional feature representation, decoding yields preliminary detection results, including: Based on the feature similarity between the optimized node features, the feature distance between each pair of optimized node features is calculated using a hierarchical clustering algorithm. Optimized node features whose feature distance is less than a preset initial clustering threshold are merged into an initial cluster. The different initial clusters with the smallest inter-cluster distance are repeatedly merged until the inter-cluster distance is greater than the preset merging distance threshold, thus obtaining optimized clusters and using the pin regions corresponding to the optimized clusters as pin instances. The three-dimensional topography data within the spatial range enclosed by the boundary contour in the pin instance is obtained as a topography data block. The direction distribution statistics of the surface normal vectors in the topography data block are performed to obtain a normal vector distribution histogram. The height distribution statistics of the spatial points in the topography data block are performed to obtain a height distribution histogram and are spliced ​​with the normal vector distribution histogram to obtain the surface topography features. The optimized node features corresponding to the pin regions contained in the pin instance are averaged to obtain aggregated optimized node features, which are then concatenated with the surface morphology features to obtain a high-dimensional feature representation. Multi-scale dilated convolution is performed on the high-dimensional feature representation to obtain adaptive feature maps of multiple receptive field scales. Channel splicing and decoding are performed on the adaptive feature maps to obtain the defect type prediction probability distribution and defect location prediction coordinates, which are then combined to obtain preliminary detection results.

[0011] In one alternative implementation, Multi-scale dilated convolution is performed on the high-dimensional feature representation to obtain adaptive feature maps with multiple receptive field scales. Channel concatenation and decoding are performed on the adaptive feature maps to obtain the defect type prediction probability distribution and defect location prediction coordinates. These are then combined to obtain preliminary detection results, including: The high-dimensional feature representation is input into multiple pre-defined dilated convolutional layers with increasing dilation rates to perform dilated convolution operations to obtain initial feature maps at multiple receptive field scales. For each initial feature map, the offset of the pixel position is learned through convolution operations. The sampling position of the convolution kernel is adaptively adjusted according to the offset and the convolution operation is performed again to obtain adaptive feature maps at multiple receptive field scales. The adaptive feature maps are concatenated along the channel dimension to obtain a concatenated feature map. The concatenated feature map is then subjected to global average pooling and max pooling in the spatial dimension and a spatial attention weight map is calculated through a convolutional layer. Finally, the spatial attention weight map is multiplied pixel-by-pixel with the concatenated feature map to obtain a spatially weighted feature map. The spatially weighted feature map is mapped using a fully connected layer and a normalization layer to obtain the defect type prediction probability distribution. The spatially weighted feature map is then mapped using a fully connected layer and a regression activation layer to obtain the defect location prediction coordinates. The category with the highest probability value is selected from the defect type prediction probability distribution as the predicted defect type. The predicted defect type and the defect location prediction coordinates are then combined to obtain the preliminary detection result.

[0012] In one alternative implementation, Calculate the defect propagation probability between each pair of spatially adjacent pin instances in the pin instance set, and iteratively update the defect type label of each pin in the preliminary detection result based on the defect propagation probability to obtain the defect detection result, including: For each pin instance in the pin instance set, the node correlation degree between the current pin instance and the other pin instances in the pin instance set is calculated based on the aggregated optimized node features corresponding to the current pin instance. Pin instance pairs with node correlation degrees greater than a preset correlation degree threshold are regarded as spatially adjacent pin instance pairs. The center coordinates of the spatially adjacent pin instances are obtained and the spatial Euclidean distance is calculated. The spatial attenuation factor is calculated based on the spatial Euclidean distance and the defect propagation probability between spatially adjacent pin instances is obtained by combining the node correlation degree. Obtain all spatially adjacent pin instances corresponding to the pin instance and the defect propagation probability corresponding to the spatially adjacent pin instances, and calculate the external propagation confidence vector. Use the defect type prediction probability distribution corresponding to the current pin instance as the internal detection confidence vector and the external propagation confidence vector to perform weighted fusion according to a preset fusion ratio to obtain the fused confidence distribution. Update the defect type label in the preliminary detection result based on the fused confidence distribution. The defect type labels of all pin instances in the pin instance set are repeatedly updated until the defect type labels of all pin instances remain unchanged in continuous iterations or reach a preset number of iterations. The defect type labels obtained in the last iteration are combined with the defect location prediction coordinates to obtain the defect detection result.

[0013] A second aspect of this invention provides a machine vision-based intelligent detection system for connector pin defects, comprising: The pin region construction unit is used to acquire multi-view image sequences of the connector under multi-source illumination conditions, solve for three-dimensional topography data based on the pixel grayscale values ​​of the same spatial position under different illumination angles, extract edge pixels with significant surface curvature changes from the three-dimensional topography data and group them according to commonalities to obtain candidate pin regions, perform spatial position verification on the candidate pin regions and retain pin regions that meet the preset design specifications, assign node identifiers to each pin region and establish association edges between spatially adjacent and posture-coordinated pin regions; The feature optimization detection unit is used to take the surface normal vector and boundary contour corresponding to the pin region as node features, propagate the node features to neighboring nodes along the associated edge and aggregate the neighbor propagated features to obtain optimized node features, determine the pin instance set based on the optimized node features, extract the surface morphology features of each pin instance in the pin instance set from the three-dimensional morphology data and concatenate them with the optimized node features to obtain a high-dimensional feature representation, and decode the high-dimensional feature representation after performing multi-scale convolution operation to obtain a preliminary detection result. The defect label propagation unit is used to calculate the defect propagation probability between each pair of spatially adjacent pin instances in the pin instance set, and to iteratively update the defect type label of each pin in the preliminary detection result based on the defect propagation probability to obtain the defect detection result.

[0014] A third aspect of the present invention provides an electronic device, comprising: A processor and a memory for storing processor-executable instructions, wherein the processor is configured to invoke instructions stored in the memory to perform the aforementioned method.

[0015] A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0016] In this invention, multi-view image sequences acquired under multi-source illumination conditions are used to solve three-dimensional topographic data using pixel grayscale value changes, thereby improving detection accuracy. Candidate pin regions are extracted based on surface curvature change features and verified in conjunction with preset design specifications, effectively eliminating interference factors and improving the reliability of pin identification. By assigning node identifiers to pin regions and establishing associated edges, the spatial topological relationship between pins is expressed, allowing features to propagate along associated edges between neighboring nodes, enhancing feature representation capabilities. Surface normals, boundary contours, and three-dimensional topographic features are fused into a high-dimensional feature representation, achieving comprehensive capture of pin geometric features and improving the sensitivity of defect detection. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating the intelligent detection method for connector pin defects based on machine vision, according to an embodiment of the present invention. Figure 2 This is a flowchart illustrating the defect propagation and iterative optimization process of the intelligent detection method for connector pin defects based on machine vision, as described in an embodiment of the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0020] Figure 1 This is a flowchart illustrating the intelligent detection method for connector pin defects based on machine vision, as described in an embodiment of the present invention. Figure 1 As shown, the method includes: A multi-view image sequence of the connector to be tested under multi-source illumination conditions is acquired. The three-dimensional topography data is obtained by solving the pixel grayscale values ​​of the same spatial position under different illumination angles. Edge pixels with significant surface curvature changes are extracted from the three-dimensional topography data and grouped according to common characteristics to obtain candidate pin regions. After spatial position verification of the candidate pin regions, pin regions that meet the preset design specifications are retained. Node identifiers are assigned to each pin region and associated edges are established between spatially adjacent and posture-coordinated pin regions. The surface normal vector and boundary contour corresponding to the pin region are used as node features. The node features are propagated to neighboring nodes along the associated edge and the neighbor propagated features are aggregated to obtain optimized node features. Based on the optimized node features, a set of pin instances is determined. The surface morphology features of each pin instance in the set of pin instances are extracted from the 3D morphology data and concatenated with the optimized node features to obtain a high-dimensional feature representation. After performing multi-scale convolution operation on the high-dimensional feature representation, the initial detection result is obtained by decoding. Calculate the defect propagation probability between each pair of spatially adjacent pin instances in the pin instance set, and iteratively update the defect type label of each pin in the preliminary detection result based on the defect propagation probability to obtain the defect detection result.

[0021] In one alternative implementation, A multi-view image sequence of the connector to be tested under multi-source illumination conditions is acquired. Three-dimensional topography data is obtained by calculating the pixel grayscale values ​​of the same spatial location under different illumination angles. Edge pixels with significant surface curvature changes are extracted from the three-dimensional topography data and grouped according to common characteristics to obtain candidate pin regions, including: Multiple light sources are lit sequentially according to a preset time sequence. When each light source is lit, the camera captures an image of the connector to be detected. The number of images of the connector to be detected is the same as the number of light sources, and they are combined to obtain a multi-view image sequence. For each pixel position in the multi-view image sequence, extract multiple pixel gray values ​​corresponding to the pixel position when different light sources are lit. Substitute the multiple pixel gray values ​​and the incident angle of the corresponding light source into the preset photometric stereo constraint equation for solving, and obtain the surface normal vector and reflectivity of the spatial point corresponding to each pixel position. Traverse all pixel positions to obtain three-dimensional topography data. The surface curvature value is obtained by calculating the gradient of the surface normal vector of each spatial point in the three-dimensional topography data in the horizontal and vertical directions. Spatial points with surface curvature values ​​greater than a curvature threshold are marked as candidate edge pixels. The spatial distance between any two candidate edge pixels is calculated. Candidate edge pixels with spatial distances less than a preset connectivity distance are grouped into the same connectivity group. For each connectivity group, the spatial coordinates of all candidate edge pixels in the connectivity group are extracted and convex hull calculation is performed to obtain the minimum closed contour surrounding the connectivity group. The minimum closed contour is used as the boundary contour, and candidate pin regions are determined based on the boundary contour.

[0022] Multiple light sources are sequentially illuminated according to a preset time sequence. For each light source illuminated, a camera captures an image of the connector under inspection. In practical applications, eight LED light sources arranged in a ring can be used, with incident angles of 15°, 30°, 45°, 60°, 75°, 90°, 105°, and 120° relative to the connector surface. The camera is fixed directly above the connector, with its imaging plane parallel to the connector surface, acquiring eight images of the connector under different lighting conditions, which are then combined to form a multi-view image sequence.

[0023] For each pixel location in a multi-view image sequence, multiple pixel grayscale values ​​are extracted under different light source illumination conditions. Taking pixel locations 100 and 200 as examples, the grayscale values ​​at this location under eight different light source conditions are 85, 120, 160, 210, 180, 150, 125, and 95, respectively. These eight grayscale values ​​and the incident angle of the corresponding light source are substituted into a preset photometric stereo constraint equation for solution. The constraint equation, based on the Lambertian reflection model, describes the relationship between pixel grayscale values, light source incident angle, surface normal vector, and reflectivity. By solving the aforementioned equation, the surface normal vector and reflectivity of the corresponding spatial point at the pixel location can be obtained. The surface normal vector is represented as a three-dimensional unit vector, such as 0.2, 0.3, and 0.9, and the reflectivity is 0.75. By traversing all pixel locations in the image, complete three-dimensional topographic data can be obtained.

[0024] For each spatial point in the 3D topographic data, calculate the gradient of the corresponding surface normal vector in the horizontal and vertical directions. Taking a spatial point as the center, take a 3×3 neighborhood around it and calculate the difference in normal vectors in the horizontal and vertical directions. The horizontal gradient can be obtained by calculating the angle between the normal vectors of this point and its left and right adjacent points, while the vertical gradient is obtained by calculating the angle between the normal vectors of the point and its upper and lower adjacent points. The square root of the sum of the squares of the horizontal and vertical gradients is taken as the surface curvature value of this point. For example, if the horizontal gradient of a point is 0.15 and the vertical gradient is 0.22, then its surface curvature value is 0.267.

[0025] Spatial points with surface curvature values ​​greater than a preset curvature threshold are marked as candidate edge pixels. In this embodiment, the curvature threshold can be set to 0.2, meaning points with surface curvature values ​​greater than 0.2 are marked as candidate edge pixels. The spatial distance between any two candidate edge pixels is calculated, and candidate edge pixels with a spatial distance less than a preset connectivity distance are grouped into the same connectivity group. The preset connectivity distance can be set to 5 pixels, grouping pixels belonging to the same pin edge together.

[0026] For each connected component, extract the spatial coordinates of all candidate edge pixels within the current connected component and perform convex hull calculation to obtain the minimum closed contour enclosing the connected component. This minimum closed contour is then used as the boundary contour. The convex hull calculation can be implemented using the Graham scan algorithm. Find the point with the smallest y-coordinate in the connected component as the starting point, sort the remaining points according to their polar angle relative to the starting point, connect each point sequentially according to the polar angle, and check if a concave angle is formed. If a concave angle is formed, delete the previous point until the convex hull is obtained, thus obtaining the closed contour of the stitch edge.

[0027] Candidate stitch regions are determined based on the boundary contour, with the area inside the boundary contour considered as a candidate stitch region. To further confirm the stitch region, features such as the area, perimeter, and roundness of the contour are calculated. For example, if the area of ​​the contour is between 500 and 1000 square pixels, the perimeter is between 80 and 150 pixels, and the roundness is greater than 0.7, then the contour can be determined to correspond to a normal stitch region. The roundness calculation formula is 4π multiplied by the area and then divided by the square of the perimeter; the closer the roundness value is to 1, the closer the contour is to a circle.

[0028] In this embodiment, by sequentially illuminating multiple light sources with known incident angles and constructing a multi-view image sequence, and combining photometric stereo constraints to jointly solve pixel-level reflection information, the normal distribution and reflection characteristics of the connector surface can be accurately recovered without additional structured light or multi-camera systems. This effectively eliminates the influence of uneven illumination, surface reflection, and texture interference on structural recognition, and improves the ability to perceive subtle geometric changes. By introducing curvature calculation based on surface normal changes in the three-dimensional topography space, edge determination is upgraded from gray-scale abrupt change to geometric abrupt change determination, so that the recognition of pin edges no longer depends on color or contrast differences. By performing connectivity analysis on high-curvature spatial points and constructing a minimum closed contour in three-dimensional space, the problems of edge breakage, adhesion, or missegmentation in traditional projection planes are avoided, making the obtained boundary contour more complete, regular, and highly consistent with the actual physical structure, significantly improving the accuracy and consistency of candidate pin region positioning.

[0029] In one alternative implementation, After spatial position verification of the candidate pin regions, pin regions that conform to the preset design specifications are retained. Each pin region is assigned a node identifier, and association edges are established between spatially adjacent and attitude-coordinated pin regions, including: The centroids corresponding to the three-dimensional coordinates of all spatial points within the candidate pin region are calculated to obtain the center coordinates of the candidate pin region. Principal component decomposition is performed on the boundary contour of the candidate pin region to obtain the principal direction vector, which is used as the attitude direction of the candidate pin region. The center coordinates are projected onto the horizontal reference plane of the spatial coordinate system to obtain the projected coordinates. The candidate pin regions are then arranged in a grid according to the row and column positions of the projected coordinates to obtain a two-dimensional array distribution. For each grid position in the two-dimensional array distribution, when the number of candidate pin regions in the grid position is greater than one, the candidate pin region with the closest boundary contour area to the standard pin area and the highest boundary contour integrity is retained, and other candidate pin regions in the grid position are removed to obtain the redundant pin region. The absolute value of the difference between the center coordinate distance of the redundant pin region corresponding to the adjacent grid positions in the two-dimensional array distribution and the preset pin design spacing is calculated as the center distance deviation. The redundant pin regions with a center distance deviation less than the position verification threshold are retained as pin regions that meet the design specifications. For each pin region, a node identifier is generated according to the row index and column index in the two-dimensional array distribution. The inner product of the attitude direction vector of the pin region and the corresponding pin region of the adjacent grid position is calculated as the attitude cooperation metric. An association edge is established between adjacent pin regions whose attitude cooperation metric is greater than the cooperation threshold.

[0030] For each identified candidate pin region, the centroid corresponding to the three-dimensional coordinates of all spatial points within the region is calculated to obtain the center coordinates of the candidate pin region. The three-dimensional coordinate values ​​of all points within the candidate pin region are then summed and divided by the total number of points. For example, if a candidate pin region contains 500 spatial points, and the summed three-dimensional coordinate values ​​are 25000, 35000, and 15000, then the center coordinates of this region are 50, 70, and 30. These center coordinates represent the position of the pin in three-dimensional space, providing a basis for subsequent pin arrangement and defect analysis.

[0031] Principal component decomposition (PCD) is performed on the boundary contours of candidate stitch regions to obtain principal direction vectors, which are then used as the orientation directions of the candidate stitch regions. PCD is a statistical analysis method that calculates the covariance matrix of boundary contour points and solves for its eigenvalues ​​and eigenvectors to obtain the main directions of data change. In practice, the coordinates of the boundary contour points are centered by subtracting the mean of the contour point coordinates; the covariance matrix of the centered coordinates is calculated; and eigenvalue decomposition is performed on the covariance matrix to obtain eigenvalues ​​and corresponding eigenvectors. The eigenvector corresponding to the largest eigenvalue is taken as the principal direction vector. For example, if the principal direction vectors obtained after PCD of a certain stitch boundary contour are 0.866, 0.5, and 0, it indicates that the main orientation of the stitch forms a 30-degree angle with the horizontal axis.

[0032] The projected coordinates are obtained by projecting the center coordinates onto the horizontal reference plane of the spatial coordinate system. The projection operation simply sets the height component of the three-dimensional coordinates to zero, retaining the two coordinate components in the horizontal plane. The candidate pin regions are then meshed according to the row and column positions of the projected coordinates, resulting in a two-dimensional array distribution. In practice, the mesh size is set to 1.2 times the standard pin spacing; for example, if the standard pin spacing is 2.54 mm, the mesh size is set to 3 mm. Dividing the projected coordinates by the mesh size and rounding down yields the row and column indices of the pins. For example, if the projected coordinates are 15.2 mm and 27.8 mm, and the mesh size is 3 mm, the corresponding row and column indices are 5 and 9.

[0033] For each grid location in the 2D array distribution, redundancy elimination is required when the number of candidate pin regions within that grid location is greater than one. The candidate pin region with the closest boundary contour area to the standard pin area and the highest boundary contour integrity is retained, while other candidate pin regions within the grid location are removed, resulting in a de-redundant pin region. The standard pin area can be preset to 700 square pixels, and the boundary contour integrity is defined as the degree of contour closure, which can be evaluated by calculating the connectivity between contour points. For example, if a grid location contains two candidate pin regions with areas of 680 and 720 square pixels, and integritys of 0.95 and 0.88 respectively, the candidate region with an area of ​​680 square pixels is retained because its area is closer to the standard value and its integrity is higher.

[0034] The absolute value of the difference between the center coordinate distance of adjacent grid positions corresponding to the de-redundant pin regions in the two-dimensional array distribution and the preset pin design spacing is calculated as the center distance deviation. The pin design spacing can be set to 2.54 mm. De-redundant pin regions with center distance deviations less than the position verification threshold are retained as pin regions conforming to the design specifications. The position verification threshold can be set to 0.3 mm. For example, if the center distance between two adjacent pin regions is 2.62 mm, the absolute value of the difference between this and the design spacing of 2.54 mm is 0.08 mm, which is less than the threshold of 0.3 mm; therefore, these two pin regions are retained.

[0035] Each pin region is encoded using its row and column indices in a two-dimensional array to generate a node identifier. The encoding method can use a combination of row and column indices. For example, a pin with a row index of 3 and a column index of 5 has a node identifier of 305, which uniquely identifies the position of each pin in the connector, facilitating subsequent defect analysis and location.

[0036] The inner product of the attitude direction vectors of the corresponding pin regions in adjacent grid positions is calculated as the attitude coordination metric. The inner product is calculated as the sum of the products of the corresponding components of the two vectors. An association edge is established between adjacent pin regions where the attitude coordination metric is greater than the coordination threshold. The coordination threshold can be set to 0.9, meaning that the cosine similarity of the attitude directions must be greater than 0.9 to establish an association. For example, if the attitude direction vectors of two adjacent pins are 0.866, 0.5, 0 and 0.87, 0.49, 0, respectively, the inner product is 0.866 multiplied by 0.87 plus 0.5 multiplied by 0.49, which equals 0.9983, greater than the threshold of 0.9. Therefore, an association edge is established between these two pins.

[0037] In this embodiment, by calculating the centroid of the three-dimensional coordinates of the candidate pin regions and determining their principal orientation through principal component decomposition, the representation of pin position and attitude is improved from simple bounding box or two-dimensional angle estimation to a stable geometric description based on real spatial distribution, which significantly improves the reliability of attitude estimation. By projecting the center of the pin region onto a unified horizontal reference plane and arranging it in a regular grid, the originally disordered candidate regions are transformed into a two-dimensional array distribution that conforms to the structural characteristics of the connector. This effectively suppresses the problems of repeated identification and position drift caused by noise, reflection, or local false detection. By comparing the actual center distance of adjacent pin regions with the design spacing and setting position verification constraints, the identification results are not only reasonable locally, but also consistent with the design parameters on the overall array scale. This effectively filters out abnormal pin regions with large positional offsets or disordered arrangement, improving the screening accuracy of pin regions that conform to design specifications.

[0038] In one alternative implementation, Using the surface normal vector and boundary contour corresponding to the pin region as node features, the node features are propagated to neighboring nodes along the associated edges, and the neighbor propagation features are aggregated to obtain optimized node features, including: The surface normal vectors of all spatial points within the boundary contour corresponding to the stitch area are obtained and statistical analysis is performed to obtain the mean vector and variance value of the surface normal vectors. The mean vector and variance value are then concatenated to form the surface normal vector feature. The boundary contour of the stitch region is sampled to obtain a sequence of key points. The spatial distance and angle change between adjacent key points in the sequence of key points are calculated and spliced ​​to obtain the boundary contour features. The surface normal vector features and the boundary contour features are spliced ​​to obtain the node features corresponding to the stitch region. For each pin region, the node identifiers of all pin regions connected to the current pin region through the associated edge are found according to the node identifier of the pin region to obtain the neighboring node identifiers. The node features of the pin regions corresponding to all neighboring node identifiers are extracted to obtain the neighboring node feature set. The weight value is determined based on the associated edge, and the node features in the neighboring node feature set are weighted and summed according to the weight value to obtain the neighborhood propagation feature. Calculate the feature similarity between the node features and the neighborhood propagation features, determine the adaptive weight coefficient based on the feature similarity, and perform feature fusion on the node features and neighborhood propagation features corresponding to the pin region according to the adaptive weight coefficient to obtain optimized node features.

[0039] For a defined pin area, the surface normal vectors of all spatial points within the boundary contour are obtained and statistically analyzed. Surface normal vectors reflect the geometric properties of the pin surface and are crucial for detecting defects such as pin bending and deformation. All spatial points within the pin area are traversed, and the surface normal vector of each point is extracted. For example, a pin area may contain 500 spatial points, each with a corresponding 3D normal vector. The mean vector of these 500 normal vectors is calculated to obtain the overall surface orientation of the pin area, such as a mean vector of (0.05, 0.08, 0.99). Simultaneously, the variance of the normal vectors is calculated, representing the dispersion of the normal vector distribution and reflecting surface smoothness. A larger variance indicates a less smooth pin surface, potentially indicating the presence of depressions or protrusions. For example, the calculated variance is 0.025. The mean vector and variance are concatenated to form the surface normal vector feature, i.e., (0.05, 0.08, 0.99, 0.025). This surface normal vector feature reflects the overall orientation and smoothness of the pin surface.

[0040] Contour sampling is performed on the boundary contour of the stitch area to obtain a sequence of key points. The sampling method can employ equal arc length sampling, where a point is selected as a key point at regular arc intervals along the contour. In practical applications, a sampling interval of 5 pixels can be set, starting from the contour's starting point and selecting a point every 5 pixels until returning to the vicinity of the starting point. For example, sampling the boundary contour of a stitch yields 16 key points, forming a key point sequence. The spatial distance and angular variation between adjacent key points in the contour key point sequence are calculated. Spatial distance is calculated as the Euclidean norm of the difference between the three-dimensional coordinates of two points, and angular variation is calculated as the angle between two vectors formed by three adjacent points. For example, the spatial distance from the first key point to the second key point is 5.1 pixels, and the angular variation formed by the first, second, and third points is 170 degrees. Calculations are performed on all adjacent key points, yielding 15 spatial distance values ​​and 14 angular variation values. These values ​​are concatenated to form the boundary contour feature, such as (5.1, 4.9, 5.2, ..., 170, 168, 175, ...). The boundary contour feature describes the shape and smoothness of the stitch boundary.

[0041] The surface normal vector features are concatenated with the boundary contour features to obtain the node features corresponding to the pin region. The node features comprehensively reflect the surface properties and boundary shape of the pin and are an important basis for pin defect detection. For example, the node features of a certain pin are (0.05, 0.08, 0.99, 0.025, 5.1, 4.9, 5.2, ..., 170, 168, 175, ...).

[0042] For each pin region, the node identifiers of all pin regions connected to the current pin region through associated edges are found based on its node identifier, thus obtaining the neighboring node identifiers. For example, for a pin with node identifier 305, the neighboring node identifiers might be 304, 306, 205, and 405, representing the four pins adjacent to this pin. The node features of the pin regions corresponding to all neighboring node identifiers are extracted, resulting in a neighboring node feature set. Weight values ​​are determined based on the associated edges, and the node features in the neighboring node feature set are weighted and summed according to these weight values ​​to obtain the neighborhood propagation features. The weight values ​​can be determined based on the pose coordination metric corresponding to the associated edge; the higher the pose coordination metric, the larger the weight value. For example, the pose coordination metric between neighboring node 304 and the current node 305 is 0.98, corresponding to a weight value of 0.28; the weight value of neighboring node 306 is 0.25; the weight value of node 205 is 0.24; and the weight value of node 405 is 0.23. The neighboring node features are then weighted and summed to obtain the neighborhood propagation features.

[0043] Calculate the feature similarity between node features and neighborhood propagation features. Feature similarity can be obtained by calculating the cosine similarity of two feature vectors, i.e., the inner product of the two vectors divided by the product of their norms. The higher the feature similarity, the closer the characteristics of the current node are to those of its neighboring nodes. For example, the calculated similarity between the node features and neighborhood propagation features of a certain node is 0.92. Determine adaptive weight coefficients based on feature similarity. Adaptive weight coefficients are used to control the proportion of node features and neighborhood propagation features in the fusion process. When feature similarity is high, the weight of neighborhood propagation features is increased; when feature similarity is low, the weight of node features is increased. This can be achieved by setting a smoothing function; for example, when the feature similarity is 0.92, the calculated adaptive weight coefficient is 0.65.

[0044] The node features and neighborhood propagation features corresponding to the pin region are fused using an adaptive weighting coefficient to obtain optimized node features. The feature fusion process involves multiplying the node feature by the adaptive weighting coefficient, adding the neighborhood propagation feature multiplied by one, and subtracting the adaptive weighting coefficient. For example, when the adaptive weighting coefficient is 0.65, the optimized node feature is the node feature multiplied by 0.65 plus the neighborhood propagation feature multiplied by 0.35. The optimized node features comprehensively consider both the pin's own features and the neighborhood propagation features, thus more comprehensively reflecting the pin's state, effectively reducing the impact of noise and outliers, and improving the stability and representativeness of the features.

[0045] In this embodiment, by statistically analyzing the surface normal vectors of all spatial points within the pin region, the mean and variance are introduced to characterize the overall orientation and dispersion of the surface. This elevates the pin surface morphology from local, discrete point-level information to statistical features with overall physical meaning. By sampling key points of the boundary contour and constructing contour features using the spatial distance and angle changes between adjacent key points, a fine description of the continuity, regularity, and geometric consistency of the pin boundary shape is achieved. This can more effectively distinguish abnormal pin regions with similar shapes but edge defects, deformations, or burrs. By calculating the similarity between its own node features and the neighboring propagation features and adaptively adjusting the fusion weight accordingly, dynamic control of the degree of neighboring information introduction is achieved, avoiding excessive smoothing or abnormal feature diffusion problems. This ensures that the optimized node features maintain individual differences while possessing array consistency.

[0046] In one alternative implementation, Based on the optimized node features, a set of pin instances is determined. The surface morphology features of each pin instance in the set are extracted from the 3D morphology data and concatenated with the optimized node features to obtain a high-dimensional feature representation. After performing multi-scale convolution operations on the high-dimensional feature representation, decoding yields preliminary detection results, including: Based on the feature similarity between the optimized node features, the feature distance between each pair of optimized node features is calculated using a hierarchical clustering algorithm. Optimized node features whose feature distance is less than a preset initial clustering threshold are merged into an initial cluster. The different initial clusters with the smallest inter-cluster distance are repeatedly merged until the inter-cluster distance is greater than the preset merging distance threshold, thus obtaining optimized clusters and using the pin regions corresponding to the optimized clusters as pin instances. The three-dimensional topography data within the spatial range enclosed by the boundary contour in the pin instance is obtained as a topography data block. The direction distribution statistics of the surface normal vectors in the topography data block are performed to obtain a normal vector distribution histogram. The height distribution statistics of the spatial points in the topography data block are performed to obtain a height distribution histogram and are spliced ​​with the normal vector distribution histogram to obtain the surface topography features. The optimized node features corresponding to the pin regions contained in the pin instance are averaged to obtain aggregated optimized node features, which are then concatenated with the surface morphology features to obtain a high-dimensional feature representation. Multi-scale dilated convolution is performed on the high-dimensional feature representation to obtain adaptive feature maps of multiple receptive field scales. Channel splicing and decoding are performed on the adaptive feature maps to obtain the defect type prediction probability distribution and defect location prediction coordinates, which are then combined to obtain preliminary detection results.

[0047] Based on the feature similarity between optimized node features, a hierarchical clustering algorithm is used to calculate the pairwise feature distance between optimized node features. The feature distance can be calculated using Euclidean distance, which is the square root of the sum of the squares of the differences in each dimension of two feature vectors. Optimized node features with feature distances less than a preset initial clustering threshold are merged into an initial cluster. The initial clustering threshold can be set to 0.15, meaning that two pin regions with feature distances less than 0.15 are grouped into the same initial cluster. For example, the optimized node feature distance between pin regions 305 and 306 is 0.12, which is less than the initial clustering threshold of 0.15, so these two pin regions are merged into the same initial cluster. The merging of different initial clusters with the smallest inter-cluster distance is repeated until the inter-cluster distance is greater than the preset merging distance threshold, resulting in optimized clusters. The inter-cluster distance is defined as the minimum feature distance between any two nodes in two clusters. The merging distance threshold can be set to 0.25, meaning that merging stops when the inter-cluster distance is greater than 0.25. The pin region corresponding to the optimized cluster is used as the pin instance.

[0048] The three-dimensional topographic data within the spatial range enclosed by the boundary contour of the pin instance is acquired as a topographic data block. This topographic data block contains detailed three-dimensional information about the pin surface, which is crucial for detecting defects such as dents and scratches on the pin surface. A histogram of normal vector distribution is obtained by statistically analyzing the direction distribution of the surface normal vectors in the topographic data block. Specifically, the direction space of the surface normal vectors is discretized into 36 direction intervals, and the proportion of normal vectors in each direction interval is statistically analyzed to form a 36-dimensional normal vector distribution histogram. For example, the histogram of the normal vector distribution for a certain pin instance is (0.02, 0.03, 0.04, 0.05, 0.15, 0.25, 0.20, 0.10, 0.05, 0.02, 0.01, 0.01, 0.01, 0.01, 0.01, 0.01, 0.01, 0.01, 0.01, 0.01, 0.01, 0.01, 0.01, 0.01, 0.01, 0.01, 0.01, 0.01, 0.01, 0.01, 0.01, 0.01, 0.01, 0.01, 0.01, 0.01, 0.01). A height distribution histogram is obtained by statistically analyzing the height distribution of spatial points in the topographic data block. The height range of the pin is discretized into 20 height intervals, and the proportion of points within each height interval is counted to form a 20-dimensional height distribution histogram. For example, the height distribution histogram of a certain pin instance is (0.01, 0.02, 0.03, 0.05, 0.08, 0.12, 0.15, 0.18, 0.15, 0.10, 0.05, 0.02, 0.01, 0.01, 0.01, 0.01, 0.01, 0.01, 0.01). The surface morphology features are obtained by concatenating the normal vector distribution histogram and the height distribution histogram.

[0049] The aggregated optimized node features are obtained by averaging the optimized node features corresponding to the pin regions contained in the pin instance. The feature averaging process involves summing the optimized node features of all pin regions in the pin instance according to their dimensions and then dividing by the number of pin regions. For example, if a pin instance contains two pin regions, 305 and 306, with optimized node features of (0.06, 0.09, 0.98, 0.022, 5.2, 4.8, 5.1, ...) and (0.05, 0.08, 0.99, 0.024, 5.0, 4.9, 5.3, ...), the aggregated optimized node features are obtained by feature averaging: (0.055, 0.085, 0.985, 0.023, 5.1, 4.85, 5.2, ...). The aggregated optimized node features are then concatenated with surface morphology features to obtain a high-dimensional feature representation. This high-dimensional feature representation integrates information from multiple aspects, including pin boundary features, surface normal vector features, and surface morphology features.

[0050] Multi-scale dilated convolution is performed on the high-dimensional feature representation to obtain adaptive feature maps with multiple receptive field scales. Dilated convolution is a special convolution operation that expands the receptive field without increasing the number of parameters by inserting holes in the convolution kernel. In practical implementation, three convolutional layers with different dilation rates of 1, 2, and 4 can be used, with a kernel size of 3×3 and 64 output channels for each layer. Multi-scale dilated convolution can capture the changes in pin features at different scales, effectively identifying defects of different sizes and shapes. For example, a small dilation rate is suitable for detecting fine scratches on the pin surface, while a large dilation rate is suitable for detecting overall deformation and bending of the pin. Channel concatenation and decoding are performed on the adaptive feature maps to obtain the defect type prediction probability distribution and defect location prediction coordinates, which are then combined to obtain preliminary detection results. Channel concatenation concatenates the output features of multiple dilated convolutions along the channel dimension to form a feature map containing multi-scale information. The decoding process uses two branches: one branch outputs the defect type prediction probability distribution through a fully connected layer, and the other branch outputs the defect location prediction coordinates through a regression layer.

[0051] The defect type prediction probability distribution represents the probability that a pin belongs to various defect types, typically including categories such as normal, bent, deformed, missing, and oxidized. For example, a defect type prediction probability distribution of (0.15, 0.75, 0.05, 0.03, 0.02) for a certain pin indicates that the pin has the highest probability of being a bent defect, at 0.75. The defect location prediction coordinates indicate the specific position of the defect on the pin, usually expressed as a coordinate offset relative to the pin's center. For example, defect location prediction coordinates of (0.3, 0.5, 0.2) indicate that the defect is located 0.3 units to the right, 0.5 units down, and 0.2 units up from the pin's center. Combining the defect type prediction probability distribution with the defect location prediction coordinates yields a preliminary detection result. For example, the detection result could be represented as pin number 305, defect type being bent, probability 0.75, and defect location (0.3, 0.5, 0.2).

[0052] In this embodiment, a hierarchical clustering mechanism is introduced based on the similarity between optimized node features. Stitch regions with high morphological consistency and structural correlation are adaptively merged into stitch instances. This eliminates the need for fixed-number assumptions or manual rules in the construction of stitch instances, significantly reducing the probability of instance fragmentation and erroneous merging. By extracting complete 3D morphological data blocks at the stitch instance scale and statistically modeling the surface normal vector distribution and height distribution, the morphological features are elevated from local geometric descriptions to overall spatial distribution descriptions. This effectively characterizes complex 3D defect features such as stitch bending, collapse, and warping, significantly enhancing the sensitivity to hidden and gradual defects. By aggregating the optimized node features of multiple stitch regions within a stitch instance and fusing them with 3D surface morphological features, a high-dimensional feature representation that simultaneously contains local structural consistency information and overall morphological features is formed, improving the separability and reliability of different defect types.

[0053] In one alternative implementation, Multi-scale dilated convolution is performed on the high-dimensional feature representation to obtain adaptive feature maps with multiple receptive field scales. Channel concatenation and decoding are performed on the adaptive feature maps to obtain the defect type prediction probability distribution and defect location prediction coordinates. These are then combined to obtain preliminary detection results, including: The high-dimensional feature representation is input into multiple pre-defined dilated convolutional layers with increasing dilation rates to perform dilated convolution operations to obtain initial feature maps at multiple receptive field scales. For each initial feature map, the offset of the pixel position is learned through convolution operations. The sampling position of the convolution kernel is adaptively adjusted according to the offset and the convolution operation is performed again to obtain adaptive feature maps at multiple receptive field scales. The adaptive feature maps are concatenated along the channel dimension to obtain a concatenated feature map. The concatenated feature map is then subjected to global average pooling and max pooling in the spatial dimension and a spatial attention weight map is calculated through a convolutional layer. Finally, the spatial attention weight map is multiplied pixel-by-pixel with the concatenated feature map to obtain a spatially weighted feature map. The spatially weighted feature map is mapped using a fully connected layer and a normalization layer to obtain the defect type prediction probability distribution. The spatially weighted feature map is then mapped using a fully connected layer and a regression activation layer to obtain the defect location prediction coordinates. The category with the highest probability value is selected from the defect type prediction probability distribution as the predicted defect type. The predicted defect type and the defect location prediction coordinates are then combined to obtain the preliminary detection result.

[0054] The high-dimensional feature representation is input into multiple pre-defined dilated convolutional layers with increasing dilation rates to perform dilated convolution operations, resulting in initial feature maps at multiple receptive field scales. Dilated convolution is a convolutional method that expands the receptive field by inserting holes in the convolutional kernel, enabling the acquisition of a wider range of contextual information without increasing the number of parameters or computational complexity. In practical applications, three dilated convolutional layers can be set with dilation rates of 1, 3, and 5, each with a kernel size of 3×3 and 64 output channels. For example, for a high-dimensional feature representation of size 128×128×256, the first dilated convolutional layer (dilation rate of 1) outputs an initial feature map of size 128×128×64, the second dilated convolutional layer (dilation rate of 3) outputs a feature map of the same size, and the third dilated convolutional layer (dilation rate of 5) also outputs a feature map of the same size. These three initial feature maps capture pin features at different receptive field scales.

[0055] For each initial feature map, the offset of the pixel position is learned through convolution operation. Based on the offset, the sampling position of the convolution kernel is adaptively adjusted and the convolution operation is re-performed to obtain adaptive feature maps at multiple receptive field scales. By learning the sampling offset of the convolution kernel, the convolution operation can adapt to the irregular shape of the stitch defect. In specific implementation, a 2×2 convolutional layer is applied to each initial feature map, with 2×3×3=18 output channels, corresponding to the horizontal and vertical offsets of 9 positions in the 3×3 convolution kernel. For example, for the first initial feature map, the learned offsets may be (0.3, -0.2), (-0.1, 0.4), (0.2, 0.3), (0.5, -0.3), (0.0, 0.0), (-0.3, 0.2), (0.4, 0.1), (0.2, -0.4), (-0.5, -0.2), representing the horizontal and vertical offsets of the 9 positions of the convolution kernel. Based on these offsets, the sampling positions of the convolution kernels are adjusted and a 3×3 convolution operation is performed to obtain an adaptive feature map.

[0056] The adaptive feature maps are concatenated along the channel dimension to obtain a concatenated feature map. The channel concatenation operation stacks three adaptive feature maps with different receptive field scales along the channel dimension, forming a concatenated feature map of size 128×128×192. Global average pooling and max pooling are performed on the concatenated feature map in the spatial dimension, and a spatial attention weight map is calculated through a convolutional layer. Global average pooling calculates the average value of each channel, resulting in a 1×1×192 feature vector; global max pooling calculates the maximum value of each channel, also resulting in a 1×1×192 feature vector. These two feature vectors are concatenated and passed through a 1×1 convolutional layer, outputting a 128×128 channel number, which is the spatial attention weight map.

[0057] The spatial attention weight map is multiplied pixel-by-pixel with the stitched feature map to obtain a spatially weighted feature map. This pixel-by-pixel multiplication assigns a corresponding spatial attention weight to each feature channel in the stitched feature map, highlighting the feature representation of the defective stitch area and suppressing the influence of irrelevant background areas. For example, the weight value of the bent stitch area in the spatial attention weight map is higher, approximately 0.85, while the weight value of the normal area is lower, approximately 0.2.

[0058] The defect type prediction probability distribution is obtained by performing feature mapping on the spatially weighted feature map through a fully connected layer and a normalization layer. The input of the fully connected layer is the feature vector of the spatially weighted feature map after global average pooling, with a dimension of 192. The output dimension of the fully connected layer is the number of defect types, for example, 5, corresponding to 5 states: normal, bent, deformed, missing, and oxidized. The normalization layer uses a soft maximum function to convert the output of the fully connected layer into a probability distribution. For example, the defect type prediction probability distribution of a certain pin is (0.08, 0.82, 0.05, 0.03, 0.02), indicating that the probability of classifying the pin as a bent defect is 0.82.

[0059] The predicted coordinates of the defect location are obtained by feature mapping the spatially weighted feature map through a fully connected layer and a regression activation layer. The input to the fully connected layer is also a feature vector obtained by global average pooling of the spatially weighted feature map, and the output dimension is 3, corresponding to the three-dimensional coordinates of the defect location. The regression activation layer uses a hyperbolic tangent function to restrict the output value to between -1 and 1, and maps it to the pin coordinate range through a linear transformation. For example, the predicted coordinates of the defect location of a certain pin are (0.35, -0.42, 0.28), which means that the defect is located at (0.35, -0.42, 0.28) in the pin coordinate system.

[0060] The category with the highest probability value is selected from the defect type prediction probability distribution as the predicted defect type. The predicted defect type is then combined with the predicted defect location coordinates to obtain the preliminary detection result. For example, given the probability distribution (0.08, 0.82, 0.05, 0.03, 0.02) and the predicted defect location coordinates (0.35, -0.42, 0.28), if the predicted defect type is bending (probability 0.82), the preliminary detection result is a bent pin defect located at coordinates (0.35, -0.42, 0.28).

[0061] In this embodiment, by setting a multi-layer dilated convolutional structure with increasing dilation rate, local details and large-scale structural information are covered simultaneously without increasing the loss of feature map resolution. This significantly enhances the ability to perceive defects of different sizes and shapes, and improves the scale adaptability of feature representation. By introducing an adaptive convolutional mechanism based on pixel offset learning on feature maps at each scale, the convolutional sampling position can be adaptively adjusted according to the actual morphological features. This breaks through the limitation of fixed regular grid sampling on irregular defect shapes, effectively improves the feature alignment ability for curved, tilted, or asymmetrical defect regions, and reduces the feature response deviation caused by deformation. By performing channel-level stitching on multi-scale adaptive feature maps and introducing spatial attention weight modeling based on global pooling, the network can automatically highlight regions that contribute more to defect judgment in the spatial dimension, suppress the interference of background regions and irrelevant structures, and significantly improve the feature saliency and signal-to-noise ratio of defect-related regions.

[0062] In one alternative implementation, Calculate the defect propagation probability between each pair of spatially adjacent pin instances in the pin instance set, and iteratively update the defect type label of each pin in the preliminary detection result based on the defect propagation probability to obtain the defect detection result, including: For each pin instance in the pin instance set, the node correlation degree between the current pin instance and the other pin instances in the pin instance set is calculated based on the aggregated optimized node features corresponding to the current pin instance. Pin instance pairs with node correlation degrees greater than a preset correlation degree threshold are regarded as spatially adjacent pin instance pairs. The center coordinates of the spatially adjacent pin instances are obtained and the spatial Euclidean distance is calculated. The spatial attenuation factor is calculated based on the spatial Euclidean distance and the defect propagation probability between spatially adjacent pin instances is obtained by combining the node correlation degree. Obtain all spatially adjacent pin instances corresponding to the pin instance and the defect propagation probability corresponding to the spatially adjacent pin instances, and calculate the external propagation confidence vector. Use the defect type prediction probability distribution corresponding to the current pin instance as the internal detection confidence vector and the external propagation confidence vector to perform weighted fusion according to a preset fusion ratio to obtain the fused confidence distribution. Update the defect type label in the preliminary detection result based on the fused confidence distribution. The defect type labels of all pin instances in the pin instance set are repeatedly updated until the defect type labels of all pin instances remain unchanged in continuous iterations or reach a preset number of iterations. The defect type labels obtained in the last iteration are combined with the defect location prediction coordinates to obtain the defect detection result.

[0063] For each pin instance in the pin instance set, the node correlation degree between the current pin instance and the remaining pin instances in the set is calculated based on the aggregated optimized node features corresponding to the current pin instance. The node correlation degree can be obtained by calculating the cosine similarity between the aggregated optimized node features of two pin instances. The cosine similarity is calculated as the inner product of two feature vectors divided by the product of their norms, and its value ranges from -1 to 1. The larger the value, the more similar the features of the two pin instances are. For example, the aggregated optimized node features of pin instance 305 are (0.055, 0.085, 0.985, 0.023, 5.1, 4.85, 5.2, ...), and the aggregated optimized node features of pin instance 306 are (0.056, 0.087, 0.984, 0.024, 5.2, 4.86, 5.3, ...). The calculated node correlation degree is 0.998, indicating that the features of these two pin instances are very similar.

[0064] Pin instance pairs with a node correlation degree greater than a preset correlation degree threshold are considered spatially adjacent pin instance pairs. The correlation degree threshold can be set to 0.95, meaning that pin instance pairs with a node correlation degree greater than 0.95 are considered spatially adjacent. The center coordinates of spatially adjacent pin instances are obtained, and the spatial Euclidean distance is calculated. The spatial Euclidean distance is calculated as the square root of the sum of the squares of the differences between the center coordinates of two pin instances. For example, the center coordinates of pin instance 305 are (25.4, 35.6, 2.8), and the center coordinates of pin instance 306 are (28.0, 35.7, 2.9), and the calculated spatial Euclidean distance is 2.61. Based on the spatial Euclidean distance, a spatial attenuation factor is calculated, and the defect propagation probability between spatially adjacent pin instances is obtained by combining the node correlation degree. The spatial attenuation factor can be calculated using an exponential attenuation function, i.e., a negative exponent multiplied by the spatial Euclidean distance, and then the exponential function value is taken. For example, when the exponent coefficient is set to 0.1, the spatial decay factor between pin instances 305 and 306 is an exponential function value (with the natural constant as the base and an exponent of -0.1 × 2.61), approximately 0.77. The defect propagation probability is calculated as the product of the node correlation degree and the spatial decay factor. For example, the defect propagation probability between pin instances 305 and 306 is 0.998 × 0.77 = 0.768.

[0065] Obtain all spatially adjacent pin instances corresponding to a pin instance, along with the defect propagation probability of each spatially adjacent pin instance, and calculate the external propagation confidence vector. The external propagation confidence vector is calculated as a weighted average of the defect type prediction probability distributions of all spatially adjacent pin instances according to their corresponding defect propagation probabilities. For example, pin instance 305 has three spatially adjacent pin instances: 306, 307, and 405, with corresponding defect propagation probabilities of 0.768, 0.723, and 0.691, respectively. The defect type prediction probability distributions of these three pin instances are (0.08, 0.82, 0.05, 0.03, 0.02), (0.07, 0.85, 0.04, 0.02, 0.02), and (0.09, 0.79, 0.06, 0.04, 0.02), respectively. When calculating the external propagation confidence vector, the three probability distributions are weighted and averaged according to their corresponding defect propagation probabilities to obtain (0.08, 0.82, 0.05, 0.03, 0.02).

[0066] The predicted probability distribution of the defect type corresponding to the current pin instance is used as the internal detection confidence vector and the external propagation confidence vector. These are then weighted and fused according to a preset fusion ratio to obtain the fused confidence distribution. The fusion ratio can be set to a weight of 0.6 for the internal detection confidence and 0.4 for the external propagation confidence. For example, the internal detection confidence vector of pin instance 305 is (0.12, 0.76, 0.06, 0.04, 0.02), and the external propagation confidence vector is (0.08, 0.82, 0.05, 0.03, 0.02). The resulting fused confidence distribution is the internal detection confidence vector multiplied by 0.6 plus the external propagation confidence vector multiplied by 0.4, i.e., (0.104, 0.784, 0.056, 0.036, 0.02). The defect type label in the preliminary detection results is updated based on the fused confidence distribution. The update method is to select the category with the highest probability in the fused confidence distribution as the new defect type label. For example, the fusion confidence distribution of pin instance 305 is (0.104, 0.784, 0.056, 0.036, 0.02). The category with the highest probability is category 2, which corresponds to the defect type "bending" with a probability value of 0.784. Therefore, the updated defect type label is "bending".

[0067] The defect type labels for all pin instances in the pin instance set are repeatedly updated until the defect type labels for all pin instances remain unchanged in consecutive iterations or a preset number of iterations is reached. The preset number of iterations can be set to 10, indicating a maximum of 10 iterations. In each iteration, the external propagation confidence vector for each pin instance is recalculated based on the defect type labels updated in the previous iteration, and fused with the internal detection confidence vector to obtain a new fused confidence distribution, which in turn updates the defect type label. Through multiple iterations, defect information between pin instances is continuously propagated and fused, ultimately resulting in a more stable and accurate defect type label. For example, after the 3rd iteration, the defect type label for pin instance 305 stabilizes as "bent" and no longer changes.

[0068] The defect type label obtained from the last iteration is combined with the predicted coordinates of the defect location to obtain the defect detection result. For example, the final defect detection result for pin instance 305 is defect type: bending, defect location: (0.35, -0.42, 0.28). The defect detection result can be represented as pin number 305, defect type: bending, defect location: (0.35, -0.42, 0.28), indicating that the pin has a bending defect, and the defect is located at (0.35, -0.42, 0.28) in the local coordinate system of the pin.

[0069] In this embodiment, by calculating the node correlation degree between pin instances based on aggregated optimized node features, the similarity determination between adjacent pins is improved from a simple spatial proximity relationship to a comprehensive correlation model that takes into account both structural morphology and feature consistency. This avoids the interference of irrelevant instances introduced by physical distance on defect determination and improves the accuracy of adjacent relationship identification. By introducing node correlation degree and spatial Euclidean distance into defect propagation probability modeling and using spatial attenuation factor to constrain propagation intensity, the propagation of defect information conforms to the physical proximity characteristics of pin arrangement and is effectively restricted by structural similarity. This avoids the problem of unconstrained diffusion of defect labels in the array and improves the rationality and credibility of propagation results. By weighted fusion of the internal detection confidence of the current pin instance and the external propagation confidence from spatially adjacent pin instances, the organic combination of single instance determination and group consistency constraint is realized. In the case of unstable or uncertain local detection results, the determination results can be corrected with the help of the collaborative information of adjacent pins, which significantly reduces the probability of misjudgment and missed judgment.

[0070] Figure 2 This is a flowchart illustrating the defect propagation and iterative optimization process of the intelligent detection method for connector pin defects based on machine vision, as described in an embodiment of the present invention.

[0071] A second aspect of this invention provides a machine vision-based intelligent detection system for connector pin defects, comprising: The pin region construction unit is used to acquire multi-view image sequences of the connector under multi-source illumination conditions, solve for three-dimensional topography data based on the pixel grayscale values ​​of the same spatial position under different illumination angles, extract edge pixels with significant surface curvature changes from the three-dimensional topography data and group them according to commonalities to obtain candidate pin regions, perform spatial position verification on the candidate pin regions and retain pin regions that meet the preset design specifications, assign node identifiers to each pin region and establish association edges between spatially adjacent and posture-coordinated pin regions; The feature optimization detection unit is used to take the surface normal vector and boundary contour corresponding to the pin region as node features, propagate the node features to neighboring nodes along the associated edge and aggregate the neighbor propagated features to obtain optimized node features, determine the pin instance set based on the optimized node features, extract the surface morphology features of each pin instance in the pin instance set from the three-dimensional morphology data and concatenate them with the optimized node features to obtain a high-dimensional feature representation, and decode the high-dimensional feature representation after performing multi-scale convolution operation to obtain a preliminary detection result. The defect label propagation unit is used to calculate the defect propagation probability between each pair of spatially adjacent pin instances in the pin instance set, and to iteratively update the defect type label of each pin in the preliminary detection result based on the defect propagation probability to obtain the defect detection result.

[0072] A third aspect of the present invention provides an electronic device, comprising: A processor and a memory for storing processor-executable instructions, wherein the processor is configured to invoke instructions stored in the memory to perform the aforementioned method.

[0073] A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0074] This invention can be a method, apparatus, system, and / or computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of the invention.

[0075] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A machine vision-based intelligent detection method for connector pin defects, characterized in that, include: A multi-view image sequence of the connector to be tested under multi-source illumination conditions is acquired. The three-dimensional topography data is obtained by solving the pixel grayscale values ​​of the same spatial position under different illumination angles. Edge pixels with significant surface curvature changes are extracted from the three-dimensional topography data and grouped according to common characteristics to obtain candidate pin regions. After spatial position verification of the candidate pin regions, pin regions that meet the preset design specifications are retained. Node identifiers are assigned to each pin region and associated edges are established between spatially adjacent and posture-coordinated pin regions. The surface normal vector and boundary contour corresponding to the pin region are used as node features. The node features are propagated to neighboring nodes along the associated edge and the neighbor propagated features are aggregated to obtain optimized node features. Based on the optimized node features, a set of pin instances is determined. The surface morphology features of each pin instance in the set of pin instances are extracted from the 3D morphology data and concatenated with the optimized node features to obtain a high-dimensional feature representation. After performing multi-scale convolution operation on the high-dimensional feature representation, the initial detection result is obtained by decoding. Calculate the defect propagation probability between each pair of spatially adjacent pin instances in the pin instance set, and iteratively update the defect type label of each pin in the preliminary detection result based on the defect propagation probability to obtain the defect detection result.

2. The method according to claim 1, characterized in that, A multi-view image sequence of the connector to be tested under multi-source illumination conditions is acquired. Three-dimensional topography data is obtained by calculating the pixel grayscale values ​​of the same spatial location under different illumination angles. Edge pixels with significant surface curvature changes are extracted from the three-dimensional topography data and grouped according to common characteristics to obtain candidate pin regions, including: Multiple light sources are lit sequentially according to a preset time sequence. When each light source is lit, the camera captures an image of the connector to be detected. The number of images of the connector to be detected is the same as the number of light sources, and they are combined to obtain a multi-view image sequence. For each pixel position in the multi-view image sequence, extract multiple pixel gray values ​​corresponding to the pixel position when different light sources are lit. Substitute the multiple pixel gray values ​​and the incident angle of the corresponding light source into the preset photometric stereo constraint equation for solving, and obtain the surface normal vector and reflectivity of the spatial point corresponding to each pixel position. Traverse all pixel positions to obtain three-dimensional topography data. The surface curvature value is obtained by calculating the gradient of the surface normal vector of each spatial point in the three-dimensional topography data in the horizontal and vertical directions. Spatial points with surface curvature values ​​greater than a curvature threshold are marked as candidate edge pixels. The spatial distance between any two candidate edge pixels is calculated. Candidate edge pixels with spatial distances less than a preset connectivity distance are grouped into the same connectivity group. For each connectivity group, the spatial coordinates of all candidate edge pixels in the connectivity group are extracted and convex hull calculation is performed to obtain the minimum closed contour surrounding the connectivity group. The minimum closed contour is used as the boundary contour, and candidate pin regions are determined based on the boundary contour.

3. The method according to claim 1, characterized in that, After spatial position verification of the candidate pin regions, pin regions that conform to the preset design specifications are retained. Each pin region is assigned a node identifier, and association edges are established between spatially adjacent and attitude-coordinated pin regions, including: The centroids corresponding to the three-dimensional coordinates of all spatial points within the candidate pin region are calculated to obtain the center coordinates of the candidate pin region. Principal component decomposition is performed on the boundary contour of the candidate pin region to obtain the principal direction vector, which is used as the attitude direction of the candidate pin region. The center coordinates are projected onto the horizontal reference plane of the spatial coordinate system to obtain the projected coordinates. The candidate pin regions are then arranged in a grid according to the row and column positions of the projected coordinates to obtain a two-dimensional array distribution. For each grid position in the two-dimensional array distribution, when the number of candidate pin regions in the grid position is greater than one, the candidate pin region with the closest boundary contour area to the standard pin area and the highest boundary contour integrity is retained, and other candidate pin regions in the grid position are removed to obtain the redundant pin region. The absolute value of the difference between the center coordinate distance of the redundant pin region corresponding to the adjacent grid positions in the two-dimensional array distribution and the preset pin design spacing is calculated as the center distance deviation. The redundant pin regions with a center distance deviation less than the position verification threshold are retained as pin regions that meet the design specifications. For each pin region, a node identifier is generated according to the row index and column index in the two-dimensional array distribution. The inner product of the attitude direction vector of the pin region and the corresponding pin region of the adjacent grid position is calculated as the attitude cooperation metric. An association edge is established between adjacent pin regions whose attitude cooperation metric is greater than the cooperation threshold.

4. The method according to claim 1, characterized in that, Using the surface normal vector and boundary contour corresponding to the pin region as node features, the node features are propagated to neighboring nodes along the associated edges, and the neighbor propagation features are aggregated to obtain optimized node features, including: The surface normal vectors of all spatial points within the boundary contour corresponding to the stitch area are obtained and statistical analysis is performed to obtain the mean vector and variance value of the surface normal vectors. The mean vector and variance value are then concatenated to form the surface normal vector feature. The boundary contour of the stitch region is sampled to obtain a sequence of key points. The spatial distance and angle change between adjacent key points in the sequence of key points are calculated and spliced ​​to obtain the boundary contour features. The surface normal vector features and the boundary contour features are spliced ​​to obtain the node features corresponding to the stitch region. For each pin region, the node identifiers of all pin regions connected to the current pin region through the associated edge are found according to the node identifier of the pin region to obtain the neighboring node identifiers. The node features of the pin regions corresponding to all neighboring node identifiers are extracted to obtain the neighboring node feature set. The weight value is determined based on the associated edge, and the node features in the neighboring node feature set are weighted and summed according to the weight value to obtain the neighborhood propagation feature. Calculate the feature similarity between the node features and the neighborhood propagation features, determine the adaptive weight coefficient based on the feature similarity, and perform feature fusion on the node features and neighborhood propagation features corresponding to the pin region according to the adaptive weight coefficient to obtain optimized node features.

5. The method according to claim 1, characterized in that, Based on the optimized node features, a set of pin instances is determined. The surface morphology features of each pin instance in the set are extracted from the 3D morphology data and concatenated with the optimized node features to obtain a high-dimensional feature representation. After performing multi-scale convolution operations on the high-dimensional feature representation, decoding yields preliminary detection results, including: Based on the feature similarity between the optimized node features, the feature distance between each pair of optimized node features is calculated using a hierarchical clustering algorithm. Optimized node features whose feature distance is less than a preset initial clustering threshold are merged into an initial cluster. The different initial clusters with the smallest inter-cluster distance are repeatedly merged until the inter-cluster distance is greater than the preset merging distance threshold, thus obtaining optimized clusters and using the pin regions corresponding to the optimized clusters as pin instances. The three-dimensional topography data within the spatial range enclosed by the boundary contour in the pin instance is obtained as a topography data block. The direction distribution statistics of the surface normal vectors in the topography data block are performed to obtain a normal vector distribution histogram. The height distribution statistics of the spatial points in the topography data block are performed to obtain a height distribution histogram and are spliced ​​with the normal vector distribution histogram to obtain the surface topography features. The optimized node features corresponding to the pin regions contained in the pin instance are averaged to obtain aggregated optimized node features, which are then concatenated with the surface morphology features to obtain a high-dimensional feature representation. Multi-scale dilated convolution is performed on the high-dimensional feature representation to obtain adaptive feature maps of multiple receptive field scales. Channel splicing and decoding are performed on the adaptive feature maps to obtain the defect type prediction probability distribution and defect location prediction coordinates, which are then combined to obtain preliminary detection results.

6. The method according to claim 5, characterized in that, Multi-scale dilated convolution is performed on the high-dimensional feature representation to obtain adaptive feature maps with multiple receptive field scales. Channel concatenation and decoding are performed on the adaptive feature maps to obtain the defect type prediction probability distribution and defect location prediction coordinates. These are then combined to obtain preliminary detection results, including: The high-dimensional feature representation is input into multiple pre-defined dilated convolutional layers with increasing dilation rates to perform dilated convolution operations to obtain initial feature maps at multiple receptive field scales. For each initial feature map, the offset of the pixel position is learned through convolution operations. The sampling position of the convolution kernel is adaptively adjusted according to the offset and the convolution operation is performed again to obtain adaptive feature maps at multiple receptive field scales. The adaptive feature maps are concatenated along the channel dimension to obtain a concatenated feature map. The concatenated feature map is then subjected to global average pooling and max pooling in the spatial dimension and a spatial attention weight map is calculated through a convolutional layer. Finally, the spatial attention weight map is multiplied pixel-by-pixel with the concatenated feature map to obtain a spatially weighted feature map. The spatially weighted feature map is mapped using a fully connected layer and a normalization layer to obtain the defect type prediction probability distribution. The spatially weighted feature map is then mapped using a fully connected layer and a regression activation layer to obtain the defect location prediction coordinates. The category with the highest probability value is selected from the defect type prediction probability distribution as the predicted defect type. The predicted defect type and the defect location prediction coordinates are then combined to obtain the preliminary detection result.

7. The method according to claim 1, characterized in that, Calculate the defect propagation probability between each pair of spatially adjacent pin instances in the pin instance set, and iteratively update the defect type label of each pin in the preliminary detection result based on the defect propagation probability to obtain the defect detection result, including: For each pin instance in the pin instance set, the node correlation degree between the current pin instance and the other pin instances in the pin instance set is calculated based on the aggregated optimized node features corresponding to the current pin instance. Pin instance pairs with node correlation degrees greater than a preset correlation degree threshold are regarded as spatially adjacent pin instance pairs. The center coordinates of the spatially adjacent pin instances are obtained and the spatial Euclidean distance is calculated. The spatial attenuation factor is calculated based on the spatial Euclidean distance and the defect propagation probability between spatially adjacent pin instances is obtained by combining the node correlation degree. Obtain all spatially adjacent pin instances corresponding to the pin instance and the defect propagation probability corresponding to the spatially adjacent pin instances, and calculate the external propagation confidence vector. Use the defect type prediction probability distribution corresponding to the current pin instance as the internal detection confidence vector and the external propagation confidence vector to perform weighted fusion according to a preset fusion ratio to obtain the fused confidence distribution. Update the defect type label in the preliminary detection result based on the fused confidence distribution. The defect type labels of all pin instances in the pin instance set are repeatedly updated until the defect type labels of all pin instances remain unchanged in continuous iterations or reach a preset number of iterations. The defect type labels obtained in the last iteration are combined with the defect location prediction coordinates to obtain the defect detection result.

8. A machine vision-based intelligent detection system for connector pin defects, used to implement the method of any one of claims 1-7, characterized in that, include: The pin region construction unit is used to acquire multi-view image sequences of the connector under multi-source illumination conditions, solve for three-dimensional topography data based on the pixel grayscale values ​​of the same spatial position under different illumination angles, extract edge pixels with significant surface curvature changes from the three-dimensional topography data and group them according to commonalities to obtain candidate pin regions, perform spatial position verification on the candidate pin regions and retain pin regions that meet the preset design specifications, assign node identifiers to each pin region and establish association edges between spatially adjacent and posture-coordinated pin regions; The feature optimization detection unit is used to take the surface normal vector and boundary contour corresponding to the pin region as node features, propagate the node features to neighboring nodes along the associated edge and aggregate the neighbor propagated features to obtain optimized node features, determine the pin instance set based on the optimized node features, extract the surface morphology features of each pin instance in the pin instance set from the three-dimensional morphology data and concatenate them with the optimized node features to obtain a high-dimensional feature representation, and decode the high-dimensional feature representation after performing multi-scale convolution operation to obtain a preliminary detection result. The defect label propagation unit is used to calculate the defect propagation probability between each pair of spatially adjacent pin instances in the pin instance set, and to iteratively update the defect type label of each pin in the preliminary detection result based on the defect propagation probability to obtain the defect detection result.

9. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1 to 7.

Images