Precision connector pin full-element visual detection method, system, medium and device

Abstract

The application discloses a kind of precision connector PIN pin full-element visual detection method, system, medium and device, method includes: target data is obtained by processing camera image acquisition, target data includes backlight correction diagram, texture correction diagram, height diagram, amplitude diagram and curvature diagram;Based on target data, multiple modal adaptive fusion is carried out using encoder to obtain fusion feature map;Initial detection result is obtained by using the parallel decoding and prediction of the preset three branches to the fusion feature map;Based on initial detection result, result association, quantitative calculation and comprehensive decision are carried out to output target detection result.The application realizes the integrated intelligent detection of the size, shape, position and surface full-element of precision PIN pin, has nanoscale three-dimensional quantitative capacity and excellent robustness, through multimodal optical synchronous imaging and adaptive deep learning, significantly improves the detection rate of weak and hidden defects, reduces false alarm, meets high-speed production line beat, substantially reduces the cost of manual debugging and maintenance.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically, to a method, system, medium, and device for full-element visual inspection of precision connector PIN pins. Background Technology

[0002] As a key basic component for signal and power transmission in electronic systems, precision connectors rely heavily on their core component—PIN pins—to determine the reliability of the entire connector and even electronic devices. PIN pins can be subject to a variety of defects during manufacturing, including: dimensional inconsistencies (such as mismatched pin diameter, width, and thickness), shape defects (such as head collapse, bending, twisting, and root burrs), poor positional accuracy (such as poor coplanarity, uneven spacing, and reversed polarity), and surface defects (such as scratches, plating peeling, and oxidation). These micron-level defects can lead to abnormal insertion and extraction forces, increased contact resistance, signal transmission interruptions, and even serious consequences such as short circuits.

[0003] Traditional inspection methods mainly rely on manual visual inspection and contact measurement. Manual visual inspection is inefficient, easily affected by subjective factors and fatigue, and has a high rate of missing minute defects. Contact measurement (such as micrometers and coordinate measuring machines, CMMs) is highly accurate, but it is slow, easily causes secondary scratches on the surface of precision pins, and is difficult to achieve 100% inspection. Summary of the Invention

[0004] The purpose of this invention is to provide a method, system, medium, and device for full-element visual inspection of precision connector PIN pins, which solves the problems in the prior art such as the single information dimension leading to missed detection of three-dimensional defects, sensitivity to light fluctuations, weak detection capability of hidden areas, lack of adaptive intelligent judgment, and difficulty in balancing detection efficiency and comprehensiveness.

[0005] The first aspect of this invention provides a method for full-element visual inspection of precision connector pins, comprising the following steps:

[0006] The target data is obtained by acquiring and processing camera images. The target data includes a backlight correction map, a texture correction map, a height map, a magnitude map, and a curvature map.

[0007] Based on the target data, a multimodal adaptive fusion is performed using a preset encoder to obtain a fused feature map;

[0008] The initial detection result is obtained by parallel decoding and prediction of the fused feature map using three preset branches;

[0009] Based on the initial detection results, result correlation, quantitative calculation and comprehensive decision-making are performed to output the target detection results.

[0010] In this solution, the process of acquiring camera images and obtaining target data specifically includes:

[0011] The captured backlight PIN silhouette image, multi-frame structured light deformation image, and PIN surface texture image are used as camera images;

[0012] The camera image is subjected to image correction to obtain a corrected image, which includes the backlight correction image and the texture correction image;

[0013] A mask is generated based on the corrected image, and an initial three-dimensional height map is obtained by combining the generated binary mask with the corrected image for three-dimensional reconstruction.

[0014] The height map, the magnitude map, and the curvature map are calculated based on the initial three-dimensional height map;

[0015] Five-channel target data is obtained based on the backlight correction map, the texture correction map, the height map, the amplitude map, and the curvature map.

[0016] In this scheme, the step of performing multimodal adaptive fusion based on the target data using a preset encoder to obtain a fused feature map specifically includes:

[0017] Based on the encoder, feature pyramids of different scales are obtained by extracting features from the target data.

[0018] Global average pooling is performed on the feature map of each channel in the feature pyramid to obtain the channel descriptor vector;

[0019] Based on the channel descriptor vector, a fusion weight is generated for each feature map using a preset attention network;

[0020] Based on the fusion weights and the corresponding feature maps, a weighted fusion is performed to obtain a fused feature map, calculated as follows:

[0021] ;

[0022] in, To fuse feature maps, The number of pyramid levels. For the first The fusion weights corresponding to each channel For the first Feature maps corresponding to each channel.

[0023] In this scheme, the initial detection result is obtained by parallel decoding and prediction of the fused feature map using three preset branches, specifically including:

[0024] The fused feature map is input into three branches for processing: an instance segmentation branch, a semantic segmentation branch, and a regression branch.

[0025] The instance segmentation branch is used to detect and segment each independent PIN pin instance, and the PIN pin bounding box and PIN pin instance segmentation mask are output.

[0026] The semantic segmentation branch is used to output a pixel-level classification map of the same resolution, in which pixels are classified into background, pin body, pin root and defect area.

[0027] The regression branch is used to predict a parametric map related to geometry, which includes a height regression map, a key point heatmap, and an orientation map.

[0028] In this scheme, the height map is optimized and denoised to obtain the height regression map. The key point heat map is obtained by predicting the position of the PIN head vertex and the PIN root center point based on the PIN body pixel set and the PIN root pixel set. The orientation map is obtained by predicting the normal direction of the surface of each PIN.

[0029] In this solution, the step of performing result correlation, quantitative calculation, and comprehensive decision-making based on the initial detection results to output the target detection results specifically includes:

[0030] Each PIN pin instance output by the instance segmentation branch is correlated with the processing results of the semantic segmentation branch and the regression branch;

[0031] Calculate pin dimensions, pin position, pin shape parameters, and defect statistics to complete the quantitative calculation;

[0032] The calculated quantitative parameters are combined with preset process specifications to make a comprehensive decision to output the target detection result, which includes a structured report, visualization instructions and IO interface control signals.

[0033] A second aspect of the present invention also provides a full-element visual inspection system for precision connector PIN pins, comprising a memory and a processor. The memory includes a method program for full-element visual inspection of precision connector PIN pins. When the processor executes the method program for full-element visual inspection of precision connector PIN pins, it performs the following steps:

[0034] The target data is obtained by acquiring and processing camera images. The target data includes a backlight correction map, a texture correction map, a height map, a magnitude map, and a curvature map.

[0035] Based on the target data, a multimodal adaptive fusion is performed using a preset encoder to obtain a fused feature map;

[0036] The initial detection result is obtained by parallel decoding and prediction of the fused feature map using three preset branches;

[0037] Based on the initial detection results, result correlation, quantitative calculation and comprehensive decision-making are performed to output the target detection results.

[0038] In this solution, the process of acquiring camera images and obtaining target data specifically includes:

[0039] The captured backlight PIN silhouette image, multi-frame structured light deformation image, and PIN surface texture image are used as camera images;

[0040] The camera image is subjected to image correction to obtain a corrected image, which includes the backlight correction image and the texture correction image;

[0041] A mask is generated based on the corrected image, and an initial three-dimensional height map is obtained by combining the generated binary mask with the corrected image for three-dimensional reconstruction.

[0042] The height map, the magnitude map, and the curvature map are calculated based on the initial three-dimensional height map;

[0043] Five-channel target data is obtained based on the backlight correction map, the texture correction map, the height map, the amplitude map, and the curvature map.

[0044] In this scheme, the step of performing multimodal adaptive fusion based on the target data using a preset encoder to obtain a fused feature map specifically includes:

[0045] Based on the encoder, feature pyramids of different scales are obtained by extracting features from the target data.

[0046] Global average pooling is performed on the feature map of each channel in the feature pyramid to obtain the channel descriptor vector;

[0047] Based on the channel descriptor vector, a fusion weight is generated for each feature map using a preset attention network;

[0048] Based on the fusion weights and the corresponding feature maps, a weighted fusion is performed to obtain a fused feature map, calculated as follows:

[0049] ;

[0050] in, To fuse feature maps, The number of pyramid levels. For the first The fusion weights corresponding to each channel For the first Feature maps corresponding to each channel.

[0051] In this scheme, the initial detection result is obtained by parallel decoding and prediction of the fused feature map using three preset branches, specifically including:

[0052] The fused feature map is input into three branches for processing: an instance segmentation branch, a semantic segmentation branch, and a regression branch.

[0053] The instance segmentation branch is used to detect and segment each independent PIN pin instance, and the PIN pin bounding box and PIN pin instance segmentation mask are output.

[0054] The semantic segmentation branch is used to output a pixel-level classification map of the same resolution, in which pixels are classified into background, pin body, pin root and defect area.

[0055] The regression branch is used to predict a parametric map related to geometry, which includes a height regression map, a key point heatmap, and an orientation map.

[0056] In this scheme, the height map is optimized and denoised to obtain the height regression map. The key point heat map is obtained by predicting the position of the PIN head vertex and the PIN root center point based on the PIN body pixel set and the PIN root pixel set. The orientation map is obtained by predicting the normal direction of the surface of each PIN.

[0057] In this solution, the step of performing result correlation, quantitative calculation, and comprehensive decision-making based on the initial detection results to output the target detection results specifically includes:

[0058] Each PIN pin instance output by the instance segmentation branch is correlated with the processing results of the semantic segmentation branch and the regression branch;

[0059] Calculate pin dimensions, pin position, pin shape parameters, and defect statistics to complete the quantitative calculation;

[0060] The calculated quantitative parameters are combined with preset process specifications to make a comprehensive decision to output the target detection result, which includes a structured report, visualization instructions and IO interface control signals.

[0061] A third aspect of the present invention provides a computer-readable storage medium comprising a machine program for a full-element visual inspection method of a precision connector PIN pin, wherein when executed by a processor, the full-element visual inspection method program of the precision connector PIN pin implements the steps of the full-element visual inspection method of a precision connector PIN pin as described in any of the preceding claims.

[0062] A fourth aspect of the present invention provides a full-element visual inspection device for precision connector pins, comprising:

[0063] Optical imaging module and detection module, wherein,

[0064] The optical imaging module includes an industrial camera, a dual telecentric lens, and a light source. The industrial camera and the dual telecentric lens are used to acquire images, and the light source is used to provide illumination or grating stripes.

[0065] The detection module includes an industrial control computer, used to implement the steps of a full-element visual inspection method for precision connector PIN pins as described in any of the above.

[0066] In this solution, the industrial camera includes a high frame rate camera, and the light source includes a high uniformity backlight source, a multi-angle composite ring light source, and a micro structured light projection module. The high uniformity backlight source is used to generate a high-contrast PIN silhouette, the multi-angle composite ring light source is used to provide uniform surface illumination or illumination at a specific angle, and the micro structured light projection module is used to project high-precision digital grating stripes onto the PIN surface.

[0067] The present invention discloses a method, system, medium, and device for full-element visual inspection of precision connector PIN pins, which has the following beneficial effects:

[0068] 1. Unprecedented comprehensiveness of detection capabilities: For the first time, simultaneous and integrated detection of four major categories of defects in PIN pins—size, shape, position, and surface—is achieved in a single system. Through multimodal optics and a multi-task pyramid fusion network, it utilizes two-dimensional texture, three-dimensional morphology, and their derived features (gradient, curvature) to eliminate detection blind spots.

[0069] 2. Nanoscale 3D Quantization Capability: Structured light-based 3D reconstruction provides high-precision topographic information, making it possible to accurately measure 3D parameters such as coplanarity, curvature, and spherical curvature. The measurement accuracy can reach the micrometer or even submicrometer level, which is far superior to traditional 2D methods.

[0070] 3. Excellent robustness and high sensitivity: The multimodal feature adaptive fusion module enables the network to adapt to illumination and process fluctuations, and the attention mechanism enables it to focus on suspicious areas. For weak defects (such as hairline scratches), low-contrast defects (such as slight oxidation), and occluded root burrs, the detection sensitivity and stability are far higher than traditional threshold or template methods.

[0071] 4. True intelligence and self-adaptation: The network is trained with a large amount of good and bad product data, and learns the normal appearance statistical characteristics and defect patterns of PIN pins. It has a strong feature representation and discrimination ability. For new defects or "gray area" products, its judgment is closer to the level of human experts than algorithms based on fixed rules, and there is no need to frequently adjust parameters manually.

[0072] 5. High efficiency and cost-effectiveness: One imaging and one algorithm process complete all inspection items, which greatly improves the inspection efficiency and meets the cycle time requirements of high-speed production lines. Although the cost of a single hardware set may be higher than that of a simple two-dimensional system, its comprehensive inspection capability far exceeds that of a solution that requires multiple systems, and the overall cost-effectiveness is higher. Attached Figure Description

[0073] Figure 1 This invention illustrates the steps of a method for full-element visual inspection of precision connector PIN pins according to the present invention.

[0074] Figure 2 This diagram illustrates the visualization of the detection results of a full-element visual inspection method for precision connector PIN pins according to the present invention.

[0075] Figure 3 A block diagram of a full-element visual inspection system for precision connector pins according to the present invention is shown.

[0076] Figure 4 The diagram illustrates the application of the precision connector PIN pin full-element visual inspection device of the present invention. Detailed Implementation

[0077] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments of the present invention and the features thereof can be combined with each other.

[0078] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0079] Existing machine vision-based PIN detection solutions, based on their technical approaches, include methods based on two-dimensional grayscale image processing, contour detection methods based on laser triangulation, and methods based on traditional template matching and feature extraction.

[0080] Background Technology and Existing Technology Solution 1: A method based on two-dimensional grayscale images and threshold segmentation, which is the most basic visual inspection scheme. The system typically consists of an area scan camera, a backlight source, and a side-illuminated source. During inspection, the silhouette image of the PIN pin is first acquired using backlight illumination. The outline of the PIN pin is obtained through binarization and edge extraction (such as the Canny operator), and its macroscopic dimensions (such as total length and maximum width) and position (such as center coordinates) are measured. Then, the illumination is switched to side or ring light source to acquire the surface image of the PIN pin. Obvious surface scratches or stains are detected by analyzing the grayscale distribution or texture features of the image. The core of this method lies in threshold segmentation, the formula of which can be expressed as: For the image... By setting a global threshold Generate a binary image :

[0081] ;

[0082] This solution is simple in structure and low in cost, and it is effective for detecting the size and position of PIN pins with clear outlines. However, its drawbacks are extremely prominent: firstly, it is extremely sensitive to changes in lighting conditions, and its threshold value is very high. It requires frequent manual adjustments and has poor stability. Secondly, it cannot acquire the three-dimensional morphological information of the PIN, and is powerless to detect three-dimensional shape defects such as bending, coplanarity, and spherical curvature of the head. Finally, it has poor detection performance and a high false negative rate for defects with low contrast to the background (such as shallow scratches and slight oxidation), as well as for the PIN root area that is obscured by other structures.

[0083] Background Technology and Existing Technology Solution 2: Contour Detection Methods Based on Laser Triangulation or Structured Light. To obtain 3D information, some systems use a line laser to project a laser line onto a pin array. Due to the varying heights of the pins, the laser line deforms from the camera's perspective. Using the principle of laser triangulation, the height information of the cross-section containing the laser line can be calculated. The entire PIN array's 3D point cloud can be reconstructed by scanning line by line or using a motion platform. This method can effectively measure the coplanarity, curvature, and head shape of the PINs. Its underlying formula is as follows:

[0084] ;

[0085] in, It is the baseline distance between the laser and the camera's optical center. It is the angle between the laser plane and the camera's optical axis. It's the camera's focal length. This refers to the pixel displacement of the laser line in the image. While this approach provides valuable 3D data, it has high hardware requirements, complex system calibration, and relatively slow detection speed. More importantly, it primarily focuses on macroscopic contour morphology and is insensitive to microscopic defects on the PIN pin surface (such as micro-scratches and tiny tip collapses) as well as color and texture variations (such as plating anomalies). These require supplementary 2D vision systems, increasing system complexity and cost.

[0086] Background Technology and Existing Technology Solution 3: A method based on rigid template matching and geometric features, which is a more "intelligent" traditional method. The system first learns an image of a standard, high-quality PIN as a template. During detection, in the image to be inspected... To find the region most similar to the template, normalized cross-correlation (NCC) is typically used as the similarity measure.

[0087] ;

[0088] in, These are the average values ​​of the template and the current window of the image to be inspected, respectively. When If the score falls below a certain threshold, it is considered a defect. Alternatively, the system extracts a series of hand-designed features from the image, such as HOG (Histogram of Oriented Gradients) and LBP (Local Binary Pattern), and then uses a classifier (such as SVM) to determine whether a feature is good or bad. This method is more robust than a simple thresholding method. However, its performance heavily depends on the perfection of the template and the experience of feature design. When there are permissible minor manufacturing variations or novel defects not seen in the training set, this method has insufficient generalization ability and is prone to misclassification. Furthermore, it is also difficult to accurately quantify three-dimensional geometric parameters.

[0089] Through in-depth analysis of the above-mentioned existing technical solutions, the following key shortcomings of existing technologies in meeting the comprehensive testing requirements of precision PIN pins can be summarized:

[0090] 1. Limited information dimension and incomplete detection capabilities: Two-dimensional methods lack depth information and cannot handle three-dimensional defects such as coplanarity and curvature; three-dimensional contour methods are insensitive to surface micro-defects. There is a lack of an effective means to simultaneously acquire and fuse two-dimensional texture and three-dimensional shape information.

[0091] 2. Insufficient detection sensitivity and stability: Methods based on global thresholds or rigid templates have poor adaptability to changes in lighting, angle, and allowable process fluctuations, resulting in missed detection of weak defects (such as shallow scratches) or false alarms for normal fluctuations.

[0092] 3. Weak detection capability for complex defects and hidden areas such as the root: The root of the PIN pin usually has a complex structure and is easily obscured, and defects such as burrs have irregular shapes, making it difficult for traditional algorithms to work stably in this area.

[0093] 4. Lack of adaptive and intelligent judgment capabilities: Thresholds and template parameters require manual setting and maintenance, and cannot be self-optimized through data. For the "grey area" between good and defective products, there is a lack of intelligent decision-making capabilities based on statistical learning.

[0094] 5. The contradiction between inspection efficiency and comprehensiveness: Achieving comprehensive inspection (size + shape + position + surface) often requires multiple systems or multiple inspections, which restricts the production line cycle time.

[0095] To address these shortcomings, the present invention aims to provide a comprehensive intelligent visual inspection system and method for precision connector PIN pins based on multimodal optical imaging and deep learning feature fusion. This invention aims to simultaneously capture the two-dimensional texture and three-dimensional morphology information of PIN pins in a single imaging process through innovative optical design; and to achieve high-precision, robust, and integrated detection and quantification of various defects by constructing a deep learning network capable of adaptively learning fluctuations in good product characteristics and deeply fusing multi-source information. Specific objectives include: achieving simultaneous online detection of PIN pin size, shape, position, and surface quality; significantly improving the detection rate of weak defects, three-dimensional deformations, and hidden defects at the root; greatly reducing the system's debugging and maintenance complexity and enhancing its adaptability; and maintaining an extremely high detection rate while keeping the false alarm rate at an extremely low level to meet the cycle time requirements of high-speed production lines. The core innovations and key protection points are:

[0096] 1. Multimodal optical synchronous imaging system for full-element detection of PIN needles: Special protection for a specific hardware architecture and control system that integrates backlighting, structured light projection and surface texture illumination, and can quickly trigger acquisition in sequence.

[0097] 2. A high-precision reconstruction method for the 3D topography of PIN pins based on structured light projection: Under the guidance of a backlight mask, the height map of the PIN pin region is rapidly reconstructed using the phase-shifting method and phase unwrapping. And calculate its gradient map. curvature diagram The complete algorithm flow.

[0098] 3. Overall architecture of Multi-Task Pyramid Fusion Network (MTPF-Net): Protection based on multimodal image stacking The design incorporates a multi-modal feature adaptive fusion module as input and simultaneously outputs instance segmentation, semantic segmentation, and regression graphs with multiple parameters into a multi-task learning network.

[0099] 4. Specific implementation of the multimodal feature adaptive fusion module: Protect the technical solution of dynamically weighting and fusing different modal features at each level of the network encoder through an attention mechanism.

[0100] 5. Multi-task decoder design for PIN needle detection: a specific architecture that protects the parallel operation of three branches: instance segmentation (localization), semantic segmentation (defect classification), and parameter regression (morphology quantization), and the final results are fused.

[0101] 6. A comprehensive quantification and decision-making method for PIN pins based on network output: This method protects the systematic process of associating instance, semantic, and regression information from network output to calculate the size, position, shape parameters, and defect severity of each PIN pin.

[0102] Specifically, Figure 1 The diagram illustrates the steps of a full-element visual inspection method for precision connector pins according to the present invention.

[0103] like Figure 1 As shown, this invention discloses a method for full-element visual inspection of precision connector PIN pins, comprising the following steps:

[0104] S102, acquire camera images and process them to obtain target data, the target data including backlight correction map, texture correction map, height map, amplitude map and curvature map;

[0105] S104, Based on the target data, multimodal adaptive fusion is performed using a preset encoder to obtain a fused feature map;

[0106] S106, the fused feature map is decoded and predicted in parallel using three preset branches to obtain the initial detection result;

[0107] S108, based on the initial detection results, perform result correlation, quantitative calculation and comprehensive decision-making to output the target detection results.

[0108] It should be noted that, in this embodiment, backlight silhouette, structured light stripes, and surface texture images are first acquired simultaneously. Then, the three-dimensional height map is reconstructed using structured light phase calculation, and combined with gradient and curvature maps to form a multimodal input. Then, the multimodal features are adaptively fused through a multi-task pyramid fusion network to simultaneously complete instance segmentation, semantic segmentation, and parameter regression. Finally, the outputs of each task are correlated, and the size, position, shape parameters, and defect information of each pin are quantitatively calculated. The quality is comprehensively judged to output the target detection result, which can be displayed through a visualization report, for example.

[0109] According to an embodiment of the present invention, the process of acquiring camera images and processing them to obtain target data specifically includes:

[0110] The captured backlight PIN silhouette image, multi-frame structured light deformation image, and PIN surface texture image are used as camera images;

[0111] The camera image is subjected to image correction to obtain a corrected image, which includes the backlight correction image and the texture correction image;

[0112] A mask is generated based on the corrected image, and an initial three-dimensional height map is obtained by combining the generated binary mask with the corrected image for three-dimensional reconstruction.

[0113] The height map, the magnitude map, and the curvature map are calculated based on the initial three-dimensional height map;

[0114] Five-channel target data is obtained based on the backlight correction map, the texture correction map, the height map, the amplitude map, and the curvature map.

[0115] It should be noted that, in this embodiment, the acquired silhouette image of the backlight PIN pins is used. Multi-frame structured light deformation diagram and PIN pin surface texture map as camera image The camera images are subjected to image correction to obtain a corrected image, including distortion correction and brightness uniformity correction of all original images. The corrected image includes the backlight correction image and the texture correction image. A mask is generated based on the corrected image to obtain the corrected backlight pin silhouette image. Thresholding and binarization are performed to generate a binary mask containing only the PIN region. This mask is used in subsequent steps to eliminate background interference. The generated binary mask is then combined with the corrected image to perform 3D reconstruction, resulting in an initial 3D height map. Based on the initial three-dimensional height map Calculate the height map The amplitude diagram and the curvature diagram Then, based on the backlight correction map, the texture correction map, the height map, the amplitude map, and the curvature map, five-channel target data is obtained.

[0116] According to an embodiment of the present invention, the step of performing multimodal adaptive fusion based on the target data using a preset encoder to obtain a fused feature map specifically includes:

[0117] Based on the encoder, feature pyramids of different scales are obtained by extracting features from the target data.

[0118] Global average pooling is performed on the feature map of each channel in the feature pyramid to obtain the channel descriptor vector;

[0119] Based on the channel descriptor vector, a fusion weight is generated for each feature map using a preset attention network;

[0120] Based on the fusion weights and the corresponding feature maps, a weighted fusion is performed to obtain a fused feature map, calculated as follows:

[0121] ;

[0122] in, To fuse feature maps, The number of pyramid levels. For the first The fusion weights corresponding to each channel For the first Feature maps corresponding to each channel.

[0123] It should be noted that, in this embodiment, feature pyramids of different scales are obtained by extracting features from the target data based on the encoder. In each layer of the encoder, a multimodal feature adaptive fusion module is introduced to intelligently fuse information from different modalities. This module receives feature maps from multiple modalities at the same level. First, global average pooling is performed on the feature maps of each modality to obtain channel descriptor vectors. Then, a fusion weight is generated for each modality through a small attention network (two fully connected layers). This characterizes the importance of the modality at the current level, and finally, a weighted fusion is performed: This allows the network to dynamically select the appropriate methods for the current task (such as detecting small scratches, which may require more reliance on other methods). Detecting bending requires more reliance on and The most relevant information.

[0124] According to an embodiment of the present invention, the method of using three preset branches to perform parallel decoding and prediction of the fused feature map to obtain an initial detection result specifically includes:

[0125] The fused feature map is input into three branches for processing: an instance segmentation branch, a semantic segmentation branch, and a regression branch.

[0126] The instance segmentation branch is used to detect and segment each independent PIN pin instance, and the PIN pin bounding box and PIN pin instance segmentation mask are output.

[0127] The semantic segmentation branch is used to output a pixel-level classification map of the same resolution, in which pixels are classified into background, pin body, pin root and defect area.

[0128] The regression branch is used to predict a parametric map related to geometry, which includes a height regression map, a key point heatmap, and an orientation map.

[0129] It should be noted that, in this embodiment, a multi-task learning architecture is adopted, which includes three main decoding branches: Instance segmentation branch: This branch is responsible for detecting and segmenting each individual pin instance through an architecture similar to Mask R-CNN, and outputting its bounding box and instance segmentation mask. This is crucial for distinguishing densely packed pins and locating the precise position of each pin; Semantic segmentation branch: This branch outputs a pixel-level classification map with the same resolution as the input. Each pixel is classified as: background, pin body, pin root, and defect region (which can be further subdivided into scratches, burrs, collapses, etc.). This provides fine-grained defect localization and category information; Regression branch: This is a key innovative branch, responsible for outputting a series of quantization parameter maps.

[0130] According to an embodiment of the present invention, the height map is optimized and denoised to obtain the height regression map. The key point heat map is obtained by predicting the position of the PIN head vertex and the PIN root center point based on the PIN body pixel set and the PIN root pixel set. The orientation map is obtained by predicting the normal direction of the surface of each PIN.

[0131] It should be noted that, in this embodiment, the height regression map : Regarding the height map Optimization and denoising are performed; key point heatmap: predicts the position of key points such as the apex of the pin head and the center of the root; orientation map: predicts the normal direction of the surface of each pin, used to determine bending. In the decoding process, an improved feature pyramid network and cross-level attention mechanism are used to fuse deep high semantic features with shallow high resolution features to ensure that the network has good detection capabilities for defects of different scales (from micron-level scratches to bending of the entire pin).

[0132] According to an embodiment of the present invention, the step of performing result correlation, quantitative calculation, and comprehensive decision-making based on the initial detection result to output the target detection result specifically includes:

[0133] Each PIN pin instance output by the instance segmentation branch is correlated with the processing results of the semantic segmentation branch and the regression branch;

[0134] Calculate pin dimensions, pin position, pin shape parameters, and defect statistics to complete the quantitative calculation;

[0135] The calculated quantitative parameters are combined with preset process specifications to make a comprehensive decision to output the target detection result, which includes a structured report, visualization instructions and IO interface control signals.

[0136] It should be noted that, in this embodiment, each PIN pin instance output by the instance segmentation branch is associated with the results of the semantic segmentation and regression branches. For each PIN pin... A series of quantization parameters can be calculated: Size: Length, diameter, head width, etc. are calculated from the instance mask and optimized height information; Positionality: The coplanarity of all PIN head vertices is calculated (defined as the difference between the maximum and minimum Z coordinates of all vertices). ), and the spacing between adjacent pins; shape parameters: from the key points and orientation information obtained from regression, the curvature and twist can be calculated, and from the curvature diagram Analysis of the radius of curvature of the head sphere Defect Area / Severity: From the semantic segmentation results, the number and distribution of various defective pixels are statistically analyzed. The calculated parameters are compared with preset process specifications to comprehensively determine whether the PIN is a good product. Different alarm levels can be set; for example, severe defects (such as breakage or severe bending) are directly rejected, while minor defects (such as tiny scratches) are recorded but not rejected. (See attached...) Figure 2 As shown, the visualization interface will highlight all detected defects and indicate the defect type, location and severity with different colors and labels. At the same time, it will generate a detailed inspection report and trigger the sorting mechanism through the IO interface.

[0137] Figure 3 A block diagram of a precision connector PIN pin full-element visual inspection system according to the present invention is shown.

[0138] like Figure 3 As shown, this invention discloses a full-element visual inspection system for precision connector PIN pins, including a memory and a processor. The memory includes a program for a full-element visual inspection method for precision connector PIN pins. When the processor executes the program for the full-element visual inspection method for precision connector PIN pins, it performs the following steps:

[0139] The target data is obtained by acquiring and processing camera images. The target data includes a backlight correction map, a texture correction map, a height map, a magnitude map, and a curvature map.

[0140] Based on the target data, a multimodal adaptive fusion is performed using a preset encoder to obtain a fused feature map;

[0141] The initial detection result is obtained by parallel decoding and prediction of the fused feature map using three preset branches;

[0142] Based on the initial detection results, result correlation, quantitative calculation and comprehensive decision-making are performed to output the target detection results.

[0143] It should be noted that when the precision connector PIN pin full-element visual inspection system disclosed in this application is applied, the specific process corresponds to the precision connector PIN pin full-element visual inspection method described in the above embodiments. Since the specific implementation details of the system application are consistent with the content of the above precision connector PIN pin full-element visual inspection method, no further details will be provided in this embodiment.

[0144] A third aspect of the present invention provides a computer-readable storage medium including a method program for full-element visual inspection of precision connector PIN pins. When executed by a processor, the method program implements the steps of a method for full-element visual inspection of precision connector PIN pins as described in any of the preceding claims.

[0145] A fourth aspect of the present invention provides a full-element visual inspection device for precision connector pins, comprising:

[0146] Optical imaging module and detection module, wherein,

[0147] The optical imaging module includes an industrial camera, a dual telecentric lens, and a light source. The industrial camera and the dual telecentric lens are used to acquire images, and the light source is used to provide illumination or grating stripes.

[0148] The detection module includes an industrial control computer, used to implement the steps of a full-element visual inspection method for precision connector PIN pins as described in any of the above.

[0149] It should be noted that, in this embodiment, as Figure 4 The diagram shows an application schematic of a full-element visual inspection device for precision connector PIN pins. In this application, a stage is also required. The optical imaging module includes an industrial camera, dual telecentric lenses, and a light source. The industrial camera and dual telecentric lenses are used to acquire images, and the light source is used to provide illumination or grating fringes. The industrial camera includes a high frame rate camera, employing a high frame rate global shutter CMOS sensor to ensure that the image is not blurred under high-speed motion or flash illumination. The telecentric lens eliminates perspective errors, ensuring the accuracy of dimensional measurements at different depths of field.

[0150] Furthermore, in this embodiment, the light source includes a high-uniformity backlight source, a multi-angle composite ring light source, and a micro-structured light projection module (corresponding to...). Figure 4The DLP projector in the image contains a high-uniformity backlight source used to generate high-contrast PIN silhouettes, providing a benchmark for macroscopic size and position measurements; a multi-angle composite ring light source used to provide uniform surface illumination or illumination at specific angles, wherein its internal LEDs can be independently controlled in zones, providing both uniform surface illumination for observing color and texture, and illumination at specific angles (such as low angles) to enhance the contrast of surface scratches, pits and other defects; and a micro structured light projection module used to project high-precision digital grating stripes onto the PIN surface.

[0151] Specifically, in this embodiment, the backlight source is turned on, and the camera captures a silhouette image with extremely high binarization quality. This image is used for subsequent coarse positioning of the pins and generation of the projection area mask. The DLP projection module is activated, and a series of phase-shifted sinusoidal grating patterns are projected onto the pin array. The camera simultaneously acquires a set (at least three) of deformed grating images. By combining the projection area mask generated from the backlight image, background interference can be eliminated, and only the PIN area can be reconstructed in 3D. Using the phase-shifting method, the wrapping phase of each pixel can be calculated. :

[0152] ;

[0153] Subsequently, the absolute phase is obtained using either a time-phase unwrapping algorithm (taking Gray code as an example) or a spatial-phase unwrapping algorithm. Finally, using the pre-calibrated phase-height mapping relationship, the three-dimensional point cloud of the PIN needle surface is reconstructed. This allows us to obtain a height map. Each pixel value represents the relative height of that point; a high-resolution surface texture image is acquired under uniform ring illumination. This is used to detect surface defects such as color, plating, and scratches. Thus, three core data sources for the same PIN array were obtained: backlit PIN silhouette images. Initial 3D height map And 2D texture map of PIN pin surface texture map as camera image .

[0154] This invention discloses a method, system, medium, and device for full-element visual inspection of precision connector PIN pins. While ensuring sub-pixel level measurement accuracy, it significantly improves computational efficiency, exhibits strong robustness to complex contours and noise, avoids global optimization calculations by greatly narrowing the search range through coarse positioning, and ensures stable convergence to the global optimal solution through an adaptive gradient descent algorithm, thus achieving the best balance between efficiency and accuracy, stability and adaptability.

[0155] In the several embodiments provided by this invention, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0156] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0157] In addition, in the various embodiments of the present invention, each functional unit can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0158] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0159] Alternatively, if the integrated units of this invention are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

Claims

1. A method for full-element visual inspection of precision connector pins, characterized in that, Includes the following steps: The target data is obtained by acquiring and processing camera images. The target data includes a backlight correction map, a texture correction map, a height map, a magnitude map, and a curvature map. Based on the target data, a multimodal adaptive fusion is performed using a preset encoder to obtain a fused feature map, specifically including: Based on the encoder, feature pyramids of different scales are obtained by extracting features from the target data. Global average pooling is performed on the feature map of each channel in the feature pyramid to obtain the channel descriptor vector; Based on the channel descriptor vector, a fusion weight is generated for each feature map using a preset attention network; Based on the fusion weights and the corresponding feature maps, a weighted fusion is performed to obtain a fused feature map, calculated as follows: ； in, To fuse feature maps, The number of pyramid levels. For the first The fusion weights corresponding to each channel For the first Feature maps corresponding to each channel; The fused feature map is decoded and predicted in parallel using three preset branches to obtain the initial detection result. The three branches include an instance segmentation branch, a semantic segmentation branch, and a regression branch. Based on the initial detection results, result correlation, quantitative calculation and comprehensive decision-making are performed to output the target detection results.

2. The method for full-element visual inspection of precision connector PIN pins according to claim 1, characterized in that, The process of acquiring camera images and processing them to obtain target data specifically includes: The captured backlight PIN silhouette image, multi-frame structured light deformation image, and PIN surface texture image are used as camera images; The camera image is subjected to image correction to obtain a corrected image, which includes the backlight correction image and the texture correction image; A mask is generated based on the corrected image, and an initial three-dimensional height map is obtained by combining the generated binary mask with the corrected image for three-dimensional reconstruction. The height map, the magnitude map, and the curvature map are calculated based on the initial three-dimensional height map; Five-channel target data is obtained based on the backlight correction map, the texture correction map, the height map, the amplitude map, and the curvature map.

3. The method for full-element visual inspection of precision connector PIN pins according to claim 1, characterized in that, The method of using three preset branches to perform parallel decoding and prediction of the fused feature map to obtain the initial detection result specifically includes: The fused feature map is input into three branches for processing, wherein... The instance segmentation branch is used to detect and segment each independent PIN pin instance, and the PIN pin bounding box and PIN pin instance segmentation mask are output. The semantic segmentation branch is used to output a pixel-level classification map of the same resolution, in which pixels are classified into background, pin body, pin root and defect area. The regression branch is used to predict a parametric map related to geometry, which includes a height regression map, a key point heatmap, and an orientation map.

4. The method for full-element visual inspection of precision connector PIN pins according to claim 3, characterized in that, The height map is optimized and denoised to obtain the height regression map. The key point heat map is obtained by predicting the position of the PIN head vertex and the PIN root center point based on the PIN body pixel set and the PIN root pixel set. The orientation map is obtained by predicting the normal direction of the surface of each PIN.

5. The method for full-element visual inspection of precision connector PIN pins according to claim 4, characterized in that, The process of performing result correlation, quantification calculation, and comprehensive decision-making based on the initial detection results to output target detection results specifically includes: Each PIN pin instance output by the instance segmentation branch is correlated with the processing results of the semantic segmentation branch and the regression branch; Calculate pin dimensions, pin position, pin shape parameters, and defect statistics to complete the quantitative calculation; The calculated quantitative parameters are combined with preset process specifications to make a comprehensive decision to output the target detection result, which includes a structured report, visualization instructions and IO interface control signals.

6. A full-element visual inspection system for precision connector pins, characterized in that, The system includes a memory and a processor. The memory contains a program for a full-element visual inspection method for precision connector PIN pins. When the processor executes the program for the full-element visual inspection method for precision connector PIN pins, it performs the following steps: The target data is obtained by acquiring and processing camera images. The target data includes a backlight correction map, a texture correction map, a height map, a magnitude map, and a curvature map. Based on the target data, a multimodal adaptive fusion is performed using a preset encoder to obtain a fused feature map, specifically including: Based on the encoder, feature pyramids of different scales are obtained by extracting features from the target data. Global average pooling is performed on the feature map of each channel in the feature pyramid to obtain the channel descriptor vector; Based on the channel descriptor vector, a fusion weight is generated for each feature map using a preset attention network; Based on the fusion weights and the corresponding feature maps, a weighted fusion is performed to obtain a fused feature map, calculated as follows: ； in, To fuse feature maps, The number of pyramid levels. For the first The fusion weights corresponding to each channel For the first Feature maps corresponding to each channel; The fused feature map is decoded and predicted in parallel using three preset branches to obtain the initial detection result. The three branches include an instance segmentation branch, a semantic segmentation branch, and a regression branch. Based on the initial detection results, result correlation, quantitative calculation and comprehensive decision-making are performed to output the target detection results.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a method program for full-element visual inspection of precision connector PIN pins. When the method program is executed by a processor, it implements the steps of a method for full-element visual inspection of precision connector PIN pins as described in any one of claims 1 to 5.

8. A full-element visual inspection device for precision connector pins, characterized in that, include: Optical imaging module and detection module, wherein, The optical imaging module includes an industrial camera, a dual telecentric lens, and a light source. The industrial camera and the dual telecentric lens are used to acquire images, and the light source is used to provide illumination or grating stripes. The detection module includes an industrial control computer, used to implement the steps of the full-element visual inspection method for precision connector PIN pins as described in any one of claims 1 to 5.

9. A full-element visual inspection device for precision connector PIN pins according to claim 8, characterized in that, The industrial camera includes a high frame rate camera, and the light source includes a high uniformity backlight source, a multi-angle composite ring light source, and a micro structured light projection module. The high uniformity backlight source is used to generate a high-contrast PIN silhouette, the multi-angle composite ring light source is used to provide uniform surface illumination or low-angle illumination, and the micro structured light projection module is used to project high-precision digital grating stripes onto the PIN surface.

Images