Method and device for online detection and sorting of polyhedral electronic components based on machine vision

Abstract

This invention relates to the field of machine vision and electronic component inspection equipment technology, and discloses a method and equipment for online inspection and sorting of the posture and appearance of polyhedral electronic components based on machine vision. The method includes: product image acquisition; image preprocessing; AI segmentation model inference; defect region annotation; connected component analysis and suspicious region extraction; ROI candidate region extraction; ROI region image enhancement and filtering; edge detection and defect contour extraction; defect contour geometric feature analysis; and pass / fail determination and sorting control. The method and equipment provided by this invention, through the cooperation of hardware and software, can be adapted to various types of polyhedral electronic components such as metal-clad inductors and optocouplers / solid-state relays. It is adaptable to a wide variety of defects, has a fast detection speed, and a high defect recognition rate. It has advantages such as no blind spots across the entire surface, high automation, low computational resource consumption, and strong adaptability. It can be directly deployed on production lines, significantly improving inspection efficiency and quality, and reducing labor costs.

Description

Technical Field

[0001] This invention relates to the field of machine vision and inspection equipment technology, specifically to a method and equipment for online multi-pose detection and sorting of polyhedral components based on machine vision. It can realize online high-speed detection and automatic sorting of polyhedral components and can be adapted to a variety of electronic components. Background Technology

[0002] With the rapid development of high-end electronic systems such as AI servers and data centers that require high current, high density, low interference, and compact space, low-inductance power inductors with relatively complex structures have been widely used, especially integrally molded low-inductance power inductors (see Appendix). Figure 12 This type of inductor features a fully shielded structure with a snap-fit ​​or one-piece metal casing. Multiple inductors are integrated into the same package using a snap-fit ​​metal casing. A metal shielding cover is also provided outside the primary and secondary coils to control the magnetic field distribution and enhance overall integrity. Due to its strong electromagnetic interference resistance, low DC resistance, high current capacity, and low core loss, this low-power inductor meets the requirements of data center power supplies for small size, high current capacity, and electromagnetic interference resistance. To improve its electromagnetic shielding performance, an auxiliary shielding / heat dissipation coating is applied to the outer surface of the metal casing. When testing this type of product, a comprehensive inspection of its six-sided appearance, three-dimensional posture, and tilt is required. Simultaneous inspection of its posture (center of gravity) stability, surface defects, deformation, damage points, unevenness, edge deformation, and abnormal chamfer angles is also necessary to ensure product quality and performance. Electronic components with similar complex structural features also include: high-power inductors, low-inductance power capacitors, QFP / TQFP / LQFP, BGA / FBGA / CBGA, QFN / DFN, SOP / TSSOP / SSOP, PLCC / CLCC, passive components (rectangular chip type such as 0402~1210), optocouplers / solid-state relays, and other electronic components with hexahedral structures where all six surfaces need to be tested.

[0003] Comprehensive quality inspection of the aforementioned polyhedral electronic components (including appearance, center of gravity, and orientation) is a crucial step in the production and assembly of electronic components, directly impacting the accuracy of subsequent processes and component quality. Currently, the inspection of such electronic components in the market mainly relies on manual or semi-manual inspection, which has several drawbacks: First, the inspection speed is slow and inefficient, only one component and one type of defect can be inspected at a time, failing to meet the online inspection needs of large-scale production; second, electronic components have poor adaptability, requiring manual adjustment of various mechanical parts when changing inspection components, making automatic switching impossible, cumbersome, and time-consuming; third, it is highly dependent on manual labor, requiring manual loading of electronic components onto inspection fixtures before inspection can be performed, increasing labor costs and making it susceptible to human error affecting inspection accuracy; fourth, the automation level of defective product removal is low, as unqualified products cannot be automatically separated, requiring manual sorting, further reducing production efficiency and failing to meet the comprehensive quality inspection and rapid switching of various polyhedral electronic components during high-speed production lines, thus failing to meet the product quality inspection needs of diverse continuous production scenarios. Summary of the Invention

[0004] The purpose of this invention is to overcome the above-mentioned defects of the prior art and provide a machine vision-based online multi-pose detection and sorting method and equipment for polyhedral components. Through close cooperation between hardware and software, it can realize online comprehensive quality detection and automatic sorting of the six-sided appearance, three-dimensional pose and tilt of polyhedral components, improve detection efficiency, detection quality and component compatibility, and increase detection processing speed and reduce system resource requirements.

[0005] To achieve the above objectives, the present invention provides the following technical solution: A machine vision-based online detection and sorting method for the posture and appearance of polyhedral electronic components, characterized by the following steps: S1, Product Image Acquisition Acquire image data of all surfaces of the polyhedral electronic component product to be inspected, and ensure that the acquired images contain a 10%-15% overlap area to guarantee no blind spots in the inspection; S2, Image Preprocessing The original image is grayscaled, denoised, and standardized to eliminate metallic reflections and noise interference, improve image contrast, and enable the data to be used for subsequent rapid defect detection. S3, AI segmentation model inference The preprocessed grayscale image is copied twice along the channel dimension to generate a pseudo-RGB image, which is then input into a pre-trained AI segmentation model for inference operations. A pre-trained Mask R-CNN model based on the ResNet-50 backbone network is constructed as an AI segmentation model to perform forward inference and accurately segment various types of product defects (improper posture, surface scratches, dents, edge deformation, abnormal chamfer angles, uneven metal fasteners). S4, Defect Area Marking After the AI ​​segmentation model is processed, it outputs a binary mask image of the defect area with the same size as the preprocessed image (640×480 pixels). In the mask image, the area with a pixel value marked as 1 corresponds to the suspected defect area, and the area with a pixel value marked as 0 corresponds to the background area. S5. Connectivity Analysis and Suspicious Region Extraction An 8-neighborhood connected component analysis algorithm is used to extract connected components from the mask image output in step 4; then small region filtering is performed, and then suspicious regions are marked as the basis for subsequent ROI extraction. S6. ROI Candidate Region Extraction Defect area images are extracted from the original image through coordinate mapping and ROI cropping. S7, ROI Region Image Enhancement and Filtering Targeted enhancements are implemented to address the issue of blurred edges in defective product areas (such as deformed fastening edges and abnormal chamfers), including contrast enhancement and noise reduction in the ROI area. S8. Edge Detection and Defect Contour Extraction Defect contours are extracted through edge detection and morphological optimization; S9. Defect Contour Geometric Feature Analysis Extract key geometric feature parameters of product defects, including basic geometric features and targeted features, and integrate these features to accurately determine whether the defects exceed the standards. S10. Conformity Assessment and Sorting Control Based on a preset threshold, the product is judged to be qualified, and then qualified and unqualified products are further sorted.

[0006] 10. A machine vision-based online inspection and sorting device for polyhedral electronic components, characterized in that it implements the machine vision-based online inspection and sorting method for the posture and appearance of polyhedral electronic components as described in any one of claims 1 to 9, comprising: a control unit, a continuous feeding unit, a transfer and rotation unit, a machine vision unit, and a sorting and discharging unit. The control unit is equipped with a control program that implements steps 1 to 10, including a pre-trained AI segmentation model and an SQLite database, for running image preprocessing, AI defect detection, feature analysis and result determination algorithms. The continuous feeding unit includes a continuous conveyor belt, a material handling mechanism, and a discharge conveyor, used to continuously transport the products to be tested to the feeding station; The transfer and rotation unit includes a PPU picking and placing mechanism and a rotation mechanism, which are used to pick up the product to be inspected from the feeding station, first transport it to the front inspection station, collect image information for the first time, and then transport it to the rear inspection station for a second image information collection. The machine vision unit includes four industrial cameras, two of which are horizontal acquisition cameras and the other two are oblique upward acquisition cameras, used to acquire image information of the entire surface of the product to be inspected at the inspection station. The sorting and unloading unit includes a sorting and unloading cylinder and an NG box, which are used to sort out non-conforming products from the conveyor channel and collect them into the NG box, while qualified products are transported to the next processing station by the conveyor channel.

[0007] Compared with the prior art, the present invention has at least the following beneficial effects: 1. Full-surface blind-zone-free detection improves the adaptability of polyhedral components. This system employs a combination of four cameras for two-step data acquisition and a rotary station cylinder-assisted PPU transport scheme to achieve complete coverage of the six surfaces of a polyhedral inductor. The images include a 10%-15% overlap area to ensure no blind spots and can extract information such as its three-dimensional orientation, center of gravity, and stability. It breaks through the field of view limitations of traditional single-camera inspection, accurately adapts to the inspection needs of various polyhedral electronic components, and avoids problems such as missed inspections due to surface omissions, inability to fully determine the product's three-dimensional spatial coordinates, and incorrect orientation. This significantly improves the versatility and comprehensiveness of inspection for various types of polyhedral components.

[0008] 2. Customized optimization of AI models to improve computing speed and reduce resource consumption. A Mask R-CNN model was customized to address the defect characteristics of polyhedral components. ResNet50-FPN was used for pruning, and an optimization strategy of backend deployment + INT8 quantization was employed to simplify redundant calculations, significantly improving the model's inference speed. The average detection time per component during continuous operation can be controlled within 0.5 seconds, meeting the high-speed continuous production requirements of electronic product production lines. Simultaneously, it reduces the GPU memory usage of the control unit (industrial PC), enabling stable operation without high-end hardware configuration and lowering system deployment costs.

[0009] 3. Software and hardware collaborative automation significantly improves detection and sorting efficiency. The PPU loading and unloading mechanism, gripper positioning, rotary cylinder, and detection and sorting modules are linked and controlled by the built-in software system of the control unit to achieve full automation of the "feeding-detection-sorting-reset" process. This can completely replace traditional manual detection, improving efficiency by more than 15 times. The single-piece sorting response time is ≤200ms, avoiding delays caused by manual operation, adapting to the pace of high-speed production lines, and eliminating production bottlenecks.

[0010] 4. Accurate multi-dimensional feature determination improves detection quality and reliability. By integrating basic geometric features (area, roundness, etc.) with the specific features of polyhedral components (edge ​​deformation, chamfer angle, etc.) and combining them with preset thresholds, multi-dimensional defect judgment can be achieved. The defect recognition rate can reach over 99.5%, with a false positive rate of 0.25% and a false negative rate of 0.2%, which is far superior to the accuracy of manual inspection. It can accurately distinguish minute defects such as tiny scratches and abnormal chamfers, ensuring the stability of product quality.

[0011] 5. Modular architecture design enhances system scalability and maintainability. At the hardware level, the system is modularly divided into "image acquisition, transfer and positioning, data processing and sorting". At the software level, it is layered into "preprocessing, AI model, defect analysis and judgment control". The interfaces between modules are standardized. It can be quickly adapted to the detection of other types of polyhedral components by replacing camera parameters and updating model files without the need for overall system modification. When a module fails, it can be repaired individually, reducing maintenance costs and downtime losses.

[0012] 6. End-to-end data traceability enhances production control capabilities. The control unit is equipped with an SQLite database, which records information such as product number, defect type, and inspection time in real time, and generates traceable inspection logs to facilitate production batch quality traceability and problem review, and help optimize production processes; it can also statistically analyze defect distribution patterns to provide a basis for improvement in upstream production links and enhance the overall production chain management level. Attached Figure Description

[0013] Figure 1 This is a schematic diagram of the overall process of the online detection and sorting method for the posture and appearance of polyhedral electronic components based on machine vision according to an embodiment of the present invention. Figure 2 This is a flowchart illustrating step 1 of the online detection and sorting method for the posture and appearance of polyhedral electronic components based on machine vision, according to an embodiment of the present invention. Figure 3 This is a schematic diagram of the overall structure of the online detection and sorting equipment for polyhedral electronic components based on machine vision, according to an embodiment of the present invention. Figure 4 This is a schematic diagram of the overall external structure of the machine vision-based online detection and sorting equipment for polyhedral electronic components according to an embodiment of the present invention. Figure 5 This is a schematic diagram of the overall external structure of the online detection and sorting equipment for polyhedral electronic components based on machine vision according to an embodiment of the present invention, taken from a second perspective. Figure 6 This is a schematic diagram of the internal three-dimensional structure of the machine vision-based online detection and sorting device for polyhedral electronic components after removing the casing, according to an embodiment of the present invention. Figure 7 This is a schematic diagram of the internal three-dimensional structure of the machine vision-based online detection and sorting device for polyhedral electronic components after removing the casing, according to a second-view perspective of an embodiment of the present invention. Figure 8 This is a schematic diagram of the internal three-dimensional structure of the machine vision-based online detection and sorting device for polyhedral electronic components after removing the casing, according to an embodiment of the present invention. Figure 9 This is a schematic diagram of the internal three-dimensional structure of the machine vision-based online detection and sorting device for polyhedral electronic components after removing the casing, according to an embodiment of the present invention. Figure 10 This is an embodiment of the present invention. Figure 8 A magnified schematic diagram of the structure at point A in the diagram; Figure 11 This is a schematic diagram of the three-dimensional shape of the fixed support bracket and prism in the machine vision-based online detection and sorting equipment for polyhedral electronic components according to an embodiment of the present invention. Figure 12 This is a schematic diagram of the overall external structure of two polyhedral electronic components in an embodiment of the present invention; Figure 13 This is a schematic diagram of a partial image of the product surface obtained after coarse positioning processing of the model in the online detection and sorting method for the posture and appearance of polyhedral electronic components based on machine vision in an embodiment of the present invention. Figure 14 This is a schematic diagram of a partial image of the product surface obtained after precise positioning processing by a model in the online detection and sorting method for the posture and appearance of polyhedral electronic components based on machine vision, as described in an embodiment of the present invention.

[0014] In the picture: 1. Shell; 11. Base; 2. Continuous feeding station; 21. Feed conveyor; 211. Removal station; 212. Feed blocking block; 22. Discharge conveyor; 221. Discharge station; 3. Continuous material handling station; 31. PPU module; 32. Connecting plate; 33. Crossbeam; 4. Gripper module; 41. Moving gripper No. 1; 42. Moving gripper No. 2; 43. Moving gripper No. 3; 44. Moving gripper No. 4; 45. Moving gripper No. 5; 46. Gripper rotary cylinder; 5. Rotary station; 51. Rotary cylinder; 6. Inspection station; 61. Front inspection station; 62. Rear inspection station; 63. Fixed support bracket; 64. Slot; 65. Receiving slot; 66. Prism; 7. Machine vision acquisition module; 71. Top-mounted oblique area array camera module; 72. Horizontal area array camera module; 73. Focusing motor; 8. Discharge station; 81. NG material box; 82. Discharge cylinder; 83. Push plate; 84. Push rod; 9. Industrial control computer; 10. Electronic components; 101. Metal casing; 102. Auxiliary shielding / heat dissipation coating; 103. Metal shielding cover. Detailed Implementation

[0015] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0016] Example 1 See appendix Figure 1-14 The machine vision-based online inspection and sorting method and equipment for the posture and appearance of multifaceted electronic components provided in this embodiment is designed for comprehensive quality inspection of low-inductance power inductors such as the HCB1175 series. The HCB1175 series power inductors are high-performance power inductors with a closed magnetic circuit design, magnetic shielding, low EMI, and are suitable for high-density power supply layouts. They can meet the demanding high-current, high-density power supply requirements of modern electronic equipment. See attached figure for their external structure. Figure 12 Dimensions: 11.0 mm × 7.2 mm × 7.5 mm (length × width × height), mainly used in server / data center power conversion, base station power supply, DC-DC converter, etc.

[0017] This embodiment specifically includes the following: I. Main configuration of equipment hardware 1.1 Hardware Configuration Machine vision unit: 4 Basler acA2500-14gm 5-megapixel CMOS industrial cameras, 14fps frame rate, 2.2μm×2.2μm pixel size, equipped with 0.3x telecentric lens (ensuring an imaging resolution of 0.01mm / pixel, covering the minimum defect detection requirements of HCB1175). Lighting system: 4 sets of high-angle ring LED light sources (wavelength 500-550nm), 2 sets each for the front and rear inspection stations, and the brightness of the light sources is adjustable (the brightness is set to 600-800cd / m² to avoid reflection interference, considering the reflective characteristics of the HCB1175 metal shell). Continuous feeding unit: Transfer rotation unit: pneumatically driven rotating stage (positioning accuracy ±0.5°), with anti-static rubber pads attached to the stage surface (to prevent damage to the HCB1175 electrode). Control unit (industrial computer): Intel Core i7-12700K processor (14 cores and 20 threads), NVIDIA RTX3070Ti GPU (8GB VRAM), 32GB DDR5 memory, 1TB SSD (for storing models, image data, and detection logs). Sorting and unloading unit: pneumatic grippers + single-path conveyor track, sorting response time ≤200ms, supports qualified products (continue to move forward along the original track), and unqualified products are automatically pushed out of the conveyor track by the discharge cylinder and diverted to the NG box.

[0018] 1.2 Software Configuration Operating system: Windows 10 Professional (64-bit); Development framework: Python 3.9, OpenCV 4.8.0 (image processing), PyTorch 2.0 (deep learning model); Model file: Mask R-CNN model optimized for HCB1175 defects (.pth format, file size approximately 280MB); Data management: SQLite database, recording the HCB1175 test results for each batch (product number, defect type, test time, equipment number).

[0019] II. Specific Implementation Process of Online Inspection and Sorting Method for Polyhedral Electronic Components (HCB1175 Series) Based on Machine Vision Step 1: Acquisition of six-sided images and three-dimensional spatial coordinates of the HCB1175 outline (two-step acquisition scheme using 4 cameras) Core objective: To achieve the acquisition of three-dimensional spatial coordinates of the HCB1175's six surfaces (top, bottom, left, right, front, and back) using a combination of front inspection station, rear inspection station, and 180° rotation, with the acquired images containing a 10%-15% overlap area to ensure no blind spots in the inspection.

[0020] 1-1 workstation layout: The distance between the front inspection workstation and the rear inspection workstation is 10cm. The rotating stage is located between the two workstations. The distance between the center of the stage and the optical axis of the cameras at both workstations is 280mm (camera working distance). Camera division of labor: The front inspection station is equipped with "horizontal camera 1" and "upper camera 1" at an angle, and the rear inspection station is equipped with "horizontal camera 2" and "upper camera 2" at an angle; the optical axis of the horizontal camera is flush with the center of HCB1175 (5mm in height), and the optical axis of the upper camera is tilted at 45 degrees to the surface of the stage (covering the upper surface and side surface). Step 1-3 Data Acquisition (Front Inspection Station): The HCB1175 is fed into the front inspection station by the conveyor rail. The platform positioning mechanism corrects the product position (positioning accuracy ±0.1mm). Horizontal camera 1 captures the "rear surface and lower surface", and upper camera 1 captures the "upper surface and left side surface". The four surfaces are acquired simultaneously (exposure time 200μs to avoid motion blur) to obtain an image size of 2592×1944 pixels. 1-4 Product Rotation: The product is rotated and its posture is adjusted at the rotation station, and the gripper 2 places the product into the test station; Step 2 (post-inspection station): Horizontal camera 2 captures images of the "lower surface and front surface", and vertical camera 2 captures images of the "upper surface and right side surface", simultaneously acquiring images of four surfaces; during the acquisition process, there is a 10%-15% overlap between the images of horizontal camera 2 and horizontal camera 1, and between vertical camera 2 and vertical camera 1 (used for subsequent image stitching calibration to ensure complete coverage of all 6 surfaces). Image storage for 1-6: The original images of the 6 surfaces are named according to "product number-workstation-surface type" (e.g., HCB1175-001-front workstation-upper surface.jpg) and stored in real time on the industrial computer SSD, retaining the original EXIF ​​information (shooting time, camera parameters).

[0021] Step 2: Preprocessing of raw images (optimized for HCB1175 metal surface characteristics) Core objective: To eliminate metallic reflections and noise interference, improve image contrast, and lay the foundation for subsequent defect detection.

[0022] 2-1 Grayscale Conversion: The color image is converted to a grayscale image using a weighted average method. The formula is G=0.299R+0.587G+0.114B (to match the light gray metal casing of HCB1175 and highlight the grayscale difference). 2-2 Noise Reduction Processing: A combination algorithm of "Gaussian filtering + median filtering" is adopted - first, high-frequency noise is smoothed by 3×3 Gaussian filtering (σ=1.2), and then salt-and-pepper noise generated by metallic reflection is eliminated by 2×2 median filtering; 2-3 Contrast Normalization: Adaptive Histogram Equalization (CLAHE) is used with clipLimit=2.0 and tileGridSize=(8,8) to stretch the grayscale values ​​to the range of 0-255, thereby enhancing the grayscale difference between defects (such as scratches and dents) and the background. 2-3 Image Scaling: The preprocessed image is scaled to 640×480 pixels (to reduce the computational load of subsequent models while preserving the details of HCB1175 defects). The scaling algorithm uses bilinear interpolation.

[0023] Step 3: Defect segmentation model based on Mask R-CNN (customized for HCB1175) 3-1 Core Architecture of the Model Mask R-CNN was selected as the core segmentation model, which is suitable for accurate segmentation of multiple types of defects in HCB1175 (surface scratches, dents, edge deformation, abnormal chamfer angles, and uneven metal fastening). The architecture is as follows: Backbone network: ResNet50-FPN (Feature Pyramid Network), used to extract image features at different scales (adapted to HCB1175 for multi-scale defects ranging from 0.03mm micro scratches to 1mm edge deformation). Region Proposal Network (RPN): Generates candidate defect regions with anchor points set to three scales (16×16, 32×32, 64×64) and three aspect ratios (1:1, 1:2, 2:1) to adapt to different morphological defects of HCB1175. Inspection head: includes a classification branch (outputs background + 6 types of defects, the 6 types of defects are: scratches, dents, edge deformation, chamfer abnormality, uneven snapping, and damage points), a bounding box regression branch (corrects the coordinates of the defect area), and a mask branch (outputs a 14×14 pixel defect mask). Post-processing module: Non-maximum suppression (NMS) is used to remove duplicate candidate boxes (IOU threshold = 0.3), and mask thresholding is used for segmentation (threshold = 0.5, and areas greater than the threshold are judged as defective areas).

[0024] 3-2 Model Training Process (for HCB1175 dataset) 3-2-1 Dataset Construction: (1) Sample collection: Collect 400 images of qualified products and 600 images of unqualified products of HCB1175 series (including 6 types of defects, 100 images of each type of defect), covering different production batches and different lighting conditions; (2) Labeling: The LabelMe tool is used to label the defect area and generate a JSON format label file, which includes the defect type, bounding box coordinates, and mask area; (3) Data augmentation: Randomly flip (horizontal / vertical), rotate (-10°~+10°), scale (0.8~1.2 times), adjust brightness (±15%), and add Gaussian noise (σ=0.02) to the training set to expand the sample size to 15,000 images; (4) Data set partitioning: The training set (12,000 images), validation set (1,500 images), and test set (1,500 images) are divided in a ratio of 8:1:1.

[0025] 3-2-2 Training environment: GPU is NVIDIA RTX 3070Ti, batch size = 8, input image size 640×640; 3-2-3 Training parameter settings: (1) Optimizer: AdamW, initial learning rate = 1e-4, weight decay = 0.0001; (2) Learning rate scheduling: Cosine annealing strategy is adopted, T_max=100, and the learning rate decays by 0.1 every 10 epochs; (3) Loss function: Multi-task loss = classification loss (cross-entropy loss) + bounding box regression loss (smoothing L1 loss) + mask loss (binary cross-entropy loss), with weight ratios of 1:1:2; (4) Training iteration: A total of 100 epochs were trained, and the model was evaluated on the validation set every 5 epochs. The model with the highest accuracy on the validation set was saved (the best model validation set mAP=0.968).

[0026] 3-2-4 Model Optimization: Employing INT8 quantization technology reduces computational power consumption and improves inference speed, achieving a 2.8x improvement in real-time performance. Through Quantization Awareness Training (QAT) strategy, accuracy loss is minimized (≤0.5%), meeting detection accuracy requirements.

[0027] 3-3 AI Segmentation Model Operation Process (Forward Inference) 3-3-1 Input: Six surface images of HCB1175 after preprocessing in step 2 (640×480 pixels, single-channel grayscale). 3-3-2 Feature Extraction: The image is processed by the ResNet50-FPN backbone network, which outputs feature maps at four scales: C2 (160×120), C3 (80×60), C4 (40×30), and C5 (20×15). 3-3-3 Candidate Region Generation: The RPN network performs sliding window detection on feature maps at four scales to generate 2000 candidate defect regions; 3-3-4 ROI Align: Aligns the candidate regions by features and outputs a 7×7×256 feature vector; 3-3-5 Multi-task reasoning: The detection head outputs the defect type probability, corrected bounding box coordinates, and 14×14 defect mask for each candidate region; 3-3-6 Post-processing: NMS removes duplicate candidate boxes, mask thresholding (0.5) is applied, and the final defect segmentation result is output.

[0028] Step 4: Output the defect mask image The model outputs a binary mask image with the same dimensions as the preprocessed image (640×480 pixels): regions with a pixel value of 1 in the mask image correspond to suspected defect areas, and regions with a pixel value of 0 correspond to background areas; it also outputs a preliminary classification result for each suspected defect (such as "abnormal chamfer" or "uneven snap-fit"), which is consistent with the mask image. Figure 1 One-to-one storage (naming format: HCB1175-001-upper surface-mask.png).

[0029] Step 5: Connectivity Analysis and Suspicious Region Extraction 5-1 Connected Component Detection: The 8-neighborhood connected component analysis algorithm is used to extract connected components from the mask image output in step 4; 5-2 Small Area Filtering: Based on the HCB1175 defect detection standard (minimum detectable defect size 0.03mm), calculate the area of ​​the connected region (1 pixel corresponds to 0.01mm×0.01mm, i.e. 0.0001mm²), and filter out tiny connected regions with an area < 3 pixels (0.0003mm²) (to exclude noise interference). 5-3 Suspicious Region Marking: Retain connected components that meet the conditions, and record the coordinates (x1, y1, x2, y2) of the minimum bounding rectangle of each connected component as the basis for subsequent ROI extraction.

[0030] Step 6: ROI Candidate Region Extraction 6-1 Coordinate Mapping: Map the coordinates of the bounding rectangle of the connected domain obtained in step 5 back to the image (640×480 pixels) after preprocessing in step 2. 6-2 ROI Cropping: Based on the mapped coordinates, crop out the ROI image of each suspicious region (e.g., the ROI size of a certain beveled abnormal region is 64×48 pixels). 6-3 ROI Classification and Storage: Based on the preliminary defect classification results, store the ROIs of the same type of defect in the corresponding folder (e.g., "Chamfer Anomaly-ROI / ") for easy and accurate analysis later.

[0031] Step 7: Image enhancement and filtering of the ROI region Targeted enhancements were implemented to address the edge blurring issue in HCB1175 defect areas (such as deformation of the fastening edge and abnormal chamfering). 7-1 Contrast Enhancement: Adaptive Histogram Equalization with Limited Contrast (CLAHE) is used, clipLimit=1.5, tileGridSize=(4,4), to enhance the grayscale difference between defect edges and the background; 7-2 Fine noise reduction: Bilateral filtering (d=5, σ_color=75, σ_space=75) is used to further eliminate residual noise while preserving the details of the defect edges; 7-3 Image Normalization: Normalize the grayscale values ​​of the enhanced ROI image to 0-255 to ensure consistent brightness across different ROIs.

[0032] Step 8: Edge detection and defect contour extraction 8-1 Edge Detection: Using the Canny algorithm, based on the surface characteristics of HCB1175 metal, a low threshold of 50 and a high threshold of 150 are set to extract defect edges; 8-2 Morphological optimization: Perform "dilation-erosion" operation on the edge image (dilation kernel 3×3, erosion kernel 2×2) to fill the edge breaks and eliminate small burrs; 8-3 Contour Extraction: The findContours function (search mode = RETR_EXTERNAL, approximation method = CHAIN_APPROX_SIMPLE) is used to extract closed defect contours and record the coordinates of the contour points.

[0033] Step 9: Defect contour geometric feature analysis (for HCB1175 inspection indicators) Key geometric features of HCB1175 defects were extracted for accurate determination of whether the defects exceeded the standard. The core features and calculation methods are as follows: 9-1 Basic Geometric Features: (1) Defect area: The area inside the contour is calculated using the contourArea function (unit: mm², 1 pixel = 0.0001 mm²). (2) Aspect Ratio: Calculate the aspect ratio of the minimum bounding rectangle of the defect; (3) Roundness: The formula is C=4πA / L² (A is the defect area and L is the perimeter of the outline). The closer the roundness is to 1, the closer the defect is to a circle (e.g., the roundness of a dent defect is higher and the roundness of a scratch defect is lower).

[0034] (4) Attitude stability: In the vertical state, the three-dimensional coordinates of each contour line are within the allowable error range; 9-2 Targeted Features (HCB1175 Core Detection Items): (1) Edge deformation: The extracted edge profile is matched with the HCB1175 standard edge profile (pre-stored in the system) and the maximum offset distance (unit: mm) is calculated. (2) Chamfer angle: Perform straight line fitting on the chamfer profile (using the least squares method) and calculate the angle between the two fitted straight lines (unit: °); (3) Surface flatness: Extract the depth features of surface defects (such as depressions, uneven fastening) and calculate the gray value gradient. For every 10 increase in gray value difference, the corresponding depth increases by 0.01 mm (determined in advance through calibration experiments).

[0035] 9-3 Feature integration: Integrate the seven features of each defect (area, aspect ratio, roundness, edge deformation, chamfer angle, flatness, and three-dimensional posture) into a feature vector for final judgment.

[0036] Step 10: Qualification Judgment and Sorting Control 10-1 Preset Pass Threshold (for HCB1175 series standards): (1) Defect area: ≤0.01mm² (corresponding to 100 pixels); (2) Edge deformation: ≤0.02mm; (3) Chamfer angle: standard angle 45°, allowable deviation ±0.5°; (4) Surface flatness: Depth of depressions / protrusions ≤ 0.03 mm; (5) Damage points: are not allowed (damage points with an area > 0 are considered unqualified); (6) Posture: The allowable deviation of the three-dimensional contour line coordinates is ±0.5mm.

[0037] 10-2 Judgment Logic: If any feature of a defect exceeds the standard, or if there is a damage point, it is judged as a non-conforming product; if all defect features are within the threshold, it is judged as a conforming product. 10-3 Result Output: Store the judgment result (pass / fail), defect type, characteristic parameters, and product number into the SQLite database, and display them in real time on the industrial control computer interface; 10-4 Sorting Control: The industrial computer sends control signals to the sorting device via the RS485 interface. Qualified products trigger the green track pneumatic grippers for release, while unqualified products trigger the red track pneumatic grippers for sorting. After sorting is completed, the platform is reset and waits for the next HCB1175 product to enter the testing process.

[0038] III. Implementation Results Verification 3.1 Verification Conditions Inspection target: HCB1175-121 model inductors, batch size 500 pieces (including 400 qualified pieces and 100 unqualified pieces, covering 6 types of defects, with 15-20 pieces of each type of defect); Operating environment: production line speed 30 pieces / minute (single piece inspection time ≤ 2s), room temperature 25℃, light source brightness 700cd / m²; Evaluation metrics: defect identification rate, false positive rate, false negative rate, and detection speed.

[0039] 3.2 Verification Results 1. Inspection speed: The average inspection time per piece is 0.42s (including 0.15s for image acquisition, 0.08s for preprocessing, 0.12s for model inference, and 0.07s for subsequent analysis), which meets the requirements of a production line of 60 pieces / minute; 2. Recognition accuracy: (1) Defect identification rate: 99.6% (99 out of 100 non-conforming parts were accurately identified); (2) False judgment rate: 0.25% (1 out of 400 qualified pieces was falsely judged as unqualified); (3) Miss rate: 0.2% (1 out of 100 non-conforming items was not identified, which was a tiny scratch of 0.008 mm²). 3. Stability: After 2 hours of continuous operation (inspecting 720 items), the equipment operated without failure, the inspection accuracy remained stable, the database records were complete, and the sorting accuracy was 100%.

[0040] This invention, tailored to the metal casing fastening structure of the HCB1175 series, achieves blind-spot-free detection on all six sides using a four-camera, step-by-step acquisition scheme. The defect segmentation accuracy based on a customized Mask R-CNN model meets requirements, and the detection speed and stability are well-suited to production line needs. One minor scratch that was missed can be further optimized by fine-tuning the model's mask threshold (reducing it to 0.45). False positives were caused by abnormal edge grayscale, which can be addressed by increasing the number of labeled samples to improve model robustness. The intermediate results of the detected images and processing are as follows: Figure 13-14 As shown.

[0041] IV. Online detection and sorting equipment used in this embodiment The online detection and sorting equipment for polyhedral electronic components based on machine vision provided in this embodiment implements the online detection and sorting method for polyhedral electronic components based on machine vision. It includes: a control unit, a continuous feeding unit, a transfer and rotation unit, a machine vision unit, and a sorting and discharging unit. The control unit is equipped with a control program that implements steps 1 to 10, including a pre-trained AI segmentation model for running image preprocessing, AI defect detection, feature analysis, and result determination algorithms. The continuous feeding unit includes a continuous conveyor belt, a material handling mechanism, and a discharge conveyor, used to continuously transport the products to be tested to the feeding station; The transfer and rotation unit includes a PPU picking and placing mechanism and a rotation mechanism, which are used to pick up the product to be inspected from the feeding station, transport it to the front inspection station, and perform four-sided image acquisition. The machine vision unit includes four industrial cameras, two of which are horizontal acquisition cameras and the other two are oblique upward acquisition cameras, used to acquire image information of the entire surface of the product to be inspected at the inspection station. The sorting and unloading unit includes a sorting and unloading cylinder and an NG box, which are used to sort out non-conforming products from the conveyor channel and collect them into the NG box, while qualified products are transported to the next processing station by the conveyor channel.

[0042] For more details, see Figures 4-11 This invention provides an online inspection and sorting device for polyhedral electronic components based on machine vision, specifically applied to the online posture and appearance defect comprehensive quality inspection and sorting of polyhedral electronic components.

[0043] This equipment can be seamlessly connected to the front-end SMT placement equipment and the back-end AOI inspection equipment through the continuous feeding mechanism 2. Components output from the front-end production line are transported in batches to the picking mechanism 211 via the infeed conveyor 21. Then, the gripper mechanism 4 places the components onto the rotating station 5 of this equipment. The single-axis linkage PPU picking and placing mechanism 3 controls five sets of moving grippers to achieve component posture correction, multi-station synchronous inspection, and automatic sorting. Relying on the orthogonally arranged dual machine vision acquisition modules 7, the three-dimensional posture and ±0.02° tilt angle of the components are detected. Defective components are automatically separated at the discharge station 8, and good components are accurately placed into the discharge mechanism 221 after posture adjustment and flow to the next station along the discharge conveyor 22, forming a complete online inspection closed loop. This significantly reduces the posture defect rate during component assembly and improves the automation level of the production line. The PPU picking and placing mechanism refers to the picking and placing mechanism composed of precision pick-and-place units.

[0044] The online electronic component posture detection and sorting equipment based on machine vision includes a housing 1 and a base 11. The base 11 is equipped with a continuous feeding station 2, a continuous gripping station 3, a rotating station 5, a detection station 6, a discharge station 8, and an industrial control computer 9. The continuous feeding station 2 includes an infeed conveyor 21 and an outfeed conveyor 22. The infeed conveyor 21 has a pick-up station 211 at its end, and the outfeed conveyor 22 has an outfeed station 221 at its beginning, which is close to the discharge station 8. The pick-up station 211 is equipped with an infeed blocking block 212. The component to be tested flows into the pick-up station 211 along the infeed conveyor 21. The infeed blocking block 212 blocks the component from moving forward, so that the continuous gripping station 3 can pick up the component to be tested from the pick-up station 211 and enter the next station process. The continuous feeding station 2 is used to realize the automatic infeed and outfeed of components. At the same time, part of the continuous feeding station 2 is exposed outside the housing 1 to connect the front and rear workstations.

[0045] The continuous gripping station 3 includes a PPU module 31, a connecting plate 32, a crossbeam 33, and a gripper module 4. The continuous gripping station 3 adopts a single-axis linkage precision module, equipped with a gripper module 4 that can rotate and change direction. The connecting plate 32 is installed on the PPU module 31, and the crossbeam 33 is horizontally connected to the connecting plate 32. The gripper module 4 is set below the crossbeam 33. The crossbeam 33 is used to suspend the gripper module 4, and the connecting plate 32 is used to adjust the longitudinal position of the gripper module 4 to ensure alignment with the station. The gripper module 4 is connected to the PPU module 31 by transmission. The PPU module 31 controls the left and right movement of the gripper module 4, driving the gripper module 4 to move to the corresponding station to complete the parallel assembly line operation of gripping, rotating, and transferring components.

[0046] The gripper module 4 includes five movable grippers and two gripper rotary cylinders 46. Each movable gripper corresponds to a workstation below it and the gripping direction is vertically downward. The five movable grippers are arranged in a straight line in the horizontal direction. The first movable gripper 41 is fixedly installed at the front end of the crossbeam 33, and its lower part corresponds to the material handling station 211 at the end of the feeding conveyor 21. The second, third, and fourth movable grippers 44 are all fixedly installed in the middle of the crossbeam 33. The second movable gripper 42 corresponds to the rotary workstation 5 below it, and the third and fourth movable grippers 43 and 44 correspond to the inspection workstation below them. The fifth movable gripper 45 is fixedly installed at the rear end of the crossbeam 33. Below it is the discharge station 221 at the beginning of the discharge conveyor 22. The top of the third moving gripper 43 and the fifth moving gripper 45 are both integrated with gripper rotary cylinders 46. The gripper rotary cylinders 46 can provide rotational movement in the horizontal direction within a range of 0-180 degrees, so that the third moving gripper 43 and the fifth moving gripper 45 can drive the clamped components to adjust their angle. Each of the above moving grippers can independently complete the gripping and releasing actions and is adapted to the automatic clamping of components of different specifications. The gripper module 4 is equipped with a dual gripping and positioning synchronous operation function: when the first moving gripper 41 performs the loading, the second moving gripper 42 simultaneously performs the posture correction of the component to be tested at the rotary station 5.

[0047] The rotary station 5 is equipped with a rotary cylinder 51. The rotary station 5 is located between the material handling station 211 and the inspection station 6. The rotary cylinder 51 is used to drive the component to be inspected to rotate and adjust its direction so that the direction of the component to be inspected is aligned with the positioning requirements of the next inspection station 6. The intermediate angle conversion ensures the stability of the posture of the component to be inspected at the inspection station 6, so that the inspection station 6 can obtain a standard inspection angle.

[0048] Inspection station 6 includes a front inspection station 61, a rear inspection station 62, and a machine vision acquisition module 7. Each inspection station 6 has a transparent fixed support bracket 63. Each fixed support bracket 63 has a slot 64 on its upper surface for placing the component to be inspected. The slot 64 is used to eliminate placement misalignment of the component, adjusting it to a fixed position and stabilizing it to ensure the reliability of the subsequent inspection results from the machine vision acquisition module 7. A rectangular receiving slot 65 is located directly below the slot 64 on each fixed support bracket 63. This rectangular receiving slot 65 contains a right-angled equilateral triangular prism 66 for reflecting the bottom image of the component. Its inclined surface is a reflective mirror, facing the horizontal area array camera module 72, and reflects the bottom image of the component. The image is reflected into the acquisition area of ​​the horizontal area array camera module 72; after the continuous gripping station 3 places the component to be inspected into the inspection station 6, the corresponding machine vision acquisition module 7 is triggered synchronously. After the front inspection station 61 completes the first inspection of the component, the continuous gripping station 3 drives the third moving gripper 43 to pick up the component from the front inspection station 61. After the gripper rotating cylinder 46 integrated at the top performs a 180-degree horizontal rotation of the component, it is placed on the rear inspection station 62, so that the machine vision acquisition module 7 can capture the appearance image of the component from another angle. The two inspection stations 6 sequentially perform the inspection of the six appearance surfaces, posture and tilt of the component. The cooperation of the two inspection stations 6 makes the inspection results more comprehensive and accurate, while shortening the inspection time and greatly reducing labor costs. The machine vision acquisition module 7 includes two sets of upward-facing oblique area array camera modules 71 and horizontal area array camera modules 72. Each inspection station 6 corresponds to one set of upward-facing oblique area array camera modules 71 and horizontal area array camera modules 72, which capture images of the component's appearance, orientation, and tilt from the oblique and horizontal directions, respectively. The upward-facing oblique area array camera module 71 can capture images of the component's appearance from the front, left, and top angles at once in the oblique direction. The horizontal area array camera module 72 can capture images at once through reflection from the prism 66 on the fixed support bracket 63. The front inspection station 61 obtains images of the component's rear and bottom surfaces from two angles. After the front inspection station 61 completes the inspection of the five surface images of the component, the gripper rotary cylinder 46 of the third moving gripper 43 rotates the component horizontally, allowing the upper oblique area array camera module 71 to capture the right surface image of the component to be inspected. The two inspection stations work together to complete the all-round inspection of the component. The horizontal area array camera module 72 is equipped with an independent focusing motor 73, which can automatically adjust the focal length according to the component size to achieve accurate detection of the component's posture and tilt.

[0049] The unloading station 8 includes an NG material box 81 and a horizontally positioned unloading cylinder 82. The front end of the unloading cylinder 81 has a pusher plate 83, and the pusher plate 83 has push rods 84 corresponding to the positions of the 8 components at the unloading station. The unloading station 8 is located between the rear inspection station 62 and the unloading station 221, and is used to receive components that have completed inspection and are ready for sorting. After the components undergo comprehensive inspection at both inspection stations 61, the fourth moving gripper 44 picks up the components from the rear inspection station 62 and places them into the unloading station. At material handling station 8, based on the inspection results of inspection station 6, if the component is defective, the defective component discharge action is automatically executed. The horizontal push rod of the discharge cylinder 82 retracts, driving the push rod 84 on the push plate 83 to push the defective component into the NG material box 81. If the component is good, the component is clamped out of the discharge station 8 by the No. 5 moving gripper 45, and the component is rotated and adjusted by the gripper rotary cylinder 46 integrated at the top. Finally, it is placed into the discharge station 221 and flows to the next station along the discharge conveyor 22.

[0050] The industrial computer 9 is electrically connected to the continuous feeding station 2, the continuous gripping station 3, the gripper module 4, the rotary station 5, the inspection station 6, and the discharge station 8, respectively, to control the coordinated operation of each mechanism. It also has a built-in database to record product inspection data and generate data analysis reports.

[0051] In practical use, the present invention includes the following steps; 1. After pressing the "Auto / Stop" button on the industrial control computer 9, the equipment enters the automatic operation state under the control of the industrial control computer 9. The component to be tested flows into the picking station 211 along the feeding conveyor 21. The feeding blocking block 212 blocks the component from moving forward. The first moving gripper 41 of the gripper module 4 picks up the component to be tested from the picking station 211. The continuous gripping station 3 moves towards the direction of the component to be tested. The first moving gripper 41 moves to the top of the rotating station 5 and releases the gripper, placing the component to be tested into the rotating station 5. The rotating cylinder 51 on the rotating station 5 rotates 90 degrees to adjust the direction of the component to be tested. 2. The PPU module 31 drives the first moving gripper 41 to reset to the material picking station 211 and picks up the material again from the material picking station 211. At the same time, the second moving gripper 42 picks up the component to be tested from the rotary station 5, which has been adjusted in direction. The PPU module 31 is raised to the safe position. The first moving gripper 41 moves to the top of the rotary station 5 and puts the new component to be tested into the rotary station 5. The second moving gripper 42 moves to the top of the front inspection station 61 and puts the component to be tested into the slot 64 on the front inspection station 61. After the component to be tested is adjusted to a fixed position, it is released to make the component stand still and stable. The first set of machine vision acquisition modules 7 corresponding to the front inspection station 61 starts to work and acquires and inspects the five appearance surface images of the component to be tested. 3. After the front inspection station 61 completes the inspection, simultaneously, the PPU module 31 drives the second moving gripper 42 to reset to its initial position. The first moving gripper 41 and the second moving gripper 42 respectively pick up the component to be inspected from the material handling station 211 and the rotation station 5. At the same time, the third moving gripper 43 picks up the component that has completed the first inspection from the front inspection station 61. The PPU module 31 is raised again, and the third moving gripper performs a 180-degree horizontal rotation of the component through the gripper rotation cylinder 46 integrated at the top. The first moving gripper 41 and the second moving gripper 42... 2. Move to the top of the rotary station 5 and the front inspection station 61, and place the new component to be inspected on the rotary station 5 and the front inspection station 61. The third moving gripper 43 moves to the top of the rear inspection station 62 and places the component that has completed the first inspection into the slot 64 of the rear inspection station 62. At this time, the second set of machine vision acquisition modules 7 corresponding to the rear inspection station 62 starts to work and inspects the appearance image of the remaining component that was not detected by the front inspection station 61. Thus, the comprehensive inspection of the posture, tilt and appearance of the six surfaces of the component on the station is completed. 4. When the PPU module 31 drives the gripper module 4 to perform the next pick-up and drop-off action, the fourth moving gripper 44 picks up the inspected component from the rear inspection station 62 and places it on the discharge station 8. The inspection result is judged. If the inspection result is a defective product, the discharge station 8 automatically performs the discharge action. The horizontal push rod of the discharge cylinder 82 retracts and drives the push rod 84 on the push plate 83 to push the defective product into the NG box 81. If the inspection result is a good product, the fifth moving gripper 45 picks up the qualified component from the discharge station 8 and drives the component to rotate and adjust through the gripper rotary cylinder 46 integrated at the top. Then, it is placed into the discharge station 221. The qualified component flows to the next station along the discharge conveyor 22. 5. The above process is repeated until all the components to be tested on the continuous feeding station 2 are tested and transported to the next station or collected into the NG material box. After pressing the "Auto / Stop" button on the industrial control computer 9 again, the equipment stops running.

[0052] In practical use, the present invention includes the following steps; 1. After pressing the "Auto / Stop" button on the industrial control computer 9, the equipment enters the automatic operation state under the control of the industrial control computer 9. The component to be tested flows into the material handling mechanism 211 along the feeding conveyor 21. The feeding blocking block 212 blocks the component from moving forward. The first set of moving grippers 41 of the gripper mechanism 4 picks up the component from the material handling mechanism 211. The PPU picking and unloading mechanism 3 runs in the component transmission direction. The first set of moving grippers 41 runs to the rotary station 5 and releases the grippers, placing the component into the rotary station 5. The rotary station cylinder 51 rotates 90 degrees to adjust the orientation of the component. 2. The PPU pick-and-place mechanism 31 drives the first set of moving grippers 41 to reset to the pick-and-place mechanism 211 and picks up the material from the pick-and-place mechanism 211 again. At the same time, the second set of moving grippers 42 picks up the component with the adjusted orientation from the rotary station 5. The PPU pick-and-place mechanism 31 is raised to the safe position. The second set of moving grippers 42 runs to the front inspection station 61 and puts the component into the front inspection station 61. The clamping module of the front inspection station 61 is activated, and the component is adjusted to a fixed position and then released, so that the component is stationary and stable. 3. Simultaneously, the PPU pick-and-place mechanism 31 drives the second set of moving grippers 42 to reset to the initial position. The first set of moving grippers 41 and the second set of moving grippers 42 pick up materials from the pick-up mechanism 211 and the rotary station 5 respectively. The third set of moving grippers 43 picks up materials from the front inspection station simultaneously. The PPU pick-and-place mechanism 31 is raised again. The second set of moving grippers 42 moves to the front inspection station 61 and places the new component into the front inspection station 61. The third set of moving grippers 43 rotates 180 degrees and places the product into the rear inspection station. At the same time, the dual machine vision acquisition module 7 (upper camera + horizontal camera) with the imaging angles of the front and rear inspection stations 62 is triggered to work to detect the posture, tilt and appearance of the component in the station. 4. When the PPU picking and placing mechanism 31 drives the gripper mechanism 4 to perform the next picking and placing action, the fourth set of moving grippers picks up the tested component from the rear inspection station 62 and moves it to the discharge station 8. If the inspection result is a defective product, the discharge station 8 automatically performs the discharge action, the horizontal push rod of the discharge cylinder 82 retracts, and the defective product falls into the NG material box 81. If the inspection result is a good product, the fifth set of moving grippers 44 picks up the qualified component from the discharge station 8, and after the component direction is adjusted by the gripper rotation cylinder 47, it is placed into the discharge mechanism 221. The qualified component flows to the next station along the discharge conveyor 22. 5. The above process is repeated to meet the requirements of online high-speed attitude detection until all components on the continuous feeding mechanism 2 have been detected and transported to the next workstation. After pressing the "Automatic / Stop" button again, the equipment stops running.

[0053] Example 2 In this embodiment, the electronic component to be tested is a typical optocoupler (PC817 series). In other embodiments, a solid-state relay (SSR-25DA series) can also be selected. The testing and sorting methods and equipment are basically the same as in Embodiment 1. The difference lies in that this embodiment adjusts the hardware adaptation parameters, testing process, and model parameters according to the structural characteristics and defect types of the optocoupler / solid-state relay. The differences are as follows: 2.1 Hardware Configuration Differences Industrial cameras and lenses: Optocouplers / solid-state relays are smaller in size (PC817: 5.2mm×6.5mm×2.5mm) and include pin structure. The lens is replaced with a 0.5x telecentric lens, and the imaging resolution is improved to 0.005mm / pixel, ensuring that minute defects such as pin deformation and pin spacing can be detected. Lighting system: Optocouplers / solid-state relays are mostly plastic housings with metal pins. The housings have weak reflectivity, while the pins have strong reflectivity. Adjusting the LED light source wavelength to 450-500nm reduces the brightness to 400-600cd / m². Two additional sets of oblique auxiliary light sources (at a 30° angle to the upward direction) are added to avoid overexposure of pin reflections that may mask defects. Transfer rotation unit: The grippers have been replaced with a "flexible silicone gripping head + pin avoidance groove" structure to avoid damage to the pins; the positioning accuracy has been improved to ±0.05mm, which can meet the precise detection requirements of pin spacing (such as PC817 pin spacing 2.54mm); Material sorting unit: A new pin integrity secondary verification sensor has been added to avoid pin collision and deformation during sorting.

[0054] 2.2 Differences between steps 1-10 in the detection and sorting methods Step 1: Acquire differences from six-sided images Key areas of inspection: Focus on the "pin end surface, pin side, and housing end face", conduct full inspection on all six sides, and use overlapping sampling (originally 10%-15%) to ensure that no defects such as pin deformation and pin oxidation are missed; Camera angle: The optical axis of the horizontal camera is flush with the pin end face, and the upper camera adds a "top-down posture" angle to supplement the image data of product posture angle detection.

[0055] Step 2: Image Preprocessing Differences Grayscale parameters: To address the grayscale difference between the plastic casing and the metal pins, the weighted average formula is adjusted to G=0.35R+0.5G+0.15B to enhance the distinction between the pins and the casing. Noise reduction parameters: To make the edges of defects (such as cracks) on the plastic shell more delicate, adjust the Gaussian filter kernel to 2×2 (originally 3×3), and keep the median filter kernel at 2×2 to avoid blurring crack details.

[0056] Step 3: Adjusting the differences in the AI ​​model Defect type update: The classification branch has been adjusted to "Background + 6 types of optocoupler / solid-state relay defects" (pin deformation, pin oxidation, housing cracks, abnormal pin spacing, solder joint detachment, housing damage); Model parameter optimization: Two new aspect ratios (1:3, 3:1) have been added to anchor points to accommodate slender defects such as pins; the training dataset has been replaced with "2000 qualified optocoupler / solid-state relay parts + 3000 unqualified parts", and defect annotation focuses on the pin area; Model quantization: Maintaining INT8 quantization, the inference speed remains ≥30 frames / second, meeting production line requirements.

[0057] Step 9: Geometric Feature Analysis of Differences The core feature adjustment is as follows: it is replaced with features specific to optocouplers / solid-state relays, including "pin spacing, pin perpendicularity, pin length, crack length, damaged area of ​​the housing, and percentage of pin oxidation area"; Preset thresholds (example): allowable deviation of pin spacing ±0.05mm, pin perpendicularity deviation ≤0.5°, shell crack length ≤0.2mm, pin oxidation area percentage ≤5%.

[0058] Step 10: Distinction in acceptance criteria A new "pin integrity judgment" logic has been added: if a missing pin or pin flatness > 0.1mm is detected, it is directly judged as unqualified; the judgment result is now further subdivided into "pin defect type" records to facilitate subsequent process optimization.

[0059] 2.3 Verification of the implementation effect of this embodiment Inspection target: 500 PC817 optocouplers (including 400 qualified pieces and 100 unqualified pieces, covering 6 types of optocoupler defects); Evaluation metrics have been adjusted: "Pin spacing detection accuracy" and "Pin deformation recognition rate" have been added. Verification results: Single-piece inspection time is 0.45s (originally 0.42s), defect identification rate is 99.5%, false positive rate is 0.3%, and false negative rate is 0.2%, meeting the requirements of a 60-piece / minute production line.

[0060] The core adaptation logic of this embodiment is: through "hardware precision improvement + defect type adaptation + model parameter optimization", accurate detection of optocouplers / solid-state relays is achieved, while maintaining the high speed and automation advantages of the original equipment.

[0061] The methods and devices provided in the above embodiments of the present invention, through the cooperation of hardware and software, can be adapted to various types of polyhedral electronic components such as metal-clad inductors and optocouplers / solid-state relays. They are adaptable to a wide variety of defects, have a fast detection speed, and a high defect recognition rate. They are applicable to the detection of quality problems such as surface defects, deformation, damage points, unevenness, edge deformation, and abnormal chamfer angles of polyhedral electronic components. Furthermore, they have the advantages of no blind spots on the entire surface, high degree of automation, low consumption of computing resources, and strong adaptability. They can be directly deployed on the production line, significantly improving detection efficiency and quality and reducing labor costs.

[0062] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for online detection and sorting of the posture and appearance of polyhedral electronic components based on machine vision, characterized in that, It includes the following steps: S1, Product Image Acquisition Acquire image data of all surfaces of the polyhedral electronic component product to be inspected, and ensure that the acquired images contain a 10%-15% overlap area to guarantee no blind spots in the inspection; S2, Image Preprocessing The original image is grayscaled, denoised, and standardized to eliminate metallic reflections and noise interference, improve image contrast, and enable the data to be used for subsequent rapid defect detection. S3, AI segmentation model inference The preprocessed grayscale image is copied twice along the channel dimension to generate a pseudo-RGB image, which is then input into a pre-trained AI segmentation model for inference operations. A pre-trained Mask R-CNN model based on the ResNet-50 backbone network is constructed as an AI segmentation model to perform forward inference and accurately segment various types of product defects (improper posture, surface scratches, dents, edge deformation, abnormal chamfer angles, uneven metal fasteners). S4, Defect Area Marking After the AI ​​segmentation model is processed, it outputs a binary mask image of the defect area with the same size as the preprocessed image (640×480 pixels). In the mask image, the area with a pixel value marked as 1 corresponds to the suspected defect area, and the area with a pixel value marked as 0 corresponds to the background area. S5. Connectivity Analysis and Suspicious Region Extraction An 8-neighborhood connected component analysis algorithm is used to extract connected components from the mask image output in step 4; then small region filtering is performed, and then suspicious regions are marked as the basis for subsequent ROI extraction. S6. ROI Candidate Region Extraction Defect area images are extracted from the original image through coordinate mapping and ROI cropping. S7, ROI Region Image Enhancement and Filtering Targeted enhancements are implemented to address the issue of blurred edges in defective product areas (such as deformed fastening edges and abnormal chamfers), including contrast enhancement and noise reduction in the ROI area. S8. Edge Detection and Defect Contour Extraction Defect contours are extracted through edge detection and morphological optimization; S9. Defect Contour Geometric Feature Analysis Extract key geometric feature parameters of product defects, including basic geometric features and targeted features, and integrate these features to accurately determine whether the defects exceed the standards. S10. Conformity Assessment and Sorting Control Based on a preset threshold, the product is judged to be qualified, and then qualified and unqualified products are further sorted.

2. The online detection and sorting method for the attitude and appearance of polyhedral electronic components based on machine vision according to claim 1, characterized in that, Step S1 specifically includes the following steps: S1-1. Two inspection stations are set up at the front and back on the conveyor channel of the product to be inspected, and four cameras are set up; two cameras collect data in the horizontal direction and the other two cameras collect data in the diagonal upward direction. S1-2. The product to be inspected is transported to the front inspection station, where a horizontal camera and an upward-facing camera simultaneously collect image data of no less than four surfaces of the product. S1-3. Then, after rotating the product 180°, send the product to the post-inspection station. Another horizontal camera and another upward-sloping camera will simultaneously collect image data of no less than four surfaces of the product, and achieve data collection covering all six surfaces of the product. The collected images include a 10%-15% overlap area and include the three-dimensional spatial posture information of the product's outline.

3. The online detection and sorting method for the attitude and appearance of polyhedral electronic components based on machine vision according to claim 1, characterized in that, Step S2 specifically includes the following steps: S2-1. Grayscale conversion: The color image is converted to a grayscale image using a weighted average method. S2-2, Noise Reduction Processing: A combination algorithm of Gaussian filtering and median filtering is adopted. First, high-frequency noise is smoothed by 3×3 Gaussian filtering (σ=1.2), and then salt-and-pepper noise generated by metal reflection is eliminated by 3×3 median filtering. S2-3, Contrast Normalization: Adaptive Histogram Equalization (CLAHE) is used to stretch gray values ​​to the range of 0-255, enhancing the gray value difference between defects (such as scratches and dents) and the background; S2-4, Image Scaling: The preprocessed image is scaled to 640×480 pixels (to reduce the computational load of subsequent models while preserving the details of HCB1175 defects). The scaling algorithm uses bilinear interpolation.

4. The online detection and sorting method for the posture and appearance of polyhedral electronic components based on machine vision according to claim 1, characterized in that, Step S3 specifically includes the following steps: S3-1, Constructing an AI segmentation model Backbone network: ResNet50-FPN (Feature Pyramid Network), used to extract image features at different scales, adapting to multi-scale defects in products ranging from 0.03mm micro scratches to 1mm edge deformation; Region Proposal Network (RPN): Generates candidate defect regions with anchor points set to 3 scales (16×16, 32×32, 64×64) and 3 aspect ratios (1:1, 1:2, 2:1) to adapt to different shaped defects in the product; Inspection head: includes a classification branch (outputs background + 7 types of defects, the 7 types of defects are: incorrect posture, scratches, dents, edge deformation, abnormal chamfers, uneven snapping, and damage points), a bounding box regression branch (corrects the coordinates of the defect area), and a mask branch (outputs a 14×14 pixel defect mask). Post-processing module: Non-maximum suppression (NMS) is used to remove duplicate candidate boxes (IOU threshold = 0.3), and mask thresholding is used for segmentation (threshold = 0.5, areas greater than the threshold are judged as defective areas). S3-2, AI segmentation model pre-training S3-2-2 Dataset Construction Sample collection: Collect 2,000 images of qualified products and 3,000 images of unqualified products, covering different production batches and different lighting conditions; Labeling: The LabelMe tool is used to label the defect areas and generate a JSON format label file, which includes the defect type, bounding box coordinates, and mask area; Data augmentation: Randomly flip, rotate, scale, adjust brightness (±15%), and add Gaussian noise to the training set to expand the sample size to 15,000 images; Dataset partitioning: The training set, validation set, and test set are divided in an 8:1:1 ratio. S3-2-2 Training Environment: AI server with GPU, batch size = 8, input image size 640×640; S3-2-3 Training Parameter Settings Optimizer: AdamW, initial learning rate = 1e -4 Weight decay = 0.0001; Learning rate scheduling: Cosine annealing strategy is adopted, T_max=100, and the learning rate decays by 0.1 every 10 epochs; Loss function: Multi-task loss = classification loss + bounding box regression loss + mask loss, with weights of 1:1:2; Training iterations: A total of 100 epochs are trained, and the model is evaluated on the validation set every 5 epochs. The model with the highest accuracy on the validation set is saved. S3-2-4 Model Optimization: Model quantization is adopted to reduce computing power consumption and improve the inference speed of the quantized model in order to meet the real-time detection requirements; S3-3, Model Forward Inference Execution S3-3-1 Input: Six surface images of HCB1175 after preprocessing in step S2; S3-3-2 Feature Extraction: The image is processed by the ResNet50-FPN backbone network, and feature maps of four scales, C2 (160×120), C3 (80×60), C4 (40×30), and C5 (20×15), are output. S3-3-3 Candidate Region Generation: The RPN network performs sliding window detection on feature maps at four scales to generate 2000 candidate defect regions; S3-3-4 ROI Align: Aligns the features of candidate regions and outputs a 7×7×256 feature vector. S3-3-5 Multi-task reasoning: The detection head outputs the defect type probability, corrected bounding box coordinates, and 14×14 defect mask for each candidate region; S3-3-6 Post-processing: NMS removes duplicate candidate boxes, performs mask thresholding, and outputs the final defect segmentation results. It accurately segments multiple types of product defects, including surface scratches, dents, damage points, edge deformation, abnormal chamfer angles, uneven metal fasteners, and incorrect posture. The defect characteristic of incorrect posture is that the spatial three-dimensional coordinate error of the product outline exceeds the allowable range.

5. The online detection and sorting method for the posture and appearance of polyhedral electronic components based on machine vision according to claim 1, characterized in that, Step S4 specifically includes the following steps: S4-1. The AI ​​segmentation model outputs a binary mask image of the defect area. In the mask image, the area with a pixel value of 1 corresponds to the suspected defect area, and the area with a pixel value of 0 corresponds to the background area. S4-2 The AI ​​segmentation model simultaneously outputs the preliminary classification results for each suspected defect, which are stored one-to-one with the mask image.

6. The online detection and sorting method for the attitude and appearance of polyhedral electronic components based on machine vision according to claim 1, characterized in that, Step S5 specifically includes the following steps: S6-1. Connected Component Detection: The 8-neighborhood connected component analysis algorithm is used to extract connected components from the mask image output in step S4. S6-2. Small Area Filtering: Based on the product defect detection standards, calculate the area of ​​the connected domain and filter out tiny connected domains with an area of ​​less than 9 pixels to eliminate noise interference. S6-3. Suspicious Region Marking: Retain connected components that meet the conditions and record the coordinates (x1, y1, x2, y2) of the minimum bounding rectangle of each connected component as the basis for subsequent ROI extraction.

7. The online detection and sorting method for the posture and appearance of polyhedral electronic components based on machine vision according to claim 1, characterized in that, Step S6 specifically includes the following steps: S6-1, Coordinate Mapping: Map the coordinates of the bounding rectangle of the connected domain obtained in step S5 back to the image preprocessed in step S2; S6-2, ROI cropping: Based on the mapped coordinates, crop the ROI image for each suspicious region; S6-3, ROI Classification and Storage: Based on the preliminary defect classification results, store the ROIs of the same type of defects in the corresponding folders for easy and accurate analysis later.

8. The online detection and sorting method for the attitude and appearance of polyhedral electronic components based on machine vision according to claim 1, characterized in that, Step S9 specifically includes the following steps: S8-1, Basic Geometric Feature Parameters Defect area: The area within the contour is calculated using the contourArea function; Aspect Ratio: Calculates the aspect ratio of the minimum bounding rectangle of the defect; Roundness: The formula is C=4πA / L², where A is the defect area and L is the perimeter of the outline. The closer the roundness is to 1, the closer the defect is to a circle. S8-2, Targeted Feature Parameters Edge deformation: Match the extracted edge profile with the edge profile of the standard product and calculate the maximum offset distance; Chamfer angle: Fit a straight line to the chamfer profile and calculate the angle between the two fitted straight lines; Surface smoothness: Extract the depth features of surface defects and calculate the depth by grayscale gradient. For every 10 increase in grayscale value difference, the depth increases by 0.01 mm. S8-3. Perform feature integration: Integrate the seven features of each defect, namely area, aspect ratio, roundness, edge deformation, chamfer angle, flatness, and conformity of the three-dimensional coordinates of the outline, into a feature vector for final result determination.

9. The online detection and sorting method for the posture and appearance of polyhedral electronic components based on machine vision according to claim 1, characterized in that, Step 10 specifically includes the following steps: 10-1 Preset qualified parameter thresholds include: Defect area, edge deformation, chamfer angle, surface flatness, pit / protrusion depth, damage point, and allowable error range of three-dimensional coordinates of the outline. 10-2 Judgment Logic: If any feature of a defect exceeds the standard, or if there is a damage point, it is judged as a non-conforming product; if all defect features are within the threshold, it is judged as a conforming product. 10-3 Result Output: Store the judgment result (pass / fail), defect type, characteristic parameters, and product number into the SQLite database, and display them in real time on the industrial control computer interface; 10-4 Sorting Control: Qualified products continue to be conveyed forward along the product conveyor channel to the next workstation; unqualified products trigger the pneumatic switch and are sorted into the unqualified product collection container.

10. A machine vision-based online inspection and sorting device for polyhedral electronic components, characterized in that, The method for online detection and sorting of the posture and appearance of polyhedral electronic components based on machine vision, as described in any one of claims 1 to 9, includes: a control unit, a continuous feeding unit, a transfer and rotation unit, a machine vision unit, and a sorting and discharging unit. The control unit is equipped with a control program that implements steps 1 to 10, including a pre-trained AI segmentation model and an SQLite database, for running image preprocessing, AI defect detection, feature analysis and result determination algorithms. The continuous feeding unit includes a continuous conveyor belt, a material handling mechanism, and a discharge conveyor, used to continuously transport the products to be tested to the feeding station; The transfer and rotation unit includes a PPU picking and placing mechanism and a rotation mechanism, which are used to pick up the product to be inspected from the feeding station, first transport it to the front inspection station, collect image information for the first time, and then transport it to the rear inspection station for a second image information collection. The machine vision unit includes four industrial cameras, two of which are horizontal acquisition cameras and the other two are oblique upward acquisition cameras, used to acquire image information of the entire surface of the product to be inspected at the inspection station. The sorting and unloading unit includes a sorting and unloading cylinder and an NG box, which are used to sort out non-conforming products from the conveyor channel and collect them into the NG box, while qualified products are transported to the next processing station by the conveyor channel.

11. The online inspection and sorting equipment for polyhedral electronic components based on machine vision according to claim 10, characterized in that, Each testing station is equipped with a transparent fixed support bracket. Each fixed support bracket has a slot on its upper surface for placing the component to be tested. Directly below the slot on each fixed support bracket is a rectangular receiving slot. Inside the rectangular receiving slot is a right-angled equilateral triangular prism for reflecting the bottom image of the product under test. Its inclined surface is a reflective mirror, facing the side of the horizontally acquired industrial camera, and reflects the bottom image of the product to the acquisition area of ​​the horizontally acquired industrial camera.

Images