A layered joint detection method and system for connector terminal electroplating appearance

Abstract

This invention discloses a layered joint inspection system and method for the electroplated appearance of connector terminals, belonging to the field of materials analysis technology. The method employs a dual-track strategy of hardware extinction and software efficiency improvement: at the hardware level, adjustable polarizers are used to physically filter out specular reflections; at the algorithm level, a four-level layered joint inspection architecture is constructed, including ROI coarse inspection, reflection suppression, multi-feature fine inspection, and meta-learning self-calibration. Specifically, through dual-parameter linked meta-learning, differentiable virtual optical parameters and detection model weights are jointly trained offline to obtain rapidly self-calibrated joint initial parameters. This invention requires only a small number of new samples to simultaneously complete the automatic adaptation of optical parameters and detection model weights during task switching, significantly reducing downtime during model changeovers. It also balances high-reflectivity curved surface defect detection accuracy, low-cost hardware configuration, and online inspection efficiency, meeting the needs of flexible production lines.

Description

Technical Field

[0001] This invention relates to the field of materials analysis technology, specifically to a method and system for combined layer detection of the electroplated appearance of connector terminals. Background Technology

[0002] Connector terminals are core connection components in electronic and electrical systems, enabling the transmission of current and signals. Their surface plating (such as gold, tin, or silver plating) directly determines the terminal's conductivity, corrosion resistance, and mating durability. During the electroplating process, defects such as uneven plating thickness, pinholes, incomplete plating, scratches, and blistering are prone to occur on the terminal surface. These defects significantly increase contact resistance and lead to premature corrosion failure. Therefore, electroplating appearance inspection is a crucial step in terminal quality control. Currently, industry inspection technologies are mainly divided into two categories: one is based on color and spectral characteristics to identify defects such as incomplete plating and discoloration; the other is based on 2D image morphology and edge analysis to identify defects such as scratches and pinholes. However, traditional solutions typically separate these two types of inspections into independent modules or devices, each with dedicated imaging hardware and algorithm models, resulting in complex systems and high costs.

[0003] Current technologies still face the following bottlenecks: First, connector terminal surfaces are often complex curved surfaces, and highly reflective areas can easily cause minor defects to go undetected. Fixed polarizers cannot adapt to the dynamically changing reflection angles of curved surfaces, while multi-exposure fusion solutions sacrifice detection speed. Second, switching between different plating types or product models requires manual recalibration of detection thresholds and even model retraining, resulting in long downtime and difficulty in adapting to flexible production needs. Third, high-precision inspection relies on expensive hardware such as hyperspectral cameras and multi-angle light sources, while pure algorithm alternatives face the problem of excessive online real-time computational burden. Fourth, low-contrast defects such as white haze and slight discoloration on tin-plated terminals are difficult to effectively distinguish using only color or morphological features. Fifth, key parameters such as ROI partitioning, layer thresholds, and feature weights rely on manual experience for setting, making it difficult to quickly adapt to different products. Therefore, there is an urgent need for an efficient, low-cost, and high-precision connector terminal electroplating appearance inspection solution that can simultaneously solve the above problems. Summary of the Invention

[0004] To address the aforementioned problems in the prior art, this invention provides a method for inspecting the appearance of electroplated connector terminals.

[0005] According to a first aspect of the present invention, a method for combined layer detection of the electroplated appearance of connector terminals is provided, comprising: Offline optimization phase P1: Using a sample set containing defect annotations, the differentiable virtual optical parameters are optimized through end-to-end differentiable training. ROI masks are generated through defect spatial distribution statistics. Layered thresholds are obtained through threshold optimization. A mapping table between coating type and feature weight vector is obtained through coating group learning. The joint initial parameters of differentiable virtual optical parameters and detection model weights are obtained through two-parameter linked meta-learning, forming a solidified parameter set. Online detection phase P2: Step SA: Acquire an image of the terminal to be tested; Step SB: Perform a first-level coarse inspection on the image based on the ROI mask and layer threshold: If the entire image is determined to be good based on the coarse inspection result, output the detection result directly and end; otherwise, output the location of one or more suspicious regions. Step SC: Using differentiable virtual optical parameters, perform a second-level reflection suppression process on the suspicious area and output an enhanced sub-image of the suspicious area; Step SD: Obtain the corresponding feature weight vector from the mapping table according to the plating type of the terminal to be inspected, extract multi-class features from the enhanced suspicious area sub-image and perform weighted fusion, input the fused features into the classifier to perform the third-level multi-feature joint fine inspection, and output the defect type and location; Task switching self-calibration phase P3: When switching detection tasks, a small number of newly labeled samples are acquired, and several steps of gradient descent are performed starting from the joint initial parameters. At the same time, the differentiable virtual optical parameters and detection model weights are updated, and the updated parameters are used for subsequent online detection.

[0006] According to some embodiments, in the method of the first aspect of the present invention, the specific manner of optimizing the differentiable virtual optical parameters includes: A differentiable virtual optical function is constructed. The function takes the original RGB image as input and virtual polarization angle and local exposure weight as learnable parameters, and outputs an enhanced image after reflection suppression. This function is used as a preprocessing layer and connected in series with the back-end defect detection network to form an end-to-end trainable system. During training, the virtual polarization angle, local exposure weight and detection network weight are optimized simultaneously to obtain the optimal differentiable virtual optical parameters.

[0007] According to some embodiments, in the method of the first aspect of the present invention, the differentiable virtual optical function performs the following processing: Each pixel of the input image is decomposed into a specular reflection component and a diffuse reflection component, where the weight of the specular reflection component is related to the virtual polarization angle; the specular reflection component is suppressed, while the diffuse reflection component is preserved or enhanced; and adaptive exposure compensation is performed on different brightness areas based on local exposure weights.

[0008] According to some embodiments, the specific method for generating the ROI mask through defect spatial distribution statistics in the first aspect of the present invention includes: Using the defect location information already labeled in the sample set during the offline optimization stage, the spatial distribution of defects in all samples is statistically analyzed. The image is divided into multiple grid units, and the frequency of defects appearing in each grid unit is calculated to generate a defect frequency heatmap. Grid units with frequencies higher than a preset threshold are marked as key regions, and morphological dilation is performed to merge and connect adjacent key regions, outputting a binary mask as the ROI mask.

[0009] According to some embodiments, the specific methods for obtaining the hierarchical threshold through threshold optimization in the method of the first aspect of the present invention include: Based on the validation set samples, the candidate threshold sequence is traversed, and the false negative rate and good product filtering rate are calculated for each candidate threshold at the current threshold. The threshold that maximizes the good product filtering rate under the preset false negative rate constraint is selected as the optimal stratification threshold.

[0010] According to some embodiments, in the method of the first aspect of the present invention, the specific method for obtaining the mapping table between coating type and feature weight vector through coating grouping learning includes: The samples are grouped according to the coating type, and a multi-feature fusion model is trained for each group. The optimal fusion weights of color features, texture features and edge features under the coating are automatically learned through backpropagation to form a coating-specific weight vector. A mapping table from coating type to weight vector is established.

[0011] According to some embodiments, in the method of the first aspect of the present invention, the multiple features include at least color features, texture features and edge features; wherein, the color features are obtained by converting the RGB image to Lab or HSV space and then calculating the color difference distribution statistics, the texture features are extracted by local binary mode or gray-level co-occurrence matrix, and the edge features are extracted by Sobel or Canny operator.

[0012] According to some embodiments, the specific manner of dual-parameter linkage meta-learning in the method of the first aspect of the present invention includes: A model-independent meta-learning algorithm is adopted to perform meta-training on multiple source tasks. Each source task is internally divided into a support set and a query set. The inner loop performs a small amount of gradient descent on the support set of each task to obtain task-specific parameters, starting from the joint initial parameters. The outer loop calculates the loss on the query set of all tasks and backpropagates to update the joint initial parameters. Finally, the joint initial parameters that can adapt to both the optical parameters and the detection model weights of the new task are learned.

[0013] According to some embodiments, in the method of the first aspect of the present invention, step SA includes: taking pictures using a color industrial camera, wherein an adjustable polarizer is mounted on the front end of the camera lens; the transmission axis angle of the adjustable polarizer is set to the optimal polarization angle determined in the offline optimization stage to filter out specular reflection light from the physical source and suppress specular interference; wherein the optimal polarization angle is determined by optimizing differentiable virtual optical parameters in the offline optimization stage.

[0014] According to a second aspect of the present invention, a layered joint inspection system for the appearance of electroplated connector terminals is provided, comprising: The offline parameter generation module is used to execute the offline optimization stage P1 of claim 1 and generate a fixed parameter set; An online detection unit, used to perform the online detection phase P2 of claim 1, includes: An image acquisition module is used to perform step SA of claim 1, acquiring an image of the terminal to be tested. The image acquisition module includes a color industrial camera and an adjustable polarizer mounted on the front of its lens. The first-level coarse inspection module is used to perform step SB of claim 1: perform first-level coarse inspection on the image based on ROI mask and layer threshold, and directly output the detection result when the whole image is determined to be good; otherwise, output the location of one or more suspicious areas. The second-level reflection suppression module is used to perform step SC of claim 1: using differentiable virtual optical parameters, performing reflection suppression processing on the suspicious area output by the first-level coarse detection module, and outputting an enhanced sub-image of the suspicious area; The third-level fine inspection module is used to perform step SD of claim 1: obtain the corresponding feature weight vector according to the coating type, extract multiple features from the enhanced suspicious area sub-image and perform weighted fusion, perform the third-level multi-feature joint fine inspection, and output the defect type and location. The self-calibration module is used to perform the task switching self-calibration phase P3 of claim 1.

[0015] Compared with the prior art, the present invention provides the following beneficial effects: 1. Significantly improved changeover efficiency, adapting to flexible production. By employing dual-parameter linked meta-learning, differentiable virtual optical parameters and detection model weights are jointly trained offline to obtain joint initial parameters that can be rapidly self-calibrated. When a product model changes, only a small number of new samples are needed to simultaneously complete the automatic and rapid self-calibration of optical parameters and model weights, eliminating the need for manual reset of thresholds, optical parameters, or retraining of the model. This significantly reduces downtime during model changeovers, enabling efficient and seamless switching of flexible production lines.

[0016] 2. Elimination of blind spots in high-reflectivity curved surface inspection A combined strategy of hardware pre-excitation and software compensation is employed. The hardware layer uses an adjustable polarizer to adjust to the optimal offline polarization angle, filtering out most specular reflections at their physical source. The algorithm layer utilizes offline-optimized differentiable virtual optical parameters to perform pixel-level suppression and exposure compensation for residual reflections. This combination enables clear imaging of minute defects in curved areas without sacrificing detection speed, effectively reducing the false negative rate.

[0017] 3. Improved detection accuracy and enhanced ability to identify low-contrast defects. By employing an adaptive multi-feature fusion mechanism for plating layers, optimal fusion weights for color, texture, and edge features are learned separately for different plating types. Compared to globally fixed weights, white haze defects in tin-plated terminals and discoloration defects in gold-plated terminals can be classified more accurately, improving the detection rate of low-contrast defects while reducing the false detection rate.

[0018] 4. Balancing detection efficiency and accuracy, achieving unbalanced allocation of computing resources. In the four-level hierarchical architecture, the first-level coarse inspection is based on ROI masks and layering thresholds, allowing most good products to pass directly without entering subsequent levels. The second-level reflection suppression and the third-level multi-feature fine inspection are only performed on a small number of suspicious areas, and both are single-pass calculations without iteration or online optimization. The overall online detection has a low average frame time, which can match the pace of high-speed production lines, while ensuring fine detection of suspicious areas.

[0019] 5. Hardware costs are significantly reduced. The hardware layer only requires a standard color industrial camera, an adjustable polarizer, and a ring-shaped white light source, eliminating the need for hyperspectral cameras, multi-angle light sources, or dedicated polarization cameras. Compared to traditional separate inspection equipment or high-precision solutions relying on multi-exposure fusion, the hardware cost is significantly reduced, and it is easy to integrate into existing production lines.

[0020] 6. Automatic parameter adaptation reduces manual intervention. In the offline optimization phase, ROI masks are automatically generated using a defect frequency heatmap, stratification thresholds are automatically determined through validation set traversal optimization, and feature weight mapping tables are automatically generated through coating grouping learning. Key parameters are not dependent on human experience for setting, thus avoiding subjective bias, and are automatically iteratively optimized as the sample library is updated.

[0021] 7. Enhanced system robustness and scalability Offline optimization and online inference are separated. In the online phase, only forward computation of fixed parameters is performed, resulting in a stable computational load that is less affected by production line fluctuations. The initial parameters for meta-learning can cover multiple source tasks. When introducing new coatings or product forms, only a small number of new samples are needed for fine-tuning, eliminating the need to train the model from scratch and reducing system expansion costs.

[0022] Online real-time performance is guaranteed 8. All online modules do not include iterative optimization or online learning. The computation graph is static forward propagation, which can fully utilize the parallel capabilities of edge computing units, keep the detection latency stable, and meet the real-time production line control requirements. Attached Figure Description

[0023] Figure 1 This is a flowchart illustrating an embodiment 1000 of the method for combined layer detection of electroplated appearance of connector terminals according to the present invention. Figure 2 for Figure 1 A flowchart illustrating the differentiable virtual optical parameter optimization step S1 in the offline optimization stage P1 of Example 1000; Figure 3 for Figure 1 A schematic diagram of the ROI mask generation step S2 in the offline optimization stage P1 of Example 1000; Figure 4 This is a schematic diagram of an embodiment 2000 of the joint inspection system for the electroplating appearance of connector terminals according to the present invention. Detailed Implementation

[0024] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0025] Figure 1 This is a schematic flowchart of an embodiment 1000 of the present invention, which describes a method for combined layer detection of the electroplated appearance of connector terminals. Figure 1 As shown, the method of Example 1000 includes three execution phases: offline optimization phase P1, online detection phase P2, and task switching self-calibration phase P3.

[0026] The core technical concept of this invention is: synergy between hardware matting and software efficiency improvement, and separation of offline optimization and online inference. Specifically: At the hardware level, adjustable polarizers suppress specular reflections from the physical source, reducing the algorithm's reliance on reflection recovery.

[0027] At the algorithm layer, a four-level hierarchical joint detection architecture is constructed to achieve differentiated processing of tasks with different complexities. The computationally intensive parameter optimization is completed in the offline stage, while only the forward calculation of fixed parameters is performed in the online stage.

[0028] Preferably, the present invention performs joint offline meta-training of differentiable virtual optical parameters and detection model weights through dual-parameter linked meta-learning to obtain joint initial parameters that can be quickly self-calibrated. This allows optical parameters and model weights to be updated synchronously with only a small number of new samples when the product is changed, thus achieving full-process automation.

[0029] The overall design solves a range of technical challenges, including blind spots in the detection of highly reflective curved surfaces, low efficiency in changing multiple product types, high hardware costs, and difficulty in identifying defects in low-contrast areas.

[0030] In the offline optimization phase P1: using a sample set containing defect annotations, the differentiable virtual optical parameters are optimized through end-to-end differentiable training; ROI masks are generated through defect spatial distribution statistics, and layered thresholds are obtained through threshold optimization; a mapping table between coating type and feature weight vector is obtained through coating group learning, and the joint initial parameters of differentiable virtual optical parameters and detection model weights are obtained through two-parameter linked meta-learning, forming a solidified parameter set.

[0031] In some specific embodiments, the offline optimization stage P1 utilizes a batch of labeled samples covering both good products and various defects to automatically learn and solidify all parameters required for detection through a data-driven approach. This stage employs a decoupled computation approach: all complex processes requiring iterative training, global search, and statistical modeling are completed in this stage, and the generated parameters, once solidified, remain unchanged. Specifically, this includes: Differentiable Optical Optimization: Physical polarization filtering and exposure compensation are modeled as differentiable functions and jointly trained end-to-end with the detection network, so that optical parameters directly serve detection accuracy; ROI Mask Generation: Based on the statistical heatmap of defect spatial distribution, high-defect-risk areas are automatically divided; Hierarchical Threshold Optimization: Candidate thresholds are traversed on the validation set to maximize the good product filtering rate under the constraint of false negative rate; Coating Adaptive Weight Learning: Multi-feature fusion models are trained by coating group to obtain coating-specific weight vectors; Two-Parameter Linked Meta-Learning: The joint adaptability of optical parameters and detection weights is learned on multiple source tasks to obtain initial parameters that can be quickly transferred.

[0032] The ultimate goal of offline optimization is to generate a set of fixed parameters, so that the online detection stage can complete high-precision and high-efficiency detection with only one forward calculation without any iteration or adaptation, while ensuring that the product can be self-calibrated very quickly when changing models.

[0033] Reflected light from a metal surface consists of specular reflection and diffuse reflection components. Specular reflection preserves the polarization characteristics of the incident light, and its polarization direction is related to the surface normal and the angle of incidence; diffuse reflection is typically unpolarized light. When the transmission axis of the polarizer is perpendicular to the polarization direction of the specular reflection, the specular reflection component is significantly attenuated, while the diffuse reflection component is preserved. However, the surface of connector terminals is a complex curved surface, and the optimal polarization angle varies in different areas, making it impossible to globally adapt a fixed polarizer. Traditional multi-exposure fusion solutions require acquiring multiple frames of images, sacrificing detection speed.

[0034] Traditional polarization imaging and specular diffuse reflection separation methods suffer from several problems: the physical model or linear decomposition model is separated from the subsequent detection task, and the optical parameters and separation methods are not optimized with the ultimate goal of defect detection in mind, making it difficult to guarantee that the separated image is most beneficial for defect identification. This invention combines differentiable optics concepts with a polarization-reflection separation model, constructing a differentiable virtual polarization optical function in the electroplating appearance inspection scenario of connector terminals. This enables joint end-to-end training of optical preprocessing parameters and the defect detection network.

[0035] Optionally, in the offline optimization phase P1, the specific methods for optimizing the differentiable virtual optical parameters include: A differentiable virtual optical function is constructed. The function takes the original RGB image as input and virtual polarization angle and local exposure weight as learnable parameters, and outputs an enhanced image after reflection suppression. This function is used as a preprocessing layer and connected in series with the back-end defect detection network to form an end-to-end trainable system. During training, the virtual polarization angle, local exposure weight and detection network weight are optimized simultaneously to obtain the optimal differentiable virtual optical parameters.

[0036] In some specific embodiments, the differentiable virtual optical function performs the following processing: decomposing each pixel of the input image into a specular reflection component and a diffuse reflection component, wherein the weight of the specular reflection component is related to the virtual polarization angle; suppressing the specular reflection component and preserving or enhancing the diffuse reflection component; and performing adaptive exposure compensation for different brightness areas according to local exposure weights.

[0037] Optionally, in the offline optimization stage P1, the specific method for generating the ROI mask through defect spatial distribution statistics includes: using the defect location information already labeled in the sample set of the offline optimization stage, statistically analyzing the spatial distribution of defects in all samples, dividing the image into multiple grid units, calculating the sample frequency of defects in each grid unit, and generating a defect frequency heatmap; marking grid units with frequencies higher than a preset threshold as key regions, performing morphological dilation operations, merging and connecting adjacent key regions, and outputting a binary mask as the ROI mask.

[0038] Electroplating defects on connector terminals are not randomly and uniformly distributed across the entire surface, but rather exhibit significant spatial clustering characteristics. For example, defects such as incomplete plating, scratches, and pinholes are prone to occur at terminal edges, bends, and contact areas due to plating solution flow or concentrated mechanical stress. By statistically analyzing the frequency of defect locations in a large number of offline samples, a heatmap reflecting the spatial probability distribution of defects can be constructed. Areas with a defect occurrence probability higher than a preset threshold are automatically designated as critical ROIs. During online inspection, these high-risk areas receive more refined inspection resources, while low-risk areas undergo only simplified threshold inspection. This achieves an unbalanced allocation of computational resources, improving overall efficiency.

[0039] Traditional ROI generation heavily relies on manual prior knowledge, making it unable to adapt to changes in defect distribution across different product forms and difficult to detect unexpected defect clustering patterns. This invention is entirely driven by offline sample data, automatically generating ROI masks by statistically analyzing the actual locations of defects, requiring no manual intervention. Furthermore, it employs a combination of grid statistics and morphological dilation, ensuring both computational efficiency and maintaining the connectivity and integrity of key regions. When products are updated or defect patterns evolve, simply updating the sample library and rerunning the statistics yields a new ROI mask.

[0040] Optionally, in the offline optimization stage P1, the specific method of obtaining the stratified threshold through threshold optimization includes: based on the validation set samples, traversing the candidate threshold sequence, calculating the false negative rate and good product filtering rate for each candidate threshold under the current threshold; and selecting the threshold that maximizes the good product filtering rate under the preset false negative rate constraint as the optimal stratified threshold.

[0041] In the first coarse detection layer of the four-level hierarchical detection architecture, the system uses a lightweight object detection model to output a good product confidence score for each key ROI region. The higher the score, the more likely the region is to be a good product; the lower the score, the more likely it is to be a defect. To determine whether the image should be directly released as a good product or sent to subsequent layers for detailed inspection, a decision threshold needs to be established. If the good product confidence score is not lower than the threshold, it is determined to be a good product and the detection ends; otherwise, it is marked as a suspicious region and enters the reflection suppression and detailed inspection stage.

[0042] The selection of stratification thresholds requires a balance between two mutually constraining metrics: false negative rate (the proportion of genuine defects mistakenly identified as good products) and good product filtering rate (the proportion of genuine good products correctly released). Users typically have strict upper limits on the false negative rate; under this constraint, the good product filtering rate should be maximized to minimize the number of suspicious samples entering subsequent strata and improve overall detection efficiency. This invention automatically determines the optimal threshold through offline validation set traversal optimization.

[0043] In some embodiments, the process of obtaining the hierarchical threshold through threshold optimization in this invention specifically includes: Prepare the validation set: Select a portion of labeled images from the offline samples, including good products and various defects, typically ranging from tens to hundreds of images.

[0044] Candidate threshold sequence: Define the search range, for example from 0.5 to 0.99, with a step size of 0.01, to obtain the sequence T∈{0.50,0.51,...,0.99}.

[0045] For each candidate threshold: input each image in the validation set into the first-level coarse detection model to obtain the good product confidence score for each image; calculate the false negative rate by dividing the number of good product confidence scores ≥ T in the real defect samples by the total number of defect samples; calculate the good product filtering rate by dividing the number of good product confidence scores ≥ T in the real good product samples by the total number of good product samples.

[0046] Selecting the optimal threshold: Among the thresholds that satisfy the false negative rate ≤ preset upper limit, select the threshold with the highest good product filtration rate as the final stratification threshold. If no threshold satisfies the false negative rate constraint, the threshold with the lowest false negative rate is selected, and an alarm is issued indicating that the model needs improvement.

[0047] According to such Figure 1 In this implementation, the present invention uses an offline, automatically optimized layered threshold to maximize the proportion of good products that can be directly released while meeting stringent requirements for missed detections. Since production line yields are typically high, most images are directly classified as good at the coarse inspection layer, skipping subsequent computationally intensive reflection suppression and fine inspection steps. This significantly reduces the average time spent on online inspection, enabling the system to match the pace of high-speed production lines. Furthermore, this threshold is optimized for specific product models; when switching products, only one optimization run based on the validation set of the new product is required, eliminating the need for manual adjustments and ensuring rapid response in flexible production.

[0048] In some specific embodiments, in the offline optimization stage P1, the specific method of obtaining the mapping table between coating type and feature weight vector by coating group learning includes: grouping samples according to coating type, training a multi-feature fusion model for each group of samples, automatically learning the optimal fusion weights of color features, texture features, and edge features under the coating through backpropagation, forming a coating-specific weight vector, and establishing a mapping table from coating type to weight vector.

[0049] Optionally, the multi-class features include at least color features, texture features, and edge features; wherein, color features are obtained by converting the RGB image to Lab or HSV space and then calculating the color difference distribution statistics, texture features are extracted by local binary mode or gray-level co-occurrence matrix, and edge features are extracted by Sobel or Canny operator.

[0050] Common plating layers for connector terminals include gold, tin, and silver, and the defect characteristics of different plating layers differ significantly. For example, defects in gold-plated terminals are often accompanied by color changes, thus color features are highly distinguishable; white haze defects in tin-plated terminals mainly manifest as blurred surface texture, making texture features more important; edge features are valuable for all plating layers. Applying the same feature fusion weight to all plating layers can lead to poor recognition performance on some plating layers.

[0051] This invention groups offline samples by coating type and trains a multi-feature fusion model independently for each group. Through backpropagation, it automatically learns the optimal fusion weights for color, texture, and edge features under that coating, forming a coating-specific weight vector. During online detection, the corresponding weights are dynamically loaded according to the actual coating type, achieving adaptive weighted fusion of features.

[0052] Optionally, the specific implementation process of coating grouping learning in the present invention includes: (1) Feature extraction: For each sample image or its suspicious region, three types of features are extracted: Color features: Convert the RGB image to Lab or HSV color space, calculate the color difference value between each pixel and the standard normal color, and statistically analyze the mean, variance, and histogram distribution of the color difference values. Output color feature vector. .

[0053] Texture features: Local Binary Pattern (LBP) or Gray-Level Co-occurrence Matrix (GLCM) are applied. LBP calculates the texture pattern of each pixel's neighborhood and generates a statistical pattern histogram; GLCM calculates statistics such as contrast, energy, entropy, and correlation. The output is a texture feature vector. .

[0054] Edge features: Edge intensity maps are extracted using Sobel or Canny operators to analyze edge continuity, density, and anomalous inflection points. Output edge feature vector.

[0055] (2) Group training and weight fusion: The offline samples were divided into K groups according to the coating type.

[0056] For each group, a multi-feature fusion model with learnable weights is constructed, with the following fusion form:

[0057] Where α, β, and γ are learnable scalar weights. Scalar weights are typically used for simplification.

[0058] Input the fused vector F into a fully connected classifier (+Softmax) and output the probability of the defect category.

[0059] The dataset is used for training, with the loss function being the classification cross-entropy. α, β, γ, and the classifier weights are updated simultaneously via backpropagation.

[0060] After training converges, the optimal weight vector for the coating is obtained. .

[0061] (3) Establish a mapping table: The plating type and its corresponding weight vector are stored in a mapping table. During online inspection, the system retrieves the plating type of the current terminal from the MES system or product model, looks up (α,β,γ) in the table, and uses it for feature fusion in step SD.

[0062] By using plating grouping learning, this invention can automatically adjust feature fusion weights based on the defect characteristics of different plating layers. Comparative experiments show that for white haze defects in tin-plated terminals, the weight of texture features automatically increases; for discoloration defects in gold-plated terminals, the weight of color features dominates. This adaptive mechanism significantly improves the recognition accuracy of low-contrast defects without increasing the online computational burden. Furthermore, due to the use of classic features and lightweight fusion, the online inference speed still meets the requirements of high-speed production lines. The mapping table mechanism allows the system to switch plating types without retraining the model; only the weights need to be switched, resulting in high efficiency in changing plating types.

[0063] Optionally, in the offline optimization phase P1, the dual-parameter linked meta-learning includes: using a model-independent meta-learning algorithm to perform meta-training on multiple source tasks; each source task is internally divided into a support set and a query set, the inner loop performs a small amount of gradient descent on the support set of each task to obtain task-specific parameters, and the outer loop calculates the loss on the query set of all tasks and backpropagates to update the joint initial parameters, finally learning joint initial parameters that can simultaneously adapt to the optical parameters and detection model weights of the new task.

[0064] The goal of meta-learning is to enable a model to learn how to learn, that is, to obtain a set of excellent initial parameters through training on multiple related source tasks. When a new task arrives, it can quickly adapt with only a small number of samples and a small number of gradient updates. Traditional meta-learning methods only optimize the network weights of the detection model. This approach innovatively incorporates differentiable virtual optical parameters, namely the virtual polarization angle θ and the local exposure weight map w, into the optimization scope of meta-learning, forming a two-parameter linkage. Specifically, it simultaneously learns the joint adaptability of optical parameters and detection weights on the source tasks, so that the finally learned joint initial parameters... When faced with new tasks, it can simultaneously update optical parameters and detection weights through a few steps of gradient descent on a small number of new samples, without any manual intervention. This solves the pain point of having to manually readjust polarization angles, exposure parameters, or retrain the detection model when changing product models.

[0065] In the offline optimization phase P1, the design of the dual-parameter linked meta-learning process in this invention is based on the characteristics of the application scenario of this invention: Optical parameters are sensitive to product characteristics: terminals with different coatings and surface shapes have significantly different optimal polarization angles and exposure compensation parameters. If optical parameters are fixed, the reflection suppression effect will decrease after the product is redesigned, affecting the detection accuracy. Therefore, optical parameters also need to be adapted quickly.

[0066] Manual parameter tuning is inefficient: Traditionally, when changing models, engineers need to manually adjust the light source and polarizer angles and reset the detection threshold, which is time-consuming and relies on experience. This solution uses meta-learning to automatically adapt optical parameters to new tasks, eliminating the need for manual intervention.

[0067] Joint meta-learning outperforms separate meta-learning: if optical parameters and detection weights are learned separately, the coupling relationship between them is ignored. Joint meta-learning can capture cross-module collaborative information on how optical parameters change in a way that is most beneficial for subsequent fine-tuning of detection weights, thereby obtaining a better initial state.

[0068] Optionally, the detailed implementation steps of the two-parameter linked meta-learning in this invention include: (1) Source task construction. Multiple source tasks are constructed from offline samples, each corresponding to a product form. Each task is internally divided into a support set and a query set.

[0069] (2) Two-parameter meta-training. Optionally, the MAML framework is used, and the specific implementation includes: Initialization: Joint initial parameters , can be randomly initialized or obtained through pre-training.

[0070] Perform an inner loop for each task: For the task Based on its support set ,from We begin a K-step gradient descent process, updating the following parameters: .in, It detects loss. After the update, task-specific parameters are obtained. .

[0071] External circulation: query set for all tasks Above, calculate the model loss after each task update:

[0072] Apply the outer loop loss to the initial parameters Differentiate and update using the second-order gradient:

[0073] Repeat the inner and outer loops multiple times until the outer loop loss converges.

[0074] (3) Output joint initial parameters. After training is completed, the final joint initial parameters are obtained. This parameter has good cross-task migration capabilities.

[0075] (4) Rapid self-calibration for new tasks. When a new product model arrives, only a small number of labeled samples need to be collected as a support set. For example, collect 5 to 10 images.

[0076] from Starting from this point, perform K steps of gradient descent, synchronizing with the inner loop to obtain the new task-specific parameters. These new parameters are then used for subsequent online detection, a process that takes only seconds to minutes and requires no manual intervention.

[0077] Through dual-parameter linked meta-learning, this invention achieves full automation of the changeover stage in connector terminal electroplating inspection. Compared to traditional methods that require manual adjustment of optical hardware, threshold resetting, and potential model retraining, this solution reduces changeover downtime from hours to minutes or even seconds. More importantly, the joint adaptation of optical parameters and detection weights enables the detection accuracy on new tasks to quickly reach a level comparable to that of fully trained systems. This mechanism is particularly suitable for flexible production lines with multiple product types and small batches, significantly improving production efficiency and equipment utilization. Furthermore, since all meta-training is completed offline, the computational load during online changeover is minimal, without impacting production line time.

[0078] In some specific embodiments, the online detection phase P2 of this invention includes steps SA-SD. The goal of the online detection phase is to perform high-precision defect determination on terminal images continuously sent from the production line with extremely low latency. Unlike the offline optimization phase, the online phase does not perform any iterative optimization, parameter search, or model training; all calculations are forward propagation of offline-fixed parameters.

[0079] The core design principle of P2 in the online detection stage is to achieve an unbalanced distribution of computing resources through a four-level hierarchical architecture. Most good products are directly released in the coarse inspection layer; only suspicious samples enter the subsequent reflection suppression and fine inspection. Moreover, reflection suppression and fine inspection only target the suspicious areas for local processing, thereby maximizing the detection speed while ensuring accuracy.

[0080] In step SA, an image of the terminal to be tested is acquired.

[0081] In some specific embodiments, in step SA, a color industrial camera is used for shooting, and an adjustable polarizer is installed at the front end of the camera lens; the transmission axis angle of the adjustable polarizer is set to the optimal polarization angle determined in the offline optimization stage to filter out specular reflection light from the physical source and suppress specular interference; wherein, the optimal polarization angle is determined by optimizing differentiable virtual optical parameters in the offline optimization stage.

[0082] Specifically, the principle of step SA is explained as follows: The hardware layer uses a color industrial camera, and an adjustable polarizer is installed at the front of the lens. During the offline optimization phase, the optimal virtual polarization angle has been obtained through differentiable virtual optical parameter optimization. This angle corresponds to the optimal transmission axis direction of the physical polarizer. During online data acquisition, the angle of the polarizer can be fixed at this value using a stepper motor or manual preset. This process filters out the specular reflection component of the incident light to the greatest extent possible, while preserving the diffuse reflection component. As a result, the raw image directly output by the camera already has significantly reduced specular interference, providing cleaner input for subsequent algorithms.

[0083] In step SA, the optimal polarization angle The end-to-end joint training results from the offline phase show that this angle makes the enhanced image most beneficial to the detection network. The angle of the physical polarizer corresponds numerically to the virtual polarization angle, but the physical adjustment only needs to be performed once when changing models or during fixed installation, without the need for online adjustment.

[0084] In step SB, a first-level coarse inspection is performed on the image based on the ROI mask and the layer threshold: if the whole image is determined to be good according to the coarse inspection result, the detection result is directly output and the process ends; otherwise, the location of one or more suspicious regions is output.

[0085] In some specific embodiments, the goal of the first-level coarse screening is to quickly identify the vast majority of good products, reducing the amount of data entering subsequent computationally intensive layers. The input is the original acquired image. The system first loads the offline-generated ROI mask. The image is divided into critical regions of interest (ROI) and non-critical ROI regions. For critical ROI regions, a lightweight object detection model is used for forward inference, outputting the good-quality confidence score for each region. For non-critical ROI regions, a simpler threshold detection method is used. If the detection results for all regions indicate that they are normal, the entire image is judged as good-quality, the detection results are directly output, and the process ends. Otherwise, all regions that fail the detection are marked as suspicious regions, and the locations of these regions are output.

[0086] In some specific embodiments, the input and data preparation in step SB includes: The input is the original RGB image acquired in step SA, which has been physically extinct using an adjustable polarizer; Offline curing parameters include: ROI mask Binary plot: 1 represents a critical ROI region, and 0 represents a non-critical region; Stratification threshold The threshold for determining the confidence level of a good product; Lightweight detection model, trained offline and with weights fixed; Normal range parameters for non-critical ROIs.

[0087] In some specific embodiments, the specific implementation process of step SB includes: (1) Region segmentation. Align the input image with the ROI mask to generate two sets of regions: Key ROI region list , for connected domains or predefined mesh blocks.

[0088] List of non-critical ROI regions This refers to the connected regions in the mask where the value is 0.

[0089] (2) Initialize the list of suspicious areas to be empty.

[0090] (3) Perform deep learning coarse inspection on each key ROI region: crop out from the original image The corresponding sub-image is input into the lightweight detection model, a forward inference is performed, and the good product confidence score for that region is output. The range is 0 to 1.

[0091] like : This The bounding box is added to the list of suspicious areas; if This area will undergo a preliminary inspection and will not be treated.

[0092] (4) For each non-critical ROI region Perform lightweight threshold detection: Calculate the statistics for the pixels within this region. Optionally, the statistics may include one or more of the following: grayscale mean, RGB channel mean, or Lab color difference mean.

[0093] Compare whether the statistical value is within the preset normal range offline: if it exceeds the range, then... The bounding box is added to the list of suspicious areas; if it is within the range, the area passes the coarse inspection.

[0094] (5) Comprehensive judgment: If the list of suspicious areas is empty, the entire image is determined to be good, and the result "Good" is output directly, ending the detection process without proceeding to subsequent steps. If the list of suspicious areas is not empty: the image is determined to be suspicious, the list of suspicious areas is output, and the process proceeds to step SC.

[0095] In step SC, a second-level reflection suppression process is performed on the suspicious area using differentiable virtual optical parameters, outputting an enhanced sub-image of the suspicious area. For the suspicious area from the coarse inspection layer, residual highlight interference needs to be further eliminated to facilitate the extraction of defect features by the fine inspection layer. The system loads the differentiable virtual optical parameters obtained through offline optimization. Apply virtual optical functions to each suspicious region sub-image. This function performs pixel-level specular / diffuse decomposition, specular suppression, and adaptive exposure compensation, outputting an enhanced sub-image. Because it only involves forward computation, it is very fast.

[0096] In some specific embodiments, the differentiable virtual optical parameters in step SC The image was trained in the offline optimization step, which makes the enhanced image more useful for detection.

[0097] In step SD, the corresponding feature weight vector is obtained from the mapping table according to the plating type of the terminal to be inspected. Multi-class features are extracted from the enhanced suspicious area sub-image and weighted fusion is performed. The fused features are input into the classifier to perform the third-level multi-feature joint fine inspection and output the defect type and location.

[0098] The fine-tuning layer in step SD precisely locates and classifies the enhanced suspicious region sub-images. First, it retrieves the corresponding feature weight vector (α, β, γ) based on the plating type of the current terminal from the offline-generated plating weight mapping table. Optionally, the plating type of the terminal is obtained from the MES system or product model input. Then, three types of features are extracted for each sub-image: Color characteristics: Convert RGB to Lab / HSV and calculate the color difference distribution statistics.

[0099] Texture features: Use LBP or GLCM to extract texture contrast, entropy, etc.

[0100] Edge features: Edge strength and continuity are extracted using Sobel / Canny.

[0101] Weighted fusion: The fused vector is input into a lightweight classifier, which outputs the defect category and confidence score. It can also regress more accurate defect bounding boxes.

[0102] The mapping table between plating types and feature weight vectors used in step SD is entirely derived from the plating grouping learning during the offline optimization phase. In the offline phase, the system groups samples according to plating types such as gold, tin, and silver, and trains a multi-feature fusion model for each group. Through backpropagation, it automatically learns the optimal fusion weights for color, texture, and edge features, forming a plating-specific weight vector that is then fixed. During online detection, step SD only needs to look up the corresponding weights based on the plating type of the current terminal, without any retraining or online weight adjustment. Furthermore, the algorithm type and parameters of the feature extraction operator are also selected and fixed in the offline phase; only forward computation is performed in the online phase.

[0103] In step SD, the feature weights are statically determined by the coating type rather than dynamically calculated. Unlike common attention mechanisms or online feature weighting, this invention does not calculate weights for each sample in real time. Instead, it obtains the optimal fixed weight vector for each coating through offline group learning, performing only one table lookup operation online. This design completely avoids the overhead of online weighted calculation while ensuring adaptability to the identification of defects in different coatings. Furthermore, step SD only processes the suspicious region sub-images output by step SB, rather than the entire image, and feature extraction uses classical operators instead of deep learning features, making the entire detailed inspection process computationally lightweight and latency-stable, meeting the real-time requirements of high-speed production lines.

[0104] Task switching self-calibration phase P3: When switching detection tasks, a small number of newly labeled samples are acquired, and several steps of gradient descent are performed starting from the joint initial parameters. At the same time, the differentiable virtual optical parameters and detection model weights are updated, and the updated parameters are used for subsequent online detection.

[0105] The task switching self-calibration stage is one of the core innovations of this invention. Its key difference from existing technologies lies in using the physical layer parameters controlling reflection suppression (i.e., the differential virtual optical parameters) and the algorithm layer parameters (i.e., the detection model weights) as joint optimization variables. Through offline meta-learning, a coordinated initial state for both is obtained, allowing for simultaneous self-calibration of both parameters with only a small number of new samples during model switching. This design breaks the module isolation of traditional methods where "optical parameters are fixed by hardware, and detection parameters are independently adapted by the algorithm," achieving rapid and automated model switching through hardware and software collaboration. Specifically, the technical effects of the task switching self-calibration stage in this invention are analyzed as follows: 1. Two-parameter linked meta-learning In connector terminal electroplating inspection, the differences between different product models are not only reflected in defect morphology but also in surface optical properties. These optical properties directly affect the optimal polarization angle and exposure compensation parameters. If the optical parameters are fixed while only adapting to the inspection model weights, the reflection suppression effect will decrease after the model change, the enhanced image quality will deteriorate, and the inspection accuracy will inevitably suffer. Conversely, if only the optical parameters are adjusted without updating the inspection model, the model will not be able to adapt to the defect feature distribution under the new plating layer. Therefore, the two must be linked. This solution internalizes this linkage into the joint initial parameters through offline meta-learning.

[0106] 2. Coordination of Inner and Outer Loops in Meta-Training Inner loop: On each source task, starting from the joint initial parameters, perform several steps of gradient descent on the support set. This step simulates the rapid adaptation process during transformation. The key point is that gradient descent acts simultaneously on both the optical parameters (θ, w) and the detection weights, updating parameters for each class. This allows the model to learn in which direction the optical parameters should be adjusted for the new task to best facilitate the rapid convergence of the detection weights.

[0107] Outer loop: Evaluates the model's loss after the inner loop update on the query set of all tasks and backpropagates this loss. The optimization goal of the outer loop is to ensure that, starting from the beginning and after a small number of gradient updates, the model achieves the minimum detection loss on new tasks. This reflects the ability to learn to adapt quickly.

[0108] 3. Second-order gradient and differentiability Since the gradient update process in the inner loop also participates in the loss calculation of the outer loop, the entire meta-training involves second-order gradients. This scheme adopts the MAML framework, which supports efficient computation under the automatic differentiation framework. The differentiable design of the virtual optical function ensures that the gradient path of the optical parameters (θ, w) is unobstructed, allowing the update amount of the optical parameters in the inner loop to be optimized by the outer loop.

[0109] 4. Mapping from virtual optical parameters to physical polarization angle In offline meta-training, the virtual polarization angle θ is optimized. It is not directly equal to the rotation angle of the physical polarizer, but rather describes an optimal specular reflection suppression effect. During the transformation self-calibration phase, the angle updated via gradient descent is converted into the actual angle of the physical polarizer through a pre-calibrated mapping relationship. This mapping is determined offline through a small number of tests, thus enabling the algorithm-optimized optical parameters to guide hardware configuration. This combination of hardware and software is one of the unique features of this solution.

[0110] 5. Feasibility of using a very small sample size Traditional supervised learning requires hundreds or even thousands of labeled samples to train a usable detection model. This approach, however, uses meta-learning with initial parameters already close to the optimal neighborhood, requiring only 5-10 new samples to converge to acceptable performance through a few steps of gradient descent. This characteristic is crucial for flexible production lines with multiple product types and small batches, as new products often cannot provide a large number of labeled samples.

[0111] In some specific embodiments, the inputs during the specific implementation of the task switching self-calibration phase P3 include: Joint initial parameters saved during the offline meta-learning phase .

[0112] A small number of labeled samples of a new product is called the support set. .

[0113] The output includes: updated dedicated task parameters .

[0114] In some embodiments, the implementation steps of the task switching self-calibration phase P3 include: (1) Prepare the support set. The operator or automated system extracts a small number of images from the continuously acquired images of the new product and labels the defects, bounding boxes, and categories in them. The support set should cover typical good and defect types and contain at least one defect instance, but does not need to be balanced or large.

[0115] (2) Loading joint initial parameters. The system loads the joint initial parameters from offline storage. This parameter is obtained through training on multiple source task elements during the offline phase and has the ability to adapt quickly across tasks.

[0116] At the same time, the same hyperparameters as those used in offline training are loaded: inner loop learning rate α, gradient descent steps K, and loss function definition.

[0117] (3) Perform gradient descent in a small number of steps. For steps k=1 to K, repeat the following operation: A forward propagation: will support the set Each image I in the input is given to the detection system with the current parameters: First, through a differentiable virtual optical function Output enhanced image Then Input detection network The prediction results are obtained; B. Calculate the loss: Calculate the total detection loss on the support set.

[0118] in, This includes classification loss and regression loss.

[0119] C. Backpropagation and parameter update: Calculate the loss over all currently trainable parameters. Find the gradient and perform one step of gradient descent:

[0120]

[0121]

[0122] Here, both optical parameters and detection weights are updated simultaneously, and the two are updated in tandem, rather than independently.

[0123] Loop D ends: After completing K steps, the updated parameters are obtained. .

[0124] According to such Figure 1 The implementation shown in this invention proposes a scheme that obtains rapidly self-calibrating joint initial parameters by using differentiable virtual optical parameters and detection model weights as joint optimization variables for offline meta-learning. During online detection, a four-level hierarchical architecture is adopted, in which the coarse detection layer quickly releases most good products based on the offline generated ROI mask and hierarchical threshold, and only performs reflection suppression and coating adaptive multi-feature fine detection sequentially on suspicious areas.

[0125] The resulting technical benefits include: only a small number of new samples are needed to simultaneously update optical parameters and detection model weights during task switching, significantly reducing downtime for model changeovers and meeting the needs of flexible production lines; at the same time, hardware extinction and software compensation work together to eliminate blind spots in the detection of highly reflective curved surfaces, and the combination of coating adaptive feature fusion improves the ability to identify low-contrast defects. Furthermore, the four-level hierarchical architecture significantly reduces the average online detection time, and the hardware cost only requires a regular color camera and an adjustable polarizer, achieving a comprehensive advantage of high precision, low cost, high efficiency, and easy model changeover.

[0126] Figure 2 for Figure 1 A flowchart illustrating the differentiable virtual optical parameter optimization sub-step S1 in the offline optimization stage P1 of Example 1000. (See attached diagram.) Figure 2 As shown, step S1 includes steps S11-S12.

[0127] In step S11, a differentiable virtual optical function is constructed. This function takes the original RGB image as input and uses the virtual polarization angle and local exposure weights as learnable parameters, outputting an enhanced image after reflection suppression. Optionally, the differentiable virtual optical function performs the following processing: decomposing each pixel of the input image into a specular reflection component and a diffuse reflection component, where the weight of the specular reflection component is related to the virtual polarization angle; suppressing the specular reflection component and preserving or enhancing the diffuse reflection component; and performing adaptive exposure compensation for different brightness areas based on the local exposure weights.

[0128] In some specific embodiments, the differentiable virtual optical function constructed in step S11 There are two core objectives: accurate modeling to reflect the difference in polarization characteristics between the specular reflection of the high-gloss region of the metal surface and the normal surface with diffuse reflection; and ease of optimization, with all operations consisting of differentiable tensor operations to ensure that the gradient can be smoothly backpropagated from the loss function of the detection network to the virtual polarization angle θ and the exposure weight map w.

[0129] Optionally, the data format definition in step S11 includes: Input raw RGB image tensor H and W represent the image height and width, and 3 represents the three RGB color channels. Pixel values ​​are normalized to [0,1] or kept between 0 and 255.

[0130] Virtual polarization angle : is a trainable scalar.

[0131] Local exposure weight map : Same resolution as the image.

[0132] Output enhanced image .

[0133] The specular reflection component and the diffuse reflection component differ fundamentally in their polarization characteristics: specular reflection has a higher degree of polarization than diffuse reflection, and the polarization directions corresponding to the two components are orthogonal to each other. It is this difference that makes it possible to filter out specular reflection using physical polarization. The function design in this invention is based on this physical principle, simulating the process of a polarizer filtering out specular reflection through a learnable virtual polarization angle.

[0134] In some specific embodiments, the detailed implementation process of step S11 includes: (1) Establish polarization-dependent differentiable transformation To map the RGB input to the specular and diffuse reflection components, this scheme constructs a differentiable polarization projection matrix M(θ):

[0135] M(θ) is a 2×3 matrix whose elements are functions of the virtual polarization angle θ. This matrix is ​​designed based on a polarization model of optical reflection: the angle between the transmission axis of the polarizer and the polarization direction of the reflected light from the specular surface determines the degree of attenuation of the specular component after transmission. By designing a reasonable function form, when θ is close to the actual optimal angle, it will contain most of the specular highlight information at that pixel. This matrix has two key characteristics: Differentiability: The derivatives of all elements with respect to θ can be obtained analytically, ensuring that the gradient can be backpropagated.

[0136] Physical constraints: The decomposition coefficients conform to the law of conservation of energy, that is, the sum of the specular and diffuse components is approximately equal to the original intensity.

[0137] (2) Targeted suppression of specular reflection: The attenuation coefficient α is a hyperparameter, typically set to 0.1. The choice of α is based on the fact that the specular reflection component in highly reflective areas is much stronger than the diffuse reflection component. Suppressing more than 90% of the specular component can effectively eliminate specular interference while retaining sufficient diffuse reflection information for defect identification.

[0138] (3) Diffuse reflection component preserved: This remains unchanged. This approach avoids actively enhancing the diffuse reflection component to prevent the introduction of additional noise.

[0139] (4) Adaptive exposure compensation:

[0140] The exposure weight map w(x,y) has a value range of 0.5 to 1.5, which should be set according to the specific implementation. A well-designed w can provide differentiated compensation for areas of different brightness in the image: increase the gain value to greater than 1 for underexposed areas (intensity < low threshold), and decrease the gain to less than 1 for overexposed areas (intensity > high threshold), thereby avoiding the loss of local information caused by uneven lighting.

[0141] This invention designs the virtual polarization angle θ as a single scalar rather than a pixel-by-pixel matrix, based on the following considerations: the connector terminal is a rigid metal component, and under single polarization illumination, the polarization direction of the specular reflected light on the entire product surface has strong global consistency; the single scalar θ makes the optimization problem extremely low-dimensional, and training is less likely to get trapped in local minima; a pixel-by-pixel angle matrix would cause the parameter space to explode (H×W variables), making training difficult and prone to overfitting under small sample conditions; even if the angle in a local area deviates from the global optimum, the subsequent local exposure weight w can be compensated and corrected.

[0142] In step S12, the function is used as a preprocessing layer and connected in series with the back-end defect detection network to form an end-to-end trainable system. During training, the virtual polarization angle, local exposure weight, and detection network weight are optimized simultaneously to obtain the optimal differentiable virtual optical parameters.

[0143] In some specific embodiments, in step S12, the differentiable virtual optical function constructed in step S11 is... As a differentiable preprocessing layer, it works in conjunction with the backend defect detection network. These are concatenated to form a unified, end-to-end trainable system. The input to the entire system is the original RGB image I, and the output is the defect detection result, including category, location, and confidence score. During training, the detection loss... The gradient can be backpropagated to the parameters θ and w of the virtual optical function, thereby achieving joint optimization of the optical parameters and the weights of the detection model.

[0144] System data flow:

[0145] In step S12, the backend defect detection network can adopt a lightweight target detection architecture to meet the speed requirements of industrial production lines. Considering the characteristics of connector terminal electroplating inspection, the network needs the following adaptations: Small target detection head: Pinholes and scratches on terminals may only account for one-thousandth of the image area. Adding a high-resolution feature layer to the feature pyramid improves the recall rate for small targets. Defect class imbalance handling: Since good products far outnumber defective products, Focal Loss or class weights are introduced into the loss function to suppress the gradient contribution of easily classified negative samples.

[0146] The network input is the enhanced image. The output includes a defect category probability vector p, bounding box coordinate regression value b, and an optional confidence score.

[0147] In some specific embodiments, the principle of end-to-end joint training in step S12 includes: virtual optical function In conjunction with the detection network, optical parameters are optimized simultaneously during training. and detection network weights The total loss is the detection loss. :

[0148] in, For classifying losses, Let be the bounding box regression loss. During backpropagation, the gradient is transmitted from the detection loss through the detection network and then through the virtual optical function back to θ and w. Since all operations of the virtual optical function are differentiable, this optimization path holds.

[0149] The optical parameters obtained after training This makes the enhanced image the most effective way to reduce detection loss, which is most beneficial for the detection network to identify defects.

[0150] Regarding end-to-end joint training, the key difference between this invention and existing technologies in step S12 lies primarily in: Traditional methods typically design optical preprocessing and defect detection networks as two independent modules, with optical parameters manually set or calibrated offline to enhance the subjective visual quality of the image. The detection network is trained separately based on the preprocessed image, with no gradient interaction between the two and their optimization objectives being separate. This invention, however, uses a differentiable virtual optical function as a trainable network layer directly connected to the detection network, and uses the detection loss as the sole optimization objective, enabling the optical parameters to directly receive gradient feedback from the detection task.

[0151] This means that optical preprocessing no longer pursues image aesthetics in a general sense, but rather adaptively adjusts to meet the needs of defect detection. For example, while a certain polarization angle may make the image more uniform overall, it will be discarded during training if it is not conducive to the network's recognition of fine scratches. In addition, all optimization calculations in step S12 of this invention are completed offline. Online inference only requires one forward propagation, eliminating the need for online searching for the optimal polarization angle or multi-frame fusion as required by some existing technologies. This improves detection accuracy while ensuring real-time production line performance.

[0152] According to such Figure 2 The illustrated implementation, through end-to-end joint training, yields differentiable virtual optical parameters that maximize the preservation of defect features and suppress specular interference in the enhanced image. Compared to manually setting polarization angles or separation optimization, detection accuracy is effectively improved. Furthermore, offline training allows for reflectivity suppression with only one forward computation in the online phase, eliminating iteration overhead and balancing the detection rate and speed of highly reflective curved surface defects, without requiring expensive hyperspectral or multi-angle light source hardware.

[0153] Figure 3 for Figure 1 A schematic diagram of the ROI mask generation sub-step S2 in Example 1000. (See attached diagram.) Figure 3 As shown, step S2 includes steps S21-S22.

[0154] Step S21: Using the defect location information already labeled in the sample set of the offline optimization stage, the spatial distribution of defects in all samples is statistically analyzed, the image is divided into multiple grid units, the sample frequency of defects in each grid unit is calculated, and a defect frequency heatmap is generated.

[0155] In some specific embodiments, the input data in step S21 is an offline sample set containing N labeled images. Each image has defect location information. Optionally, the location information can be represented by bounding boxes or pixel masks. All images are uniformly scaled to a standard size H0×W0, and the labeled coordinates are scaled synchronously.

[0156] In some specific embodiments, the specific implementation process of step S21 is as follows: (1) Divide the standard-sized image into G×G grid cells, each cell having a size of G×G.

[0157] (2) Initialize the matrix frequency Freq[i][j]=0, where .

[0158] (3) Traverse each defect in each sample: For bounding box annotations, calculate which mesh cells the bounding box covers, i.e., the intersection between the bounding box and the mesh cells. To speed up the process, the mesh index range corresponding to the bounding box can be calculated. .

[0159] For a pixel mask, iterate through all non-zero pixel coordinates in the mask and find the corresponding grid cell. Increment the frequency of the covered grid cell by 1.

[0160] (4) Normalization: The defect frequency heatmap is obtained, with a value range of [0,1].

[0161] Step S22: Mark the grid cells with frequencies higher than a preset threshold as key regions, perform morphological dilation operation, merge and connect adjacent key regions, and output a binary mask as the ROI mask.

[0162] In some specific embodiments, in step S22, a heat map H[i][j] is input, and a preset frequency threshold is used. The implementation process includes: (1) Generate a binary initial graph:

[0163] (2) Morphological dilation: Dilate B using a structuring element, iterating k times:

[0164] After expansion, previously isolated adjacent key units will merge into connected regions. Optionally, the structuring element is typically a 3×3 square kernel, and k can take the value 2.

[0165] (3) Upsample the dilated binary grid matrix to the original image resolution H0×W0 to obtain the final ROI mask. , where 1 represents a critical ROI region and 0 represents a non-critical region.

[0166] Optionally, Adjust the value based on the actual defect incidence rate. A lower value results in broader ROI coverage and fewer missed defects, but increases computational load; a higher value results in higher computational efficiency but may miss low-frequency defect areas. The specific value is determined through cross-validation.

[0167] According to such Figure 3 The embodiments shown in this invention feature special treatments that distinguish it from existing technologies, including: Sample coverage counting method: Instead of calculating the total number of defects, a defect is counted only when a defect appears in a specific grid cell within each sample. This avoids over-emphasizing the weight of a region when multiple defects cluster together in the same image. This reflects the idea of ​​using sample probability rather than the number of defects as the basis for the heatmap.

[0168] Explicit connectivity through morphological dilation: Simple threshold binarization may result in key regions appearing as discrete specks, and frequent switching between key and non-key regions during online detection reduces efficiency. The dilation operation forces the merging of adjacent high-frequency units, ensuring the integrity of the ROI region.

[0169] Offline generation, online reuse: Once the ROI mask is generated, it is fixed and can be directly loaded during online inspection. If the product batch is changed, a new mask can be obtained simply by rerunning the offline statistics, without the need for online calculation.

[0170] This invention Figure 3 The technical effects of the embodiments shown include: The ROI mask, automatically generated through offline defect spatial statistics, can accurately focus on high-defect areas. During online inspection, a lightweight target detection model is used for high-precision judgment of critical ROI areas, while non-critical ROI areas are only detected using minimal grayscale or color thresholding. Since production line yields are usually high, the vast majority of images' non-critical areas are normal, and the coarse inspection layer can quickly determine good products and release them directly. Only a small number of suspicious areas proceed to subsequent reflection suppression and multi-feature fine inspection. This unbalanced distribution significantly reduces the overall online computation load and significantly improves the detection speed, while avoiding the subjective bias and incompleteness of manually setting ROIs. More importantly, when changing products, only a small number of labeled samples of the new product need to be rerun in S21 and S22 to automatically obtain an ROI mask adapted to the new product, without any manual intervention, perfectly supporting the rapid switching needs of flexible production lines.

[0171] Figure 4 This is a schematic diagram of an embodiment 2000 of the joint inspection system for the electroplating appearance of connector terminals according to the present invention. Figure 4As shown, the system of Embodiment 2000 includes an offline parameter generation module 201, an online detection unit 202, and a self-calibration module 203. The online detection unit 202 includes: an image acquisition module 221, a first-level coarse detection module 222, a second-level reflection suppression module 223, and a third-level fine detection module 224.

[0172] Offline parameter generation module 201 is used to execute Figure 1 In the offline optimization phase P1 of Example 1000, a solidified parameter set is generated.

[0173] Online detection unit 202 is used to perform Figure 1 The online detection phase P2 in Example 1000 includes: Image acquisition module 221, used for performing Figure 1 In step SA of Example 1000, an image of the terminal to be tested is acquired. The image acquisition module includes a color industrial camera and an adjustable polarizer mounted on the front of its lens.

[0174] The first-level coarse inspection module 222 is used to perform... Figure 1 Step SB in Example 1000: Perform a first-level coarse inspection on the image based on the ROI mask and layer threshold. If the entire image is determined to be good, the detection result is directly output; otherwise, the location of one or more suspicious regions is output.

[0175] The second-level reflection suppression module 223 is used to perform... Figure 1 Step SC in Example 1000: Using differentiable virtual optical parameters, perform reflection suppression processing on the suspicious area output by the first-level coarse detection module, and output an enhanced sub-image of the suspicious area.

[0176] The third-level detailed inspection module 224 is used to perform... Figure 1 Step D in Example 1000: Obtain the corresponding feature weight vector according to the coating type, extract multiple features from the enhanced suspicious area sub-image and perform weighted fusion, perform the third-level multi-feature joint fine inspection, and output the defect type and location.

[0177] Self-calibration module 203 is used to perform... Figure 1 Task switching self-calibration phase in Example 1000.

[0178] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0179] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for combined layer detection of electroplated appearance of connector terminals, characterized in that, include: Offline optimization phase P1: Using a sample set containing defect annotations, the differentiable virtual optical parameters are optimized through end-to-end differentiable training. ROI masks are generated through defect spatial distribution statistics. Layered thresholds are obtained through threshold optimization. A mapping table between coating type and feature weight vector is obtained through coating group learning. The joint initial parameters of differentiable virtual optical parameters and detection model weights are obtained through two-parameter linked meta-learning, forming a solidified parameter set. Online detection phase P2: Step SA: Acquire an image of the terminal to be tested; Step SB: Perform a first-level coarse inspection on the image based on the ROI mask and layer threshold: If the whole image is determined to be good based on the coarse inspection result, output the detection result directly and end; Otherwise, output the location of one or more suspicious areas; Step SC: Using the differentiable virtual optical parameters, perform a second-level reflection suppression process on the suspicious region and output an enhanced sub-image of the suspicious region; Step SD: Obtain the corresponding feature weight vector from the mapping table according to the plating type of the terminal to be inspected, extract multiple features from the enhanced suspicious area sub-image and perform weighted fusion, input the fused features into the classifier to perform the third-level multi-feature joint fine inspection, and output the defect type and location; Task switching self-calibration phase P3: When the detection task is switched, a small number of newly labeled samples are acquired, and several steps of gradient descent are performed starting from the joint initial parameters. At the same time, the differentiable virtual optical parameters and the detection model weights are updated, and the updated parameters are used for subsequent online detection.

2. The method according to claim 1, characterized in that, The specific methods for optimizing the differentiable virtual optical parameters include: A differentiable virtual optical function is constructed. The function takes the original RGB image as input and virtual polarization angle and local exposure weight as learnable parameters, and outputs an enhanced image after reflection suppression. This function is used as a preprocessing layer and connected in series with the back-end defect detection network to form an end-to-end trainable system. During training, the virtual polarization angle, local exposure weight and detection network weight are optimized simultaneously to obtain the optimal differentiable virtual optical parameters.

3. The method according to claim 2, characterized in that, The differentiable virtual optical function performs the following processing: Each pixel of the input image is decomposed into a specular reflection component and a diffuse reflection component, where the weight of the specular reflection component is related to the virtual polarization angle; the specular reflection component is suppressed, while the diffuse reflection component is preserved or enhanced; and adaptive exposure compensation is performed on different brightness areas based on local exposure weights.

4. The method according to claim 1, characterized in that, The specific methods for generating ROI masks through defect spatial distribution statistics include: Using the defect location information already labeled in the sample set during the offline optimization stage, the spatial distribution of defects in all samples is statistically analyzed. The image is divided into multiple grid units, and the frequency of defects appearing in each grid unit is calculated to generate a defect frequency heatmap. Grid units with frequencies higher than a preset threshold are marked as key regions, and morphological dilation is performed to merge and connect adjacent key regions, outputting a binary mask as the ROI mask.

5. The method according to claim 1, characterized in that, The specific methods for obtaining the hierarchical thresholds through threshold optimization include: Based on the validation set samples, the candidate threshold sequence is traversed, and the false negative rate and good product filtering rate are calculated for each candidate threshold at the current threshold. The threshold that maximizes the good product filtering rate under the preset false negative rate constraint is selected as the optimal stratification threshold.

6. The method according to claim 1, characterized in that, The specific methods for obtaining the mapping table between coating type and feature weight vector through coating grouping learning include: The samples are grouped according to the coating type, and a multi-feature fusion model is trained for each group. The optimal fusion weights of color features, texture features and edge features under the coating are automatically learned through backpropagation to form a coating-specific weight vector. A mapping table from coating type to weight vector is established.

7. The method according to claim 6, characterized in that, The multiple features include at least color features, texture features, and edge features; wherein, color features are obtained by converting the RGB image to Lab or HSV space and then calculating the color difference distribution statistics, texture features are extracted by local binary mode or gray-level co-occurrence matrix, and edge features are extracted by Sobel or Canny operators.

8. The method according to claim 1, characterized in that, The specific methods of the two-parameter linked meta-learning include: A model-independent meta-learning algorithm is adopted to perform meta-training on multiple source tasks. Each source task is internally divided into a support set and a query set. The inner loop performs a small amount of gradient descent on the support set of each task to obtain task-specific parameters, starting from the joint initial parameters. The outer loop calculates the loss on the query set of all tasks and backpropagates to update the joint initial parameters. Finally, the joint initial parameters that can adapt to both the optical parameters and the detection model weights of the new task are learned.

9. The method according to claim 1, characterized in that, Step SA includes: taking pictures using a color industrial camera, with an adjustable polarizer mounted on the front end of the camera lens; the transmission axis angle of the adjustable polarizer is set to the optimal polarization angle determined in the offline optimization stage to filter out specular reflection light from the physical source and suppress specular interference; wherein, the optimal polarization angle is determined by optimizing differentiable virtual optical parameters in the offline optimization stage.

10. A layered joint inspection system for the electroplating appearance of connector terminals, characterized in that, include: An offline parameter generation module is used to execute the offline optimization stage P1 as described in claim 1 and generate a fixed parameter set; An online detection unit, used to perform the online detection stage P2 as described in claim 1, includes: An image acquisition module is used to perform step SA as described in claim 1, acquiring an image of the terminal to be tested. The image acquisition module includes a color industrial camera and an adjustable polarizer mounted on the front end of its lens. The first-level coarse inspection module is used to perform step SB as described in claim 1: perform first-level coarse inspection on the image based on the ROI mask and the layer threshold, and directly output the detection result when the entire image is determined to be good; otherwise, output the location of one or more suspicious areas. The second-level reflection suppression module is used to perform step SC as described in claim 1: using the differentiable virtual optical parameters, performing reflection suppression processing on the suspicious area output by the first-level coarse detection module, and outputting an enhanced sub-image of the suspicious area; The third-level fine inspection module is used to perform the step SD of claim 1: obtain the corresponding feature weight vector according to the coating type, extract multiple features from the enhanced suspicious area sub-image and perform weighted fusion, perform the third-level multi-feature joint fine inspection, and output the defect type and location. The self-calibration module is used to perform the task switching self-calibration phase P3 as described in claim 1.

Images