Cross-production-line environment self-calibration method and system facing downstream vision algorithm

Abstract

The invention belongs to the field of computer vision, deep learning and industrial automation, and discloses a downstream vision algorithm-oriented cross-production-line environment self-calibration method and system, and the method comprises the steps: obtaining golden feature vectors of a plurality of samples in a source production line; obtaining a current feature vector of a collection object in the target production line; calculating feature space loss between the golden feature vector and the current feature vector; and realizing closed-loop control of the target production line by using a Bayesian optimization algorithm based on the feature space loss. According to the method, target production line image acquisition hardware (an imaging device and a light source system) can be driven to automatically reproduce a golden acquisition standard of a source production line in a feature perception dimension, so that a downstream visual algorithm (not limited to types and versions) can be directly deployed on the target production line without parameter adjustment or algorithm reconstruction, and a performance index consistent with that of the source production line is kept.

Description

Technical Field

[0001] This invention belongs to the fields of computer vision, deep learning and industrial automation, and specifically relates to a cross-production line environment self-calibration method and system for downstream vision algorithms. Background Technology

[0002] In modern industrial automated production systems, computer vision algorithms serve as a core technological support and are widely used in key quality control processes such as defect detection, component positioning, dimensional measurement, and optical character recognition (OCR). The performance of these algorithm systems, whether traditional image processing toolkits (such as Halcon and OpenCV) or deep learning-based models (such as Convolutional Neural Networks (CNNs) and Visual Transformers), is highly dependent on stable and consistent image acquisition environment conditions.

[0003] However, in actual large-scale deployment, visual algorithm models that have been developed and optimized in the "source production line" (such as R&D laboratory or production line 1) generally suffer from significant performance degradation or even complete failure when migrated to the "target production line" (such as production line 2, production line 3, etc.).

[0004] The limitations of existing technologies (core technology bottlenecks) are specifically manifested in the following ways: 1. Domain Shift Mechanism Induced by Hardware Configuration Differences: In large-scale manufacturing environments, hardware systems across production lines (including different workstations on the same production line) exhibit persistent and unavoidable systematic biases. The "domain shift" effect caused by these differences is a core constraint on the performance degradation of visual inspection algorithms, and its main mechanism of action is as follows: a. Camera differences: Even cameras of the same model can have subtle differences in their sensors in terms of quantum efficiency and color response. Differences in camera models, lenses (which can cause differences in distortion and field of view), and internal image signal processors (ISPs) in color correction and noise reduction algorithms can generate drastically different image data.

[0005] b. Illumination differences: Variables such as light source type (e.g., light-emitting diode (LED) and fluorescent lamp), color temperature, brightness, spectral distribution, installation angle, and distance from the object being measured can significantly change the shadows, highlights, reflections, and contrast of an image.

[0006] c. Environmental differences: Stray light from the surrounding environment, slight wear or reflective changes of production line fixtures, equipment aging (such as light decay of LEDs) and other factors can all cause the acquisition conditions to shift over time and space.

[0007] 2. Existing technical solutions generally suffer from the dual limitations of excessively high economic costs and low operational efficiency in practical applications: Option 1: Manual Expert Tuning (Mainstream Solution): This is currently the most common but least efficient solution. Companies rely on expensive, experienced vision engineers or "camera technicians" to visit each new production line and manually adjust camera parameters (e.g., exposure, gain, white balance) and light sources (angle, brightness) to attempt to visually reproduce the acquisition conditions of the original production line. This method has significant drawbacks: (a) High cost: It requires a large investment of expert personnel, resulting in extremely high travel and labor costs for factories distributed globally; (b) Slow speed: Calibration for each production line can take hours or even days, severely delaying production line deployment and changeover; (c) Insufficient accuracy and poor reliability: Human observation or traditional histogram adjustments can only achieve "pixel-level consistency" (e.g., similar overall brightness), but there are actually "semantic inconsistencies." This phenomenon of "visual similarity but inaccurate measurement" is the underlying cause of cross-production line model failure. Option 2: Software Model Retraining (Data-Driven Solution): This solution attempts to re-collect a large number (thousands or even tens of thousands) of labeled images on each new production line and retrain or fine-tune the deep learning model to adapt it to the data distribution of the new production line. This method has the following drawbacks: (a) It deviates from the original intention of AI: It runs counter to the AI ​​industrialization concept of "develop once, deploy widely"; (b) It is costly: Data collection, especially data labeling, is the most costly and time-consuming part of the AI ​​implementation process; (c) It has poor scalability: As the number of production lines increases, the number of model branches that need to be maintained grows linearly, leading to a "combinatorial explosion" problem in model management and maintenance.

[0008] Option 3: Software Domain Adaptation (Algorithm Solution): This approach employs unsupervised or semi-supervised domain adaptation algorithms such as DANN (Domain-Adversarial Neural Networks) and CycleGAN to attempt to make the data distribution of the target production line approximate that of the source production line in terms of software features. This method has the following drawbacks: (a) Lack of stability: The robustness of the algorithm is difficult to ensure, posing a high risk in the "zero-tolerance" defect detection environment of industry; (b) Still dependent on data: A large amount of unlabeled target domain data still needs to be collected from the new production line; (c) Failure to fundamentally solve the problem: This method "fixes" downstream data problems caused by hardware differences, rather than addressing the root cause (hardware).

[0009] Unmet Needs: In summary, the industry urgently needs a solution that is automated, scalable, efficient, and addresses the root causes of the problem. Specifically, this solution should be able to automatically and algorithmically control the hardware (such as cameras and light sources) of a new production line, enabling it to accurately replicate the data acquisition environment of the "gold standard" production line in terms of feature perception (rather than human vision). This would allow the physical environment to adapt to a predetermined artificial intelligence model, rather than the other way around. This invention is designed for this purpose. Summary of the Invention

[0010] Addressing the core technical bottlenecks in existing technologies, such as visual algorithm failure due to hardware configuration differences (i.e., domain offset) between production lines, high deployment costs, lengthy changeover cycles, and heavy reliance on manual expert calibration, this invention provides a cross-production line environment self-calibration method and system for downstream visual algorithms. This enables "develop once, deploy everywhere" for visual algorithms (including deep learning models and traditional image processing operators). By constructing an automated closed-loop feature space perception hardware optimization mechanism, this invention can drive the target production line's image acquisition hardware (imaging device, light source system) to automatically reproduce the "golden acquisition standard" of the source production line in the feature perception dimension. This ensures that downstream visual algorithms (regardless of type or version) can be directly deployed to the target production line without parameter adjustment or algorithm reconstruction, while maintaining consistent performance indicators with the source production line.

[0011] To achieve the above objectives, the present invention provides the following solution: A cross-production line environment self-calibration method for downstream vision algorithms, the method comprising: Obtain the golden feature vectors of several samples from the source production line; Obtain the current feature vector of the target object in the target production line; Calculate the feature space loss between the golden eigenvector and the current eigenvector; Based on feature space loss, a Bayesian optimization algorithm is used to achieve closed-loop control of the target production line.

[0012] Preferably, the method for obtaining the golden feature vector of several samples in the source production line includes: In a preset benchmark environment, select one or more standard acquisition samples, acquire and generate a set of gold standard images; A self-supervised feature encoder trained in a vertical domain is used to perform a forward propagation operation on a set of gold standard images to extract one or more gold feature vectors.

[0013] Preferably, the method for obtaining the current feature vector of the target object in the production line includes: On a specific target production line, set the initial hardware parameters of the data acquisition device for that production line; Based on the initial hardware parameters set, the target production line is controlled to acquire the current image of the same benchmark acquisition object; The current feature vector of the current image is extracted using a self-supervised feature encoder.

[0014] Preferably, the method for calculating the feature space loss between the golden feature vector and the current feature vector includes: ; In the formula, The golden eigenvector, For initial hardware parameters, This is the current feature vector.

[0015] Preferred methods for achieving closed-loop control of the target production line based on feature space loss and using Bayesian optimization algorithms include: Feature space loss Set as the objective function to be minimized, with hardware parameters To optimize variables; Using the Bayesian optimization algorithm, based on the currently observed data... All historical data is used to update the internal agent model and calculate the acquisition function to obtain the optimal hardware parameters. ; Will The camera and light source hardware applied to the target production line are used to acquire and iterate images until the feature space loss is less than the preset convergence threshold or the maximum number of iterations is reached. Optimal hardware parameters at convergence It will be solidified and saved as the standard configuration file for the target production line, and the standard configuration file will always be loaded for image acquisition.

[0016] The present invention also provides a cross-production line environment self-calibration system for downstream vision algorithms. The system is used to implement the aforementioned method and includes: a benchmark database, an image acquisition device, a hardware control interface, and a processing unit. The benchmark database is used to store gold feature vectors; The image acquisition device includes one or more cameras and a light source located on the target production line for acquiring the current image; The hardware control interface is used to set the hardware parameters of the image acquisition device. ; The processing unit is used to achieve closed-loop control of the target production line using a Bayesian optimization algorithm.

[0017] Preferably, the processing unit includes: a feature extraction engine, a loss calculation module, and an optimization engine; The feature extraction engine embeds the model weights of a self-supervised feature encoder to extract the golden feature vector and the current feature vector. The loss calculation module is used to calculate the feature space loss between the golden feature vector and the current feature vector; The optimization engine utilizes a Bayesian optimization algorithm, receives feature space loss as feedback, and outputs new hardware parameters. This leads to the hardware control interface, forming a complete control closed loop.

[0018] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. Achieve "Develop Once, Deploy Everywhere": Completely solves the problem of algorithm readjustment caused by hardware differences. Downstream visual algorithms (whether it is SMT defect detection, OCR, or localization algorithms) only need to be developed once on the source production line, and can be seamlessly deployed to any production line that has been calibrated by this method, with algorithm performance remaining highly consistent.

[0019] 2. Significantly improves deployment efficiency and versatility: This method reduces the manual tuning process, which originally required experts to spend several days, to an automated algorithm calibration process that only takes a few minutes (e.g., 100-200 iterations). Furthermore, this method is not limited by specific downstream tasks (task-independent); only one calibration is needed, and the production line environment can be applied to all vision tasks (including classification, detection, segmentation, etc.), demonstrating extremely high versatility. 3. Significantly Reduced Costs and Dependencies: (a) Reduced Labor Costs: Avoids reliance on expensive and scarce vision experts for on-site optimization. (b) Reduced Hardware Costs: Eliminates the need for mandatory purchases of expensive, identical "standardized" cameras and light sources. This approach allows the use of different models, and even cheaper hardware, and algorithms to "calibrate" them to the same "gold standard," resulting in a significant optimization of the cost structure of industrial vision solutions.

[0020] 4. Enhanced robustness and maintainability (environmental self-healing): Production line hardware (such as LED light sources) can experience domain drift over time (months). This method can automatically operate at certain cycles (such as every night or every shift) to automatically compensate for such physical drift, ensuring that the production line environment is always maintained at the "gold standard" state, achieving "environmental self-healing," which is difficult to achieve with manual optimization. 5. Overcoming the limitations of "pixel consistency" to achieve "semantic consistency" alignment: Traditional industrial vision tuning or automatic exposure (AEC) algorithms typically aim to achieve "pixel-level statistical consistency" in image brightness, contrast, or histogram distribution. However, consistency in pixel statistical metrics is not equivalent to consistency in the input features of deep learning models. This invention breaks free from the traditional pixel-level adjustment paradigm by directly optimizing the distance in the deep feature space through closed-loop feedback, ensuring high-dimensional semantic feature consistency of the image in the "eyes" of the machine vision algorithm, rather than merely approximation based on human vision or low-level statistical metrics. This fundamentally guarantees the inference stability of downstream algorithms when deployed across production lines. Attached Figure Description

[0021] To more clearly illustrate the technical solution of the present invention, the drawings used in the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a schematic diagram of a cross-production line environment self-calibration method for downstream vision algorithms according to an embodiment of the present invention; Figure 2 This is a schematic diagram of a typical system architecture for the cross-production line environment self-calibration method according to an embodiment of the present invention; Figure descriptions: 100—Cloud / Central Server; 101—Baseline Feature Database; 102—Self-Supervised Encoder Model Library; 200—Source Production Line; 201—Baseline Acquisition Object; 202—Acquisition Device; 300—Target Production Line; 301—Baseline Acquisition Object; 302—Controllable Camera; 303—GenICam Application Programming Interface for Camera; 304—Controllable Light Source; 305—Modbus Application Programming Interface for Light Source; 306—Production Line Calibration Controller; 307—Calibration Main Program. Detailed Implementation

[0023] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0025] Example 1 like Figure 1 As shown, this invention provides a cross-production line environment self-calibration method for downstream vision algorithms. Specifically, this invention relates to an automated calibration technology applied to intelligent manufacturing, particularly in industrial scenarios such as surface mount technology (SMT), semiconductor wafer inspection (AOI), 3C product assembly, printed matter inspection, or pharmaceutical packaging. In particular, it relates to a method using self-supervised learning and black-box optimization algorithms to solve the problems of hardware inconsistency, image acquisition environment differences (i.e., "domain shift"), and the resulting domain generalization of vision algorithms caused by cross-production line deployment. The method includes: Obtain the golden feature vectors of several samples from the source production line; Obtain the current feature vector of the target object in the target production line; Calculate the feature space loss between the golden eigenvector and the current eigenvector; Based on feature space loss, a Bayesian optimization algorithm is used to achieve closed-loop control of the target production line.

[0026] In this embodiment, the method for obtaining the golden feature vector of several samples in the source production line includes: Step 1. Golden Standard Images Acquisition: In a pre-defined benchmark environment (such as a calibrated source production line or standardized laboratory environment), select one or more standard acquisition samples (such as standard PCB templates, color calibration boards, or texture calibration templates), acquire and generate a set of golden standard images. This image set Each element in the table represents a single gold standard image sample. Step 2. Extract the Golden Feature Vector: Utilize a self-supervised feature encoder trained in the vertical domain. (For example, a ResNet-50 model pre-trained on a large number of industrial images using self-supervised frameworks such as DINO and SimCLR), for the "gold standard image set". Perform a forward propagation operation to extract one or more "golden feature vectors". . Step 3. Storage Basis: Store the "Golden Eigenvector" Stored in the "benchmark database" as the "absolute standard" or "digital target" for all subsequent production line calibrations.

[0027] Specifically: Extracting the Golden Feature Vector: This step utilizes a pre-trained deep convolutional neural network (CNN) as a self-supervised feature encoder. Regarding the "gold standard image" Perform forward propagation to extract high-dimensional feature representations. The specific process and algorithm model are as follows: (1) Algorithm model structure and source (existing technology description): In this embodiment, ResNet-50 (residual network) is preferably used as the backbone network. This network structure is an existing technology (proposed by He et al.) and mainly consists of an input convolutional layer, four stages of stacked residual blocks, and a global average pooling (GAP) layer at the end.

[0028] The model's weight parameters are not randomly initialized, but rather pre-trained on general industrial datasets based on self-supervised learning frameworks such as DINO (Self-distillation with no labels) or SimCLR. This training process falls within the scope of existing technology, and its purpose is to enable the encoder to extract robust texture and structural features.

[0029] (2) Specific extraction process and function formula: Feature extraction, or the forward propagation of image data in a network, mainly involves the following mathematical transformations: Step A - Preprocessing: Preprocessing the acquired gold standard image Adjust to the input dimensions specified by the model (e.g.) ), and then normalize it. Let the pixel value be... The normalization formula is: in and These are the mean and standard deviation of the pre-trained dataset, respectively.

[0030] Step B - Feature Mapping (Convolution and Activation): The image is passed through multiple residual blocks of ResNet-50. The output of each residual block... From input and residual function The composition, the formula is: in, This represents the feature tensor input to the residual block; This represents a set of weight parameters within the residual block; Represents the residual mapping function; This represents the output result of the residual block. Specifically, it includes convolution, batch normalization, and the ReLU activation function. Convolution extracts local features of the image, such as edges and textures, while residual connections... This solves the degradation problem of deep networks.

[0031] Step C - Vectorization (Global Average Pooling): After all convolutional layers, a feature map is obtained, whose dimension is denoted as... (For example ).in, Represents the height of the feature map. The width of the feature map. This represents the number of channels. A global average pooling layer is used to compress the spatial dimension, resulting in a unique feature vector. The pooling formula is: in, Representing the One channel index; The spatial coordinate index representing the pixel; This represents the pixel value at the corresponding position.

[0032] Step D - Output: The final output golden eigenvector For one Dimensions (or determined based on the dimensions of the projection head, such as...) A real vector of dimension () is mathematically represented as: (3) Prior art statement: The ResNet-50 network topology, residual calculation formula, and DINO self-supervised training framework used in this step are all existing common techniques in the field of computer vision. The innovation of this proposal lies not in improving the network structure itself, but in improving the feature vectors extracted by this network. Creatively defined as a "digital target" in the closed-loop control system of physical hardware (camera, light source), it is used for environmental consistency calibration across production lines.

[0033] In this embodiment, the method for obtaining the current feature vector of the target object in the production line includes: Step 4. Hardware Initialization: On a specific "target production line" (e.g., production line 2), controllable hardware parameters of the production line are initialized using a software interface (e.g., GenICam, Modbus, etc.). Access granted. This parameter. For example, a vector that encompasses multiple hardware control variables. .in, This represents the camera's exposure time. This represents the camera's gain value. and These represent the proportions of the red and blue components in white balance adjustment, respectively. and The brightness intensity values ​​represent different light source channels (such as ring light and coaxial light).

[0034] Step 5. Acquire the current image: Set a set of initial hardware parameters. (in The index representing the iteration round of optimization, in the initial state. ; (Can be set to the hardware's factory default value or random initialization value). Controlling the target production line uses the [specific setting / value]. The hardware parameters of the wheel are used to obtain the "current image" of the same "benchmark acquisition object" (this object refers to the object that is physically identical to the "gold standard sample" used when constructing the benchmark on the source production line, or belongs to the same batch of standard samples with completely consistent physical characteristics). The collection.

[0035] Step 6. Extract current features: Use the same feature encoder as in Step 2. (That is, loading the same model and weights) extracts the "current feature vector" of the "current image". .

[0036] Specifically: Hardware initialization: On a target production line (e.g., production line 2), the system accesses the controllable hardware parameters of that production line through hardware driver layer software or industrial control middleware running on a local computing unit (e.g., an industrial PC). The software specifically includes: For camera devices: Use generic drivers that conform to the GenICam (Generic Interface for Cameras) standard or SDKs (Software Development Kits) provided by the device manufacturer. This software layer provides standardized APIs (Application Programming Interfaces) that allow direct reading and writing of the camera's internal registers (such as exposure time and gain) through programming code. For light source devices: Use an industrial bus communication library (such as serial communication driver software) based on Modbus RTU / TCP or RS-232 protocol to encapsulate and send control commands to the light source controller.

[0037] The system initializes controllable hardware parameters using the aforementioned software interface. This parameter is a vector encompassing multiple hardware control variables, for example... .

[0038] In this embodiment, the method for calculating the feature space loss between the golden feature vector and the current feature vector includes: Step 7. Calculate the feature space loss: for the "golden eigenvector" With "current feature vector" The "feature space loss" between them The calculation is performed. In a more ideal embodiment, this loss is the cosine distance. .

[0039] Specifically: Calculate Feature Space Loss: The system uses a built-in loss calculation module to calculate the "golden feature vector". With "current feature vector" The differences between them are quantified. This step is based on vector operations in linear algebra. The specific calculation process, formulas, and principles are as follows: (1) Input definition: The golden eigenvector from the benchmark database, denoted as .in, Represents the overall golden eigenvector; ( ) represents the first element in the vector. Feature values ​​in each dimension; This represents the total dimension of the feature vector.

[0040] The feature vector extracted in the current iteration step is denoted as... .in, Represents the current feature vector as a whole; ( ) represents the first element in the vector. Feature values ​​in each dimension; This represents the current hardware parameters. This represents the total dimension of the feature vector (e.g., 2048 dimensions as output by ResNet-50 in this embodiment).

[0041] (2) Calculation process and function formula: The system performs the following mathematical operations in sequence: Step 1 - Calculate the dot product: Calculate the sum of the products of corresponding elements of two vectors.

[0042] Step 2 - Calculate the modulus (L2 Norm): Calculate the Euclidean norm (length) of the two vectors respectively.

[0043] Step 3 - Calculate cosine similarity: Step 4 - Calculate the final loss value: Since the goal of this invention is to minimize the difference through an optimization algorithm, and a higher cosine similarity value (closer to 1) represents greater similarity, a loss function is constructed. as follows: (3) Existing Technology Description and Technology Selection Basis: The above-mentioned cosine distance calculation formula is a well-known existing technology in the fields of mathematics and signal processing. The innovative consideration for choosing this existing technology instead of other metrics (such as Euclidean distance MSE) in this embodiment is that the direction of deep learning feature vectors usually encodes the semantic and texture structure information of an image, while the magnitude is usually affected by global illumination intensity. By adopting the cosine distance formula in the existing technology, this method can mathematically decouple semantic consistency and absolute brightness difference, thereby making the calibration results more robust to linear changes in illumination intensity.

[0044] The cosine distance calculation formula described above is a well-known technique in the fields of mathematics and signal processing. This embodiment chooses this existing technique instead of other metrics (such as Euclidean distance MSE) because its innovation lies in the fact that the direction of deep learning feature vectors typically encodes the semantic and textural structural information of an image, while the magnitude is usually influenced by global illumination intensity. By applying the cosine distance formula from the existing technique, this method can mathematically decouple semantic consistency from absolute brightness differences, thereby making the calibration results more robust to linear changes in illumination intensity.

[0045] In this embodiment, the method for achieving closed-loop control of the target production line based on feature space loss and utilizing a Bayesian optimization algorithm includes: Step 8. Objective Function Construction: This involves constructing the "feature space loss" function. Defined as the objective function to be minimized, in terms of hardware parameters To optimize the variables. This function belongs to a computationally expensive, non-convex, and non-differentiable (or with unknown gradient information) black-box optimization problem, mathematically represented as: .

[0046] Step 9. Iterative Optimization: Apply the Bayesian Optimization (BO) algorithm based on the currently observed data. All historical data is used to update the internal surrogate model (such as a Gaussian process) and calculate the acquisition function (such as expected improvement) to recommend the next most valuable set of data (i.e., the set most likely to reduce loss). Hardware parameters . Step 10. Hardware closed loop: ... The camera and light source hardware applied to the target production line will return to step 5.

[0047] Step 11. Termination Condition: Iterate through steps 5 to 10 until the "feature space loss" is reached. Less than a preset convergence threshold (e.g.) This threshold corresponds to a feature similarity greater than 99%, or reaching the maximum number of iterations (e.g., 200 times). Step 12. Solidify parameters: Optimal hardware parameters at convergence. This will be solidified and saved as the "standard configuration file" for the target production line. Thereafter, the production line will always load this configuration file for image acquisition during regular production.

[0048] Specifically: Objective Function Construction: This step aims to establish a mathematical mapping between the hardware imaging quality of the physical world and the characteristic consistency of the software world, constructing an optimizable black-box function. The specific construction process is as follows: (1) Optimize variable definition: Define input variables For one A dimensional vector represents the set of all programmable controllable hardware parameters on the target production line.

[0049] in The search space is a hyperrectangular space, and each dimension of the parameter is constrained by the upper and lower limits of the physical device (e.g.: ).

[0050] (2) Construction of black-box mapping relationship: Construction of composite function Characterizing the complete physical-digital link from hardware parameters to feature loss: This function contains three nested steps: Physical components. Hardware parameters. The process of acquiring images is non-differentiable and contains physical noise. Inference stage. The neural network extracts features from the image. Calculation step. Calculate the current feature and the golden ratio. The distance (as mentioned above, cosine distance).

[0051] (3) Mathematical representation of the optimization problem: The ultimate optimization goal is to find the optimal parameters. This minimizes the value of the loss function: st (Subject to) (Hardware physical boundary constraints).

[0052] (4) Existing Technology Description: Although the mathematical concept of "objective function" falls within the existing technical scope of optimization theory, the innovation of this invention lies in constructing a composite objective function that spans "physical photoelectric imaging" and "deep learning feature space". This function has the characteristics of being non-convex, multi-modal, and derivative-free, which is the direct reason for subsequently choosing the Bayesian optimization algorithm.

[0053] Iterative Optimization: The system employs Bayesian Optimization (BO) to solve the aforementioned black-box optimization problem. This algorithm approximates the objective function by constructing a probabilistic surrogate model and uses the acquisition function to guide sampling. The specific implementation process and the involved algorithm models are as follows: (1) Surrogate Model Construction: In this embodiment, a Gaussian Process (GP) is preferably used as the surrogate model. GP is a nonparametric probabilistic model used to fit the objective function. The distribution of . For any unobserved parameter point GP predicts that its function values ​​follow a normal distribution: in: This is a historical dataset that has been observed; in this set, data pairs (in ) indicates the first The record of the next iteration, among which This represents the vector of hardware parameters used in this iteration. This represents the feature space loss value (scalar) actually collected and calculated under these parameters. This is the mean function, representing the model's expected performance prediction for the current point; is the variance function, representing the uncertainty of the model. This model relies on kernel functions to measure the similarity between points in the parameter space. In this embodiment, either the Matern kernel or the RBF (radial basis) kernel can be used. (2) Acquisition Function Calculation: In order to balance "exploration" (searching near known good points) and "exploration" (searching in unknown areas), the system calculates the acquisition function. This embodiment preferably employs an Expected Improvement (EI) strategy: Its analytical calculation formula is: in: This represents the minimum loss value among all currently observed data (i.e., the current optimal solution); and These represent the Gaussian process model at the parameter points. The predicted mean and standard deviation; The cumulative distribution function (CDF) represents the standard normal distribution. The probability density function (PDF) representing the standard normal distribution; The standardization improvement amount is represented by the following formula: (when hour, ).

[0054] (3) Next parameter recommendation: The optimization engine selects the next specific hardware parameter by maximizing the acquisition function. : The system will Send the data to the hardware interface to perform physical data acquisition and obtain a new true loss value. Then, update the dataset. And the Gaussian process model was modified.

[0055] (4) Existing technology description: The Bayesian optimization algorithm framework (covering Gaussian process regression and EI acquisition function) is an existing well-known technology in the field of machine learning (proposed by Mockus et al.). The innovation of this proposal lies in applying this mathematical tool to the specific technical category of "industrial vision hardware parameter tuning". By leveraging its sample-efficient feature, it overcomes the technical problem that traditional grid search is too time-consuming to be applied in industrial settings.

[0056] Example 2 The present invention also provides a cross-production line environment self-calibration system for downstream vision algorithms. The system is used to implement the method described in Embodiment 1. The system includes: a benchmark database, an image acquisition device, a hardware control interface, and a processing unit. A benchmark database is used to store the golden eigenvectors; An image acquisition device includes one or more programmable cameras and a programmable (e.g., multi-channel) light source located on a target production line for acquiring a current image; A hardware control interface is provided for setting the hardware parameters of the image acquisition device programmatically (e.g., via API calls). ; Processing units (such as industrial computers (IPCs), edge computing servers, or PLCs) are used to achieve closed-loop control of the target production line using Bayesian optimization algorithms.

[0057] In this embodiment, the processing unit includes: a feature extraction engine, a loss calculation module, and an optimization engine; The feature extraction engine embeds model weights of a self-supervised feature encoder to extract the golden feature vector and the current feature vector; The loss calculation module is used to calculate the feature space loss between the golden feature vector and the current feature vector. The optimization engine utilizes a Bayesian optimization algorithm, receives feature space loss as feedback, and outputs new hardware parameters. This leads to the hardware control interface, forming a complete control closed loop.

[0058] Specific implementation process: The typical system architecture of the cross-production line environment self-calibration method proposed in this invention is shown in the appendix. Figure 2 As shown. The system's logical architecture mainly includes the following core modules: Cloud / Central Server 100: Benchmark Feature Database 101: This database is used to centrally store the "golden feature vectors" of all products. .

[0059] Self-supervised encoder model library 102: This model library is used to store core self-supervised encoders. Model weights.

[0060] Source production line 200 (offline stage): Reference acquisition object 201 (such as a standard sample of “printed circuit board (PCB)”).

[0061] Acquisition device 202 (camera and light source equipped on the source production line).

[0062] Target production line 300 (online calibration stage): The reference sample 301 is the same as sample number 201, or another standard sample from the same batch.

[0063] Controllable camera 302 (e.g., an industrial camera that supports the GenICam standard).

[0064] Controllable light source 304 (e.g., a multi-channel LED light source that supports Modbus or RS-232 control).

[0065] Hardware control application programming interfaces: GenICam application programming interface 303 for the camera and Modbus application programming interface 305 for the light source. Production line calibration controller 306 (e.g., a local industrial computer IPC).

[0066] Calibration main program 307: This program runs on an industrial computer (IPC) and includes an optimization engine and hardware interface call logic.

[0067] Detailed workflow based on this architecture: 1. Offline: At point 200 on the original production line, acquire a "gold standard image" for the benchmark acquisition object 201. The encoder is used to extract the "golden feature vector". and upload This process is performed only once, and the database on cloud server 100 is accessed.

[0068] 2. Online: When the target production line 300 needs to be deployed or the product needs to be changed, start the calibration process. 3. The calibration master program 307 on the controller first downloads the corresponding product's calibration program from the cloud server 100. And the encoder model.

[0069] 4. The controller begins executing "closed-loop parameter optimization": (a) An optimization engine (e.g., a Python library that implements Bayesian optimization, such as scikit-optimize) generates a set of initial or recommended hardware parameters. .

[0070] (b) The main program controls the application programming interface (API) through the hardware. (For example: {ExposureTime:5000.0, Gain: 1.0, Light_Ch1: 128, Light_Ch2: 0}) Send to the camera and light source.

[0071] (c) The camera captures an image on the object. .

[0072] (d) Feature encoder (The ONNX or TensorRT inference engine deployed on a local compute node) performs feature extraction operations. .

[0073] (e) The loss function module performs the calculation of the feature space loss value. .

[0074] (f) The "black box function return value" is fed back to the optimization engine, which then updates its internal surrogate model (Gaussian process).

[0075] (g) Test Is it less than the convergence threshold (e.g.) If the condition is not met, repeat steps (a) to (f) to start the next iteration.

[0076] (h) If the condition is met, the optimization process terminates. At this point... That is This configuration is then saved as the "standard configuration file" for the production line. The production line then loads this configuration and enters the normal production phase.

[0077] One of the core features of this invention is the encoder. The selection of this encoder defines the perception space of the "gold standard," and its selection reflects deep technical considerations.

[0078] Why not use a task-specific model? As explained in the background section, If a downstream defect classifier is used directly, the calibration process will overfit to a specific defect (e.g., optimizing hardware to maximize the contrast of a certain scratch). This contradicts the objective of this invention. This invention aims to perform environmental calibration, not task calibration. The calibrated environment must be well-suited for all downstream tasks (including defect detection, optical character recognition (OCR), localization, etc.).

[0079] Why use self-supervised learning (SSL): Self-supervised learning (SSL) aims to learn general and robust visual representations from unlabeled data, making it an ideal "general perceptron". It does not carry any task-specific bias, but learns the inherent physical and semantic structures (such as texture, reflection, edges, and spatial relationships) in images.

[0080] Preferred embodiment (DINO): In one specific embodiment, the encoder includes a backbone network pre-trained via a DINO (Self-Distillation With No Labels) framework, and the backbone network is selected from a group consisting of ViT (Vision Transformer) and ResNet-50.

[0081] Training Data: Prior to the "Offline Benchmark Establishment" phase, a large-scale and highly diverse industrial image training dataset is pre-constructed. Specifically, this dataset contains millions of unlabeled images covering diverse acquisition conditions, including at least: different production line sources, varying lighting environments, diverse product models, and different imaging perspectives.

[0082] Training Process: The encoder is trained on the unlabeled dataset via a self-distillation method using the DINO framework. The DINO framework utilizes a student network and a teacher network (whose parameters are the EMA of the student network). The core of this training process is to force the encoder to learn feature representations robust to a set of image transformations (including but not limited to cropping, rotation, color jitter, and Gaussian blur) by optimizing a target that makes the student network output approximate the teacher network output.

[0083] Technical effect: Encoder trained by DINO Generated feature vectors It possesses a high degree of resolution for local textures, component spatial relationships, and surface materials (such as metallic reflections and PCB solder mask colors), while exhibiting a certain degree of robustness to simple global brightness or color shifts. This makes this feature vector an ideal standard for measuring "environmental perception consistency".

[0084] Alternative implementation methods (SimCLR / MoCo): The SimCLR or MoCo framework based on contrastive learning can also be used. These methods leverage the features of different augmentation versions of the same image to "bring them closer" while simultaneously "pushing away" features from different images to perform representation learning.

[0085] Feature vector extraction: Feature vectors are typically obtained from the last layer of the encoder backbone (i.e., before the classification head) or the global average pooling layer (AvgPool). The resulting feature vectors have high dimensions, such as 512, 1024, or 2048.

[0086] Another core innovation of this invention lies in the application of Bayesian Optimization (BO) algorithm to efficiently optimize the physical hardware parameter space. This method not only differs fundamentally from traditional optimization strategies, but also ensures the engineering feasibility of minute-level calibration processes in industrial environments.

[0087] 1. Define the search space (The Search Space): The first step in optimization is to accurately define the search space. The search space is multi-dimensional, encompassing different categories such as continuous and discrete types, and is subject to strict boundary constraints. Table 1 presents an example of the representation of search space variables in a specific implementation case.

[0088] Table 1 2. Selection criteria and technical advantages of Bayesian optimization (BO): Black-box property: objective function It has black-box properties, and its analytical gradient It cannot be obtained through mathematical derivation (e.g., for every 1 second increase in exposure time). The change in cosine distance per second is mainly attributed to the intervention of the physical acquisition process. Therefore, gradient-based optimization algorithms (such as stochastic gradient descent (SGD) and adaptive moment estimation (Adam) are not suitable for this type of scenario.

[0089] Expensive: As described in Example 1, a single assessment (The process, which includes hardware setup, data acquisition, reasoning analysis, and result calculation, takes approximately 1-2 seconds.)

[0090] Infeasibility traits of other algorithms: Grid Search: For 7 optimization parameters, even if each parameter is discretized into only 10 sampling points, the combined space size has reached [a certain scale]. The evaluation of the secondary function is performed. If the evaluation cycle is 2 seconds, the theoretical total time will exceed 231 days, which has significant computational complexity limitations.

[0091] Random Search: Although this method is slightly more efficient than grid search, it still requires a massive amount of evaluation (usually hundreds of thousands of iterations) in high-dimensional parameter spaces to obtain an approximate optimal solution.

[0092] The efficiency mechanism of Bayesian optimization (BO): The Bayesian optimization algorithm provides a systematic solution to computationally expensive black-box optimization problems. This method uses an objective function... A surrogate model is established, typically implemented using a Gaussian process (GP) as a probabilistic modeling tool, capable of quantitatively estimating function values ​​and their confidence intervals in the unexplored parameter space. The algorithm achieves a dynamic balance between exploration and exploitation through an acquisition function—where the Expected Improvement (EI) criterion and the Upper Confidence Bound (UCB) criterion are commonly used strategies—by iteratively optimizing the parameter points that maximize information gain, it efficiently approximates the global optimum within a finite number of evaluations. .

[0093] Technical Results: Bayesian optimization (BO) can typically find a combination of hardware parameters that approximates the global optimum within 100-300 iterations (i.e., 3-10 minutes). This effectively solves the costly problem of black-box optimization and enables its practical application in industrial settings.

[0094] 3. Selection of loss function (cosine distance) As described in the technical solution This is the preferred option.

[0095] Limitations of Euclidean distance (L2): L2 norm metric used in feature space It exhibits dual sensitivity to both the magnitude and direction of the feature vector. In visual feature representation scenarios, changes in the magnitude of the feature vector may only reflect differences in global illumination intensity or contrast (e.g., The feature pairs shown have significantly increased L2 distances, which cannot effectively represent the similarity of semantic content.

[0096] The technical advantage of cosine distance lies in the fact that this metric is only sensitive to the directional features of the feature vectors, while possessing inherent robustness to changes in their amplitude (as in the vectors mentioned above). with vector The cosine similarity is 0. In the visual feature space, the vector direction dimension encodes the essential attributes and structural relationships of the target (including semantic information such as texture features and morphological features), while changes in vector amplitude usually only reflect non-essential attribute perturbations such as illumination intensity or contrast. The optimization objective achieved by minimizing the cosine distance can promote the feature vector... With reference vector By achieving consistency in the orientation space, the relative proportions of features are maintained while avoiding interference from absolute intensity differences in similarity measurement, thus significantly improving the robustness of the perception calibration process.

[0097] Example 3 To further expand the scope of protection of this invention, the following alternative technical solutions are proposed, all of which conform to the protection concept of this invention: Alternatives to optimization algorithms: Bayesian optimization (BO) has been identified as the preferred optimization strategy. However, in high-dimensional parameter spaces (such as those with dimensions exceeding 20), other gradient-free black-box optimization algorithms (including genetic algorithms, particle swarm optimization (PSO), simulated annealing, and covariance matrix adaptive evolution strategy (CMA-ES)) are also suitable for parameter optimization.

[0098] Alternatives to the loss function: Cosine distance is the preferred metric. Feature space similarity can also be evaluated using Euclidean distance (L2 Loss), Manhattan distance (L1 Loss), Wasserstein distance (Earth-Mover's Distance), or a learnable perceptual loss network (e.g., by a discriminator). Alternatives for encoders: Self-supervised learning models are the preferred architecture for feature extraction. In specific domain scenarios, supervised models pre-trained on ultra-large-scale datasets (such as ImageNet) (such as standard ResNet-50) can also be used as feature extractors, although this may lead to a reduction in task generality.

[0099] Alternatives to the benchmark object: "Gold sample" is the preferred benchmark acquisition object. In specific application scenarios, standardized calibration boards (including checkerboard, dot matrix, standard color swatches (such as X-Rite ColorChecker) and specific texture patterns) can be used as alternative benchmarks.

[0100] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made to the technical solutions of the present invention by those skilled in the art without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A cross-production line environment self-calibration method for downstream visual algorithms, characterized in that, The method includes: Obtain the golden feature vectors of several samples from the source production line; Obtain the current feature vector of the target object in the target production line; Calculate the feature space loss between the golden eigenvector and the current eigenvector; Based on feature space loss, a Bayesian optimization algorithm is used to achieve closed-loop control of the target production line.

2. The method according to claim 1, characterized in that, Methods for obtaining the golden feature vector of several samples from the source production line include: In a preset benchmark environment, select one or more standard acquisition samples, acquire and generate a set of gold standard images; A self-supervised feature encoder trained in a vertical domain is used to perform a forward propagation operation on a set of gold standard images to extract one or more gold feature vectors.

3. The method according to claim 2, characterized in that, Methods for obtaining the current feature vector of the target object in the production line include: On a specific target production line, set the initial hardware parameters of the data acquisition device for that production line; Based on the initial hardware parameters set, the target production line is controlled to acquire the current image of the same benchmark acquisition object; The current feature vector of the current image is extracted using a self-supervised feature encoder.

4. The method according to claim 1, characterized in that, Methods for calculating the feature space loss between the golden eigenvector and the current eigenvector include: ； In the formula, The golden eigenvector, For initial hardware parameters, This is the current feature vector.

5. The method according to claim 4, characterized in that, Methods for achieving closed-loop control of the target production line based on feature space loss and using Bayesian optimization algorithms include: Feature space loss Set as the objective function to be minimized, with hardware parameters To optimize variables; Using the Bayesian optimization algorithm, based on the currently observed data... All historical data is used to update the internal agent model and calculate the acquisition function to obtain the optimal hardware parameters. ; Will The camera and light source hardware applied to the target production line are used to acquire and iterate images until the feature space loss is less than the preset convergence threshold or the maximum number of iterations is reached. Optimal hardware parameters at convergence It will be solidified and saved as the standard configuration file for the target production line, and the standard configuration file will always be loaded for image acquisition.

6. A cross-production line environment self-calibration system for downstream vision algorithms, the system being used to implement the method described in any one of claims 1-5, characterized in that, The system includes: a benchmark database, an image acquisition device, a hardware control interface, and a processing unit; The benchmark database is used to store gold feature vectors; The image acquisition device includes one or more cameras and a light source located on the target production line for acquiring the current image; The hardware control interface is used to set the hardware parameters of the image acquisition device. ; The processing unit is used to achieve closed-loop control of the target production line using a Bayesian optimization algorithm.

7. The system according to claim 6, characterized in that, The processing unit includes: a feature extraction engine, a loss calculation module, and an optimization engine; The feature extraction engine embeds the model weights of a self-supervised feature encoder to extract the golden feature vector and the current feature vector. The loss calculation module is used to calculate the feature space loss between the golden feature vector and the current feature vector; The optimization engine utilizes a Bayesian optimization algorithm, receives feature space loss as feedback, and outputs new hardware parameters. This leads to the hardware control interface, forming a complete control closed loop.

Images