Industrial product visual quality detection and parameter optimization method based on digital twinning

Abstract

This invention discloses a method for visual quality inspection and parameter optimization of industrial products based on digital twins, relating to the fields of digital twins and machine vision. The method generates a virtual sample library by constructing a digital twin model, and trains the detection model using multi-feature fusion and transfer learning. It is then deployed to the edge for real-time detection, using the digital twin to reproduce the process and locate the root cause of defects. A multi-objective optimization algorithm is employed to optimize parameters and iterate in a closed loop. This invention does not rely on a large number of real defect samples, improving detection accuracy and generalization ability, and enabling continuous optimization of assembly quality and efficiency.

Description

Technical Field

[0001] This invention relates to the fields of digital twin and machine vision technology, specifically to a method that integrates digital twin modeling, virtual sample generation, visual inspection algorithm iteration, and closed-loop optimization of production parameters. Background Technology

[0002] In precision product assembly quality control scenarios, manual-assisted visual inspection is hampered by subjective judgment differences and efficiency issues, making it difficult to cover complex assembly details. Traditional machine vision-based inspections have weak generalization capabilities and are insufficiently adaptable to complex scenarios such as assembly occlusion and posture deviation. Traditional inspection solutions are no longer adequate for large-scale, high-precision inspection needs.

[0003] While current mainstream deep learning-based visual inspection technologies demonstrate significant advantages due to their end-to-end feature learning capabilities, they face the challenge of insufficient real-world sample data. In assembly scenarios, compliant assembly accounts for a very high percentage, and defects such as misassembly, omissions, and improper installation are low-probability events, making the collection of real-world defect samples extremely difficult. Furthermore, labeling assembly defects requires precise definition of deviations in part assembly relationships, which not only consumes substantial manpower and resources but also presents challenges in ensuring label consistency, further exacerbating the sample scarcity. Insufficient sample quantity and uneven distribution directly lead to inadequate training of deep learning models, limited generalization ability, and insufficient accuracy in identifying low-probability assembly defects. When faced with complex situations in assembly scenarios, such as part occlusion and batch material differences, false negatives and missed detections are prone to occur. Summary of the Invention

[0004] To address the problems of scarce defect samples, disconnect between digital twins and detection algorithms, insufficient linkage between detection and production optimization, and poor dynamic adaptability in existing industrial visual inspection, this invention provides a method for visual quality inspection and parameter optimization of industrial products based on digital twins. By constructing a high-fidelity digital twin model, generating diverse virtual samples, adaptively iteratively upgrading the detection algorithm, and precisely optimizing production parameters, a two-way closed loop between virtual and real is achieved, reducing dependence on real defect samples while improving detection accuracy and process stability.

[0005] A method for visual quality inspection and parameter optimization of assembly defects in industrial products based on digital twins, comprising:

[0006] Step 1: Collect data related to parts, equipment, and processes involved in assembly, and combine historical defect data to establish a correlation model between assembly parameter deviations and product defect types; build and calibrate a digital twin model, and delineate the areas of interest that need to be visually inspected in the digital twin model.

[0007] Step 2: Implant target defect parameters into the digital twin model, generate multi-dimensional sample data containing defect features through high-fidelity physical rendering, and establish a virtual sample library for model training after screening.

[0008] Step 3: Based on the virtual sample library, extract multi-dimensional features including geometric shape features, texture edge features, and deep semantic features; assign corresponding relevant filters to each feature of each region of interest and train them to obtain an initial detection model; fine-tune and adapt the initial detection model using a small number of real samples obtained in actual production.

[0009] Step 4: Perform lightweight optimization on the model trained in Step 3 and deploy it to the industrial edge; during the operation of the production line, collect workpiece images, geometric shape data and equipment parameters simultaneously, associate and fuse the collected multi-source data, and update the digital twin model in real time.

[0010] Step 5: Extract multi-dimensional features based on the multi-source data collected in Step 4, calculate the response value of each feature in the detection area through a multi-level sliding filter mechanism, and fuse the response values ​​of each feature using a dynamic weight fusion algorithm to obtain the comprehensive filter response of each region of interest.

[0011] Step 6: Set the assembly defect judgment threshold, determine whether the assembly is abnormal and the type of abnormality based on the comparison result of the threshold and the comprehensive filtering response, and generate an analysis report;

[0012] Step 7: Based on the analysis results in the analysis report, use a multi-objective optimization algorithm to calculate the optimized assembly parameter combination; perform simulation verification of the optimized parameter combination in the digital twin model, and after successful verification, send it to the actual production line for execution; the system continuously collects production data, and updates the parameters of the detection model and the association rules in the digital twin model through incremental learning to achieve optimization and iteration.

[0013] Furthermore, in step 1, the association model is constructed using a supervised machine learning algorithm, including data partitioning, hyperparameter tuning, and cross-validation; the digital twin model is built based on 3D modeling technology and a physics engine, and is calibrated using real production data.

[0014] Furthermore, in step 2, the implanted target defect parameters are achieved by modifying the position coordinates, attribute parameters, or assembly relationships of the parts in the digital twin; the high-fidelity physical rendering is based on material physical properties and lighting rendering models to generate multi-dimensional sample data containing two-dimensional images and three-dimensional point clouds.

[0015] Furthermore, in step 3, the geometric shape features are represented by point cloud features, the texture edge features are represented by histogram of oriented gradients (HOG) features, and the deep semantic features are represented by deep features from a convolutional neural network (CNN).

[0016] Furthermore, the point cloud features include global features and local features. Global features include the centroid and size, while local features include the distribution of neighborhood point clouds and normal vectors. The HOG features are extracted by statistically analyzing the gradient direction distribution of local regions in the image to capture the edge and texture features of the parts. The CNN deep features are extracted by a pre-trained deep neural network model to capture the high-order features of complex defects. The correlation filter uses a kernel correlation filter (KCF), and its training formula is as follows: In the formula, H is the filter weight, A is the cyclic matrix composed of feature samples, k is the label vector, and λ is the regularization parameter.

[0017] Further, in step 4, the lightweight optimization includes model quantization and pruning lightweight processing; wherein, the model quantization converts high-precision floating-point weights and activation values ​​into low-precision integers; the structural pruning is determined based on a weight absolute value threshold, using the following formula: In the formula, α is the model weight and γ is the preset pruning threshold.

[0018] Furthermore, in step 5, the dynamic weight fusion algorithm includes:

[0019] Calculate the peak response values ​​for various features: Side lobe ratio: Response entropy: In the formula, Q(m,n) is the response value at coordinate (m,n), and μ is the average response value of the sidelobe region. Normalized probability;

[0020] The dynamic weights are calculated based on the index normalization results, using the following formula: In the formula, ρk is the weight of the k-th feature, Sk is the normalized index value of the k-th feature, and the final comprehensive filter response Qfusion is obtained by weighted summation: In the formula, Qk is the response value of the k-th feature.

[0021] Furthermore, the filtered response value Q(m,n) is expressed as: ,

[0022] In the formula, p(m,n) is the filter weight, f(m+i,n+j) is the input feature value, and the derivative transformation is combined to enhance the defect feature response, while the Gaussian envelope model smooths the curve and suppresses noise.

[0023] Furthermore, in step 6, the assembly defect judgment threshold is calibrated through a comparative experiment of qualified and unqualified samples; the parameter deviation rate is calculated using the following formula: ,

[0024] In the formula, Q represents the parameter deviation rate, Qactual represents the actual parameter value, and Qstandard represents the standard parameter value.

[0025] Furthermore, in step 7, the multi-objective optimization algorithm selected is the NSGA-III algorithm, and the objective function is constructed as follows: In the formula, p is the assembly parameter vector, D(p) is the defect rate, and E(p) is the assembly efficiency; the optimization variables include equipment motion parameters, process execution parameters, and assembly sequence, and the constraints include the rated operating range of the equipment and process requirements.

[0026] Beneficial effects

[0027] (1) The present invention provides a method for visual quality detection and parameter optimization of assembly defects of industrial products based on digital twins. By accurately implanting defect parameters into the digital twin, and combining high-fidelity physical rendering and multi-dimensional data enhancement technology to construct a virtual sample library, the detection model can be pre-trained without relying on a large number of real defect samples, effectively breaking through the limitation of the scarcity of real defect samples, and improving the generalization ability of the model.

[0028] (2) The present invention provides a method for visual quality detection and parameter optimization of assembly defects of industrial products based on digital twins. By associating multi-source data with workpiece ID and timestamp, and combining parameter deviation rate quantitative calculation and digital twin assembly process reproduction, a full-process data link is constructed to achieve accurate mapping between defects and assembly parameters, quickly locate the core root causes such as equipment parameter drift and process execution deviation, and provide a clear direction for parameter optimization.

[0029] (3) The method for visual quality detection and parameter optimization of assembly defects of industrial products based on digital twin provided by the present invention uses the NSGA-Ⅲ multi-objective optimization algorithm to solve the optimal combination of assembly parameters. After verification by digital twin simulation, it is implemented and executed. At the same time, the detection model and twin rule set are continuously updated through incremental learning to build a closed loop of detection, traceability, optimization and iteration, dynamically adapt to changes in the production process, and continuously improve assembly quality and efficiency. Attached Figure Description

[0030] Figure 1 This is an overall flowchart of the present invention;

[0031] Figure 2 This is a schematic diagram illustrating the training and transfer learning adaptation of the relevant filter in this invention;

[0032] Figure 3This is a schematic diagram of the multi-feature dynamic weight fusion mechanism in this invention;

[0033] Figure 4 This is a schematic diagram illustrating the digital twin assembly process and defect tracing in this invention;

[0034] Figure 5 This is a schematic diagram illustrating the multi-objective optimization and simulation verification of assembly parameters in this invention. Detailed Implementation

[0035] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0036] Prior to implementation, the selection and deployment of hardware and software tools were completed. On the hardware side, a 3D binocular camera was used to simultaneously acquire 2D images and 3D point clouds of the assembled workpiece. An industrial parameter acquisition gateway was used to obtain motion accuracy, execution parameters, and process status data of the assembly equipment through an industrial bus. An industrial edge computing platform with GPU computing power was configured for the deployment of the detection model. At the same time, a workpiece ID recognition module was deployed to realize the association of multi-source data. On the software and algorithm tools side, 3D solid modeling software was used to construct 3D models of parts and equipment. A physical engine was configured through a digital twin platform to recreate the assembly scene. Based on the Python development environment, point cloud processing library, image processing library, and deep learning framework were used to extract point cloud features, HOG features, and CNN deep features and train the model. A model lightweighting tool was used to complete the quantization and pruning of the detection model. The NSGA-Ⅲ algorithm was deployed with the help of a multi-objective optimization algorithm library. At the same time, data cleaning software, sample labeling tools, and data association tools were configured for data preprocessing and sample management.

[0037] like Figure 1 As shown, a method for visual quality inspection and parameter optimization of industrial products based on digital twins includes the following steps:

[0038] Step 1: Data Acquisition and Digital Twin Model Construction. First, systematically collect all core data involved in the entire assembly process, and gather past production assembly defect records and their corresponding real-time production parameters. Based on historical data, a supervised machine learning algorithm is used to construct a correlation model between assembly parameter deviations and defect types, thereby establishing a predictive mapping relationship from abnormal production parameters to specific defects. Next, in a 3D modeling environment, based on actual parts, equipment, and process information, a corresponding 3D solid model is constructed and imported into the digital twin platform. A physics engine is configured for the model on the platform to realistically reproduce the equipment movement and the mating relationships between parts during the assembly process. Subsequently, by collecting a large amount of real assembly data and comparing it with simulation results, the parameters of the digital twin model are repeatedly calibrated to minimize the deviation between the simulation scene and the actual production environment. Finally, in this calibrated high-fidelity digital twin model, based on the assembly datum, key mating surfaces, and core functional areas of the parts, all areas of interest requiring subsequent visual inspection are clearly defined using rectangular or polygonal selection methods to ensure that potential defect locations are completely covered.

[0039] In this embodiment, the core data collected throughout the entire process mainly includes three categories: first, part data, such as geometric dimensions, fit tolerances and material characteristics; second, equipment data, such as motion accuracy, execution parameters of each axis and operation sequence; and third, process data, such as assembly process steps, design fit clearance standards and fastening torque requirements.

[0040] These data were acquired through multiple methods: 3D topographic data of parts were collected using an industrial-grade 3D laser scanner; motion and execution parameters were read directly from the assembly equipment controller via an industrial fieldbus; and established processes and standards were extracted from the enterprise's process management system. Simultaneously, historical data packages containing defect records and corresponding complete production parameters from the past 12 months were retrieved. Based on this historical data, using defect types as labels and corresponding production parameters as input features, a highly accurate parameter deviation-defect correlation model was trained through a standardized data partitioning, hyperparameter tuning, and cross-validation process. In the 3D modeling stage, a 3D model of the parts, equipment, and production line environment consistent with the physical objects was constructed in professional software. After being imported into the twin platform, it was given realistic physical properties and motion constraints. To calibrate the model, a total of 500 sets of real assembly process data were collected and compared item by item with the simulation output. Based on this, model parameters were corrected, significantly improving simulation confidence. When defining the inspection area, key assembly locations on the 3D model were directly selected and marked in the twin model's interactive interface, forming a corresponding set of inspection interest areas.

[0041] Step 2: Construct a virtual sample library. By modifying the part attribute parameters, assembly relationships, or position coordinates in the digital twin model built in Step 1 to implant defects, and based on material physical properties and lighting rendering models, generate 2D images and 3D point cloud samples containing defects, ensuring consistency with the visual and spatial characteristics of real defects. Sample labels automatically extract defect type, location information, deviation degree, and corresponding assembly parameters, using a standardized format for model training. Physical rendering technology is used to restore the surface characteristics of parts and the on-site lighting effects, and then image rotation, scaling, and noise addition are used to enhance sample diversity. Valid samples are selected based on clarity and defect identifiability, and defect type, location, and deviation information are automatically labeled to construct a sufficient and high-quality virtual sample library.

[0042] In this embodiment, a complete assembly defect classification system is first established based on the root causes and external manifestations of defects. This system comprehensively covers various defect types, including misassembly, omissions, positional misalignment, excessive clearance, and fastening failure, and provides clear judgment criteria and morphological characteristic descriptions for each type of defect. Subsequently, in the constructed digital twin model, the spatial position, physical properties, or assembly logic relationships of parts within the twin are programmed to be adjusted, and the parameter settings of the target defect are implanted into the virtual assembly scene. Combining the real physical properties of the part material with a preset ambient lighting model, high-fidelity physical rendering is performed, simultaneously generating multi-dimensional sample data including two-dimensional images and three-dimensional point clouds, thereby ensuring that the generated virtual samples are consistent with the real defect characteristics in terms of visual appearance and spatial morphology. While the samples are being generated, the system automatically records the defect type, location, deviation value, and corresponding virtual assembly parameters of the sample based on the implanted defect parameters, forming a standardized sample label with a unified format.

[0043] Preferably, this embodiment employs a procedural image transformation technique to automatically generate diverse defect sample data: a random affine transformation matrix is ​​applied to the base sample image and its corresponding bounding box data. This matrix uniformly controls the scaling, rotation, and translation operations of the image. Simultaneously with the image transformation, the new coordinates and dimensions of the bounding boxes in the transformed image are calculated based on the same transformation matrix, thereby ensuring that each generated virtual sample image has precisely matching annotation information. This method achieves automated batch expansion of defect samples, effectively simulating the morphology of the same defect at different viewpoints, locations, and scales, and solves the problem of insufficient model training data caused by the scarcity of real defect samples.

[0044] To ensure sample quality, image clarity and defect feature identification were set as core screening indicators. Specific image processing algorithms were used to quantitatively evaluate and screen batch-generated samples, automatically removing invalid samples with blurry images or insignificant defect features, ultimately retaining high-quality, qualified samples. This resulted in the construction of a virtual sample library for model training, providing a sufficient and reliable data foundation for subsequent detection model training.

[0045] Step 3: Extract point cloud features (representing the 3D morphology of the part), HOG features (capturing edge texture), and CNN deep features (learning high-order semantic information) based on the virtual sample library to characterize the assembly features of the part from multiple dimensions. Assign kernel correlation filters to each type of feature for each region of interest in detection, and train the filter parameters using virtual samples. Collect a small number of real defect samples, freeze the underlying weights of the model and fine-tune the top-level parameters through transfer learning to adapt to the real scene and improve the model's detection accuracy.

[0046] like Figure 2 As shown, in this embodiment, point cloud features are extracted using point cloud processing technology to extract global and local features to characterize the three-dimensional spatial shape of the part. HOG features are extracted by statistically analyzing the gradient direction distribution of local regions in the image to capture the edge and texture features of the part. CNN deep features are extracted using a pre-trained deep neural network model to capture high-order features of complex defects. Simultaneously, a kernel correlation filter is individually assigned to each feature of each detection interest region. This embodiment uses a kernel correlation filter (KCF) to ensure targeted detection. Its training formula is: In the formula, H represents the filter weights, A is the cyclic matrix composed of feature samples, k is the label vector, and λ is the regularization parameter. Various feature data are extracted from the virtual sample library, input into the corresponding filters, and reasonable hyperparameters such as batch size, number of iterations, and learning rate are set. The overall relevance filter training is completed by minimizing the loss function. A small number of representative real assembly defect samples covering various defect types are collected. After manual verification and annotation to ensure accuracy, a transfer learning approach is adopted based on the pre-trained overall filter model. The bottom-level weights are frozen to retain the general features learned by the model on virtual samples, and only the parameters of the top-level classifier are fine-tuned to adapt to the feature distribution of real samples, reducing dependence on real samples and improving the model's generalization ability.

[0047] Step 4 involves lightweight optimization of the trained model to improve inference speed while maintaining accuracy, and then deploying it to edge computing devices. Visual data of the assembled workpiece is acquired in real-time using a binocular camera, simultaneously obtaining equipment operating parameters (including workpiece images, point cloud data, and equipment parameters) and workpiece IDs. Outlier filtering and format standardization are performed on the multi-source data, and data is associated with workpiece IDs and timestamps, synchronizing it to the digital twin model in real time. By calibrating model parameters (such as part positions and equipment status), the virtual twin more accurately reflects the dynamic changes of the real world.

[0048] In this embodiment, model quantization and structural pruning techniques are applied to achieve lightweighting of the detection model. Model quantization converts model weights and activation values ​​from high-precision floating-point numbers to low-precision integer numbers to reduce model storage and computational load. Structural pruning removes redundant neurons, convolutional kernels, and other structures from the model based on a weight absolute value threshold, as shown in the formula: In the formula, α represents the model weight, and γ represents the preset pruning threshold. Accuracy verification is performed simultaneously during the lightweighting process to ensure the optimized model meets the requirements of real-time industrial detection. The lightweight model is deployed to an industrial edge device with strong computing power, stability, and anti-interference capabilities, employing common industrial interfaces and protocols to ensure compatibility between the model and the edge hardware. Two-dimensional images and three-dimensional point cloud data of the workpiece are simultaneously acquired using a 3D binocular camera. Equipment operating parameters and process status data are simultaneously acquired via industrial bus protocols, ensuring data timestamp synchronization during acquisition. Outliers are filtered using the 3σ criterion for the acquired multi-source data, converting data from different sources and formats into a unified format. Multi-dimensional data is associated based on the workpiece ID and timestamp, and the processed data is synchronized to the digital twin in real time, providing high-quality data for subsequent detection and analysis.

[0049] Step 5: Extract three types of features (point cloud, HOG, CNN) from the field data in Step 4, input them into the filter to generate the initial response; calculate evaluation indicators such as response peak, sidelobe ratio, and response entropy, dynamically allocate feature weights based on the indicators, and perform weighted fusion to obtain the comprehensive filtered response of the detection interest area, providing a basis for defect judgment.

[0050] like Figure 3 As shown, in this embodiment, after extracting three features and obtaining their filtered response maps, they need to be synthesized into a final comprehensive response map for defect determination. In this embodiment, three quality indicators are used to assign appropriate weights to each feature. Step 5 specifically includes:

[0051] Step 5.1: Extract point cloud features, HOG features, and CNN depth features from the real-collected images and point cloud data using the same method as in the model training phase to ensure feature consistency.

[0052] Step 5.2: Construct a multi-tiered sliding filter response generation mechanism. This involves traversing the detection region of interest through a sliding window, calculating the initial response value based on the filter response output formula, and simultaneously enhancing the response intensity of defect features using derivative transformation. A Gaussian envelope model is also employed to smooth the response curve and suppress noise interference. The filter response output formula is:

[0053]

[0054] In the formula, Q(m,n) is the response value at coordinate (m,n), p(m,n) is the pre-trained filter weight template (learned from a large number of virtual defect samples in step 3, essentially an ideal pattern that memorizes a specific defect feature), and f(m+i,n+j) is the input feature value, representing the input feature map extracted from the real workpiece image to be detected. i,j are the displacements of the filter sliding on the feature map. At each position (m,n) of the feature map, the result of multiplying the filter template p with the corresponding element of the covered local input feature f and summing the results is calculated. This value Q(m,n) is the filter response value at that position. The higher the response value Q, the more similar the texture, shape, and other features of the local area are to the defect pattern memorized by the filter, and the greater the possibility of the presence of a defect. After obtaining the original filter response map Q(m,n), a Gaussian envelope model is used to smooth it to effectively filter out high-frequency noise in the real image acquisition process, prevent misjudgment, and make the distribution of response values ​​smoother and more continuous, which facilitates accurate location of response peaks in the subsequent process.

[0055] Step 5.3 calculates three core evaluation indicators—response peak value, sidelobe ratio, and entropy—based on derivative transformation and the Gaussian envelope model. The response peak value is the maximum response value within the detection area of ​​interest. A higher peak value indicates a higher degree of matching between the feature and the defect template, suggesting a greater likelihood of a defect. The sidelobe ratio is calculated by dividing the difference between the response peak value and the average response value of the sidelobe region by the standard deviation of the sidelobe region. A larger sidelobe ratio indicates a more prominent main peak relative to background clutter, resulting in more accurate defect localization and less susceptibility to noise interference. The response entropy is calculated based on the normalized probability of the response value using the information entropy formula. A lower response entropy indicates that the response energy is concentrated in a small area (likely the defect location); a higher response entropy indicates that the response energy is dispersed. The calculation formulas for the three core evaluation indicators are as follows:

[0056] 1. Peak response: ;

[0057] 2. Side lobe ratio: , where μ is the average response value of the sidelobe region and σ is the standard deviation of the sidelobe region;

[0058] 3. Response entropy: ,in This is the normalized probability.

[0059] Step 5.4: The multi-dimensional response is integrated using a dynamic weight fusion algorithm. First, the three indicators are normalized to obtain the normalized value Sk. Then, the weight is calculated using the formula... Assign weights to each feature response, where ρk is the weight of the k-th feature and Sk is the normalized index value of the k-th feature.

[0060] Step 5.5, then the final integrated filter response Qfusion for the region of interest is obtained through weighted summation: In the formula, Qk is the response value of the k-th type of feature, and a complete comprehensive filtering response evaluation system is established.

[0061] Step 6: Preset a defect judgment threshold, compare the peak value of the comprehensive response with the threshold, determine whether the assembly is abnormal and identify the type of abnormality. Call the digital twin, input the actual assembly parameters of the abnormal workpiece to reproduce the assembly process, compare the deviation between the standard and actual parameters, combine the correlation model of parameter deviation and defects to locate the root cause of the defect, and output a traceability report containing rectification suggestions.

[0062] like Figure 4 As shown in this embodiment, the assembly defect judgment threshold is calibrated through a large number of comparative experiments with qualified and unqualified samples. During the experiment, the peak distribution of the comprehensive response of the two types of samples is statistically analyzed to determine a reasonable threshold to balance the false detection rate and the false negative rate, ensuring the reliability of the judgment result. The final comprehensive filtered response peak value is compared with the judgment threshold. If the peak value is lower than the threshold, it is judged as an assembly abnormality, and the specific type of abnormality is determined according to the correlation between the characteristic response and the defect type; if the peak value is higher than the threshold, it is judged as a qualified assembly.

[0063] When an assembly anomaly is identified, a digital twin is invoked. The actual assembly parameters of the abnormal workpiece are input into the twin model, which then reconstructs the entire assembly process according to the actual assembly sequence and logic, intuitively presenting the process and scenario in which the defect occurred. The actual assembly parameters are compared item by item with the standard assembly parameters, and the deviation rate is quantified using a parameter deviation rate formula. Calculate the degree of deviation, where, Let Qactual be the parameter deviation rate, Qactual be the actual parameter value, and Qstandard be the standard parameter value. Combining the association model established in step 1, the correspondence between deviation items and defect types is matched to pinpoint the core cause of assembly anomalies. The output is a source tracing report containing basic anomaly information, parameter deviation details, core cause analysis, and rectification suggestions, providing a clear direction for subsequent assembly parameter optimization.

[0064] Step 7: Solve for the optimal assembly parameters using a multi-objective optimization algorithm. Input the optimal parameters into the twin simulation for verification. Once the parameters meet the requirements, send them to the assembly equipment for execution. Collect new data periodically, and update the detection model and twin rules through incremental learning to achieve optimization and iteration.

[0065] like Figure 5 As shown, in this embodiment, step 7 specifically includes:

[0066] Step 7.1: Optimize assembly parameters using the NSGA-III multi-objective optimization algorithm, and construct an objective function with defect rate and negative assembly efficiency as the objectives. In the formula, p is the assembly parameter vector, D(p) is the defect rate, and E(p) is the assembly efficiency.

[0067] Step 7.2: Define optimization variables such as equipment motion parameters, process execution parameters, and assembly sequence, set constraints such as equipment rated operating range and process requirements, and obtain the optimal assembly parameter combination in the Pareto optimal solution set through algorithm.

[0068] Step 7.3: Input the optimal assembly parameters into the digital twin model, simulate the assembly process, and statistically analyze key indicators such as defect incidence rate and assembly efficiency to verify the feasibility and effectiveness of the optimization scheme. If the expected goal is not achieved, return to re-optimize.

[0069] Step 7.4: Parameter delivery supports two modes: automatic execution and manual confirmation. In automatic execution mode, parameters are directly delivered to the assembly equipment control system through the industrial communication protocol and automatically adjusted during production batch intervals. In manual confirmation mode, the parameters must be reviewed and confirmed by the operator before execution, adapting to different production management needs.

[0070] Step 7.5: Collect assembly quality and equipment operation data in real time after parameter adjustment, and collect newly added on-site defect samples and production data according to the set cycle.

[0071] Step 7.6: The incremental learning method is used to update and train the detection model based on the new data. There is no need to retrain the full set of samples. Only the model parameters are adjusted to adapt to the new scenario changes. At the same time, the association rule set and model parameters of the digital twin are iterated simultaneously to ensure that the digital twin model and the detection model can continuously adapt to the dynamic changes in the production process, forming a closed loop of detection, traceability, optimization and iteration.

[0072] The method disclosed in this embodiment performs excellently in real-world industrial scenarios, enabling precise inspection and quality assessment of complex industrial components such as circuit board connector assemblies and large metal workpiece processing features. This verifies the system's applicability and reliability across various industrial environments. The system does not rely on large-scale real-world defect samples, achieving a synergistic improvement in detection accuracy, traceability, and optimization efficiency, providing a complete technical solution for intelligent manufacturing.

[0073] The embodiments described above are merely illustrative of the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. Any modifications made to the technical solutions based on the technical concept proposed in this invention shall fall within the scope of protection of this invention.

Claims

1. A digital-twin-based visual quality detection and parameter optimization method for industrial products, characterized in that, include: Step 1: Collect data related to parts, equipment, and processes involved in assembly, and combine historical defect data to establish a correlation model between assembly parameter deviations and product defect types; build and calibrate a digital twin model, and delineate the areas of interest that need to be visually inspected in the digital twin model. Step 2: Implant target defect parameters into the digital twin model, generate multi-dimensional sample data containing defect features through high-fidelity physical rendering, and establish a virtual sample library for model training after screening. Step 3: Based on the virtual sample library, extract multi-dimensional features including geometric shape features, texture edge features, and deep semantic features; assign corresponding relevant filters to each feature of each region of interest and train them to obtain an initial detection model; fine-tune and adapt the initial detection model using a small number of real samples obtained in actual production. Step 4: Perform lightweight optimization on the model trained in Step 3 and deploy it to the industrial edge; during the operation of the production line, collect workpiece images, geometric shape data and equipment parameters simultaneously, associate and fuse the collected multi-source data, and update the digital twin model in real time. Step 5: Extract multi-dimensional features based on the multi-source data collected in Step 4, calculate the response value of each feature in the detection area through a multi-level sliding filter mechanism, and fuse the response values ​​of each feature using a dynamic weight fusion algorithm to obtain the comprehensive filter response of each region of interest. Step 6: Set the assembly defect judgment threshold, determine whether the assembly is abnormal and the type of abnormality based on the comparison result of the threshold and the comprehensive filtering response, and generate an analysis report; Step 7: Based on the analysis results in the analysis report, use a multi-objective optimization algorithm to calculate the optimized assembly parameter combination; perform simulation verification of the optimized parameter combination in a digital twin model, and after successful verification, send it to the actual production line for execution; The system continuously collects production data and updates the parameters of the detection model and the association rules in the digital twin model through incremental learning to achieve optimization and iteration.

2. The method of claim 1, wherein, In step 1, the association model is constructed using a supervised machine learning algorithm, including data partitioning, hyperparameter tuning, and cross-validation. The digital twin model is built based on 3D modeling technology and a physics engine, and is calibrated using real production data.

3. The method of claim 1, wherein, In step 2, the implanted target defect parameters are achieved by modifying the position coordinates, attribute parameters, or assembly relationships of the parts in the digital twin; the high-fidelity physical rendering is based on the material physical properties and lighting rendering model to generate multi-dimensional sample data including two-dimensional images and three-dimensional point clouds.

4. The method of claim 1, wherein, In step 3, the geometric shape features are represented by point cloud features, the texture edge features are represented by histogram of oriented gradients (HOG) features, and the deep semantic features are represented by convolutional neural network (CNN) deep features.

5. The method of claim 4, wherein, The point cloud features include global and local features. Global features include centroid and size, while local features include neighborhood point cloud distribution and normal vectors. The HOG features are extracted by statistically analyzing the gradient direction distribution of local image regions, capturing the edge and texture features of the parts. The CNN deep features are extracted using a pre-trained deep neural network model, capturing high-order features of complex defects. The correlation filter uses a kernel correlation filter (KCF), and its training formula is: In the formula, H is the filter weight, A is the cyclic matrix composed of feature samples, k is the label vector, and λ is the regularization parameter.

6. The method of claim 1, wherein, In step 4, the lightweight optimization includes model quantization and structural pruning lightweight processing; wherein, the model quantization converts high-precision floating-point weights and activation values ​​into low-precision integers; the structural pruning is determined based on a weight absolute value threshold, using the following formula: In the formula, α is the model weight and γ is the preset pruning threshold.

7. The method according to any one of claims 1 to 6, characterized in that, In step 5, the dynamic weight fusion algorithm includes: Calculate the peak response values ​​for various features: Side lobe ratio: Response entropy: In the formula, Q(m,n) is the filtered response value at coordinates (m,n), μ is the average response value of the sidelobe region, σ is the standard deviation of the sidelobe region, and ROI is the region of interest. Normalized probability; The dynamic weights are calculated based on the index normalization results, using the following formula: In the formula, ρk is the weight of the k-th feature, Sk is the normalized index value of the k-th feature, and the final comprehensive filter response Qfusion is obtained by weighted summation: In the formula, Qk is the response value of the k-th feature.

8. The method of claim 7, wherein, The filter response value Q(m,n) is expressed as: In the formula, p(m,n) is the filter weight, f(m+i,n+j) is the input feature value, and the derivative transformation is combined to enhance the defect feature response, and the Gaussian envelope model smooths the curve and suppresses noise.

9. The method of claim 1, wherein, In step 6, the assembly defect judgment threshold is calibrated through a comparative experiment of qualified and unqualified samples; the parameter deviation rate is calculated using the following formula: In the formula, Qactual represents the parameter deviation rate, Qstandard represents the actual parameter value, and Qstandard represents the standard parameter value. When an anomaly is detected, the digital twin model from step 1 is invoked to reproduce the assembly process of the workpiece. By comparing the deviation between the standard assembly parameters and the actual collected parameters, the cause of the defect is located, and an analysis report is generated.

10. The method of claim 1, wherein, In step 7, the multi-objective optimization algorithm selected is the NSGA-III algorithm, and the objective function is constructed as follows: In the formula, p is the assembly parameter vector, D(p) is the defect rate, and E(p) is the assembly efficiency; the optimization variables include equipment motion parameters, process execution parameters, and assembly sequence, and the constraints include the rated operating range of the equipment and process requirements.

Images