Remote control method and system based on gold wire bonding equipment

Abstract

The invention relates to the technical field of remote control, and discloses a remote control method and system based on gold wire bonding equipment, and the method comprises the steps: obtaining a high-precision multi-dimensional detection data set; processing the multi-modal original monitoring data to obtain a gold thread quality feature vector matrix; processing the multi-dimensional feature vector set to obtain unified quality index feature mapping; processing the uniform quality index feature mapping to obtain a multi-stage defect detection report; processing the historical quality data and the process parameters to obtain a parameter optimization suggestion report; processing the multi-source monitoring data to obtain an augmented reality interaction system; and the augmented reality interaction system receives an operation decision of a remote operator and remotely controls the gold wire processing equipment. By means of the remote control method and the high-precision remote control system based on the gold wire bonding equipment, millisecond-level response control over gold wire machining equipment is achieved, the instruction execution accuracy is improved, and operation delay is reduced; and potential quality problems are warned in advance.

Description

Technical Field

[0001] The present invention relates to the field of remote control technology, and more specifically, to a remote control method and system based on a gold bonding wire device. Background Art

[0002] With the continuous miniaturization and high integration of integrated circuits and semiconductor devices, gold wire used for chip bonding is a key interconnection material, and its quality control has a decisive impact on the reliability and performance of electronic products. The diameter of gold wire is usually between 15-25μm, and even thinner is required in high-frequency electronic products, which puts extremely high demands on quality control during the processing.

[0003] Traditional gold wire processing quality control mainly relies on the following methods: Offline sampling inspection: Destructive or non-destructive testing is performed by extracting samples, such as tensile testing, metallographic analysis, etc. This method cannot achieve 100% product inspection, and is prone to missing local or random defects. In addition, the inspection results lag behind the production process, making it difficult to intervene in time.

[0004] Single-mode online inspection: For example, relying solely on optical imaging technology for surface defect detection, or using only resistance measurement to evaluate the conductive properties of gold wires. Although such methods can monitor in real time, they have obvious limitations.

[0005] Empirical parameter control: Relying on experienced operators to adjust process parameters based on experience. This method is greatly influenced by human factors and is difficult to standardize. Moreover, with the increase in process complexity and personnel turnover, it is impossible to ensure a sustained and stable level of quality control. Summary of the invention

[0006] The present invention provides a remote control method and system based on a gold bonding wire device to solve the above technical problems.

[0007] Processing gold wire machining process parameters through distributed multimodal sensor networks to obtain high-precision multidimensional inspection data sets; The multi-modal raw monitoring data is processed by a multi-level feature extraction algorithm to obtain the gold wire quality feature vector matrix; The multi-dimensional feature vector set is processed by an attention-enhanced deep fusion network to obtain a unified quality indicator feature map; The unified quality indicator feature map is processed by a hierarchical deep classification network to obtain a multi-level defect detection report; Process historical quality data and process parameters through the time series prediction network to obtain parameter optimization recommendation reports; Processing multi-source monitoring data through a spatial mapping engine to obtain an augmented reality interactive system; The augmented reality interactive system receives the remote operator's operational decisions and remotely controls the gold wire processing equipment.

[0008] Further, in the step of processing the gold wire processing parameters through a distributed multimodal sensor network to obtain a high-precision multidimensional detection data set: the multimodal sensor network includes a high-resolution optical imaging system, an acoustic sensing system, and a thermal imaging system; The high-resolution optical imaging system uses a microscopic lens array to acquire surface topography images. in Indicates the spatial position, Indicates a time point; the acoustic sensing system uses a piezoelectric sensor array to obtain acoustic characteristics during the processing in Indicates frequency; The thermal imaging system uses an infrared array sensor to obtain the temperature distribution during the gold wire processing process. .

[0009] Furthermore, the step of processing the multi-modal original monitoring data by a multi-level feature extraction algorithm to obtain a gold wire quality feature vector matrix includes: Apply adaptive threshold segmentation and edge detection algorithms to optical images to extract surface morphology feature vectors The formula is: ; represents the adaptive threshold segmentation and edge detection algorithm; Apply short-time Fourier transform and wavelet analysis to the acoustic signal to extract the acoustic feature vector The formula is: ; represents short-time Fourier transform and wavelet analysis; Apply temperature gradient analysis and thermal anomaly detection algorithms to thermal images to extract thermal feature vectors The formula is: ; represents the temperature gradient analysis and thermal anomaly detection algorithm; Forming a multimodal feature set: ; They represent surface morphology feature vector, acoustic feature vector, thermal feature vector and multimodal feature set respectively.

[0010] Furthermore, the step of processing the multidimensional feature vector set through the attention-enhanced deep fusion network to obtain a unified quality indicator feature map includes: Using the gold wire quality eigenvector matrix Building an Attention-Enhanced Deep Fusion Network Its structure consists of a modality-specific encoder, a cross-modal attention module, and a fusion decoder connected in series: Modality specific encoders Both use a fully connected neural network structure, each encoder contains 3 hidden layers and uses the ReLU activation function; Cross-modal attention module Use scaled dot product attention calculation method: are query, key, and value matrices respectively; Fusion Decoder A two-layer fully connected neural network is used to map the attention-weighted features into a unified feature space; Encode each modal feature independently: ; ; ; Apply the cross-modal attention module: ; in Indicates that from the modal To Modal The attention weight matrix of Fusion of feature representations from all modalities: ; Feature representation after fusion Contains complementary information of all modes.

[0011] Furthermore, the steps of processing the unified quality indicator feature map through the hierarchical deep classification network to obtain a multi-level defect detection report include: Build a multi-class defect classifier: ; in is the defect category label, is the confidence level, is the unified feature representation output from step 3, are model parameters; Defect area location: ; in is the defect area mask, A region localization function based on the gradient class activation map.

[0012] Quality score calculation: ; in Represents the overall quality score, is the quality scoring function;

[0013] Risk level classification: ; in The thresholds are critical, warning, and caution, respectively.

[0014] Furthermore, the step of processing historical quality data and process parameters through a time series prediction network to obtain a parameter optimization recommendation report includes: performing time series preprocessing on quality scores and process parameters in the production process; Training long short-term memory network prediction models; Predict future quality trends; Generate parameter optimization suggestions through a parameter search function based on Bayesian optimization.

[0015] Furthermore, the method further comprises the following steps: Process the data of the entire material processing process through the full-process Internet of Things system to obtain a multi-dimensional process tracking database; The multi-dimensional process tracking data is processed through a multi-physics field coupling model to obtain a digital twin system for gold wire processing.

[0016] Furthermore, the step of processing multi-dimensional process tracking data through a multi-physics field coupling model to obtain a gold wire processing digital twin system includes: Construct the material microstructure evolution function to simulate the microstructure changes of the gold wire during processing. The formula is: ; in For time The microstructural state of The process parameters, is the time step; Construct a multi-physics field coupling function to realize the interactive simulation of thermal field, force field and material field. The formula is: ; in For time The physical field state, including temperature field, stress field, and deformation field; A performance prediction function is constructed to map the microstructure and physical state to the gold wire performance. The formula is: ; in It is the performance index of gold wire, including tensile strength (MPa), elongation (%), conductivity (MS / m), and welding performance score; Construct a process-quality mapping function to directly link process parameters and quality evaluation. The formula is: ; in It is a quality evaluation indicator.

[0017] Furthermore, the method further comprises the following steps: Process quality anomaly information through the causal inference engine to obtain multi-dimensional defect traceability reports; The virtual parameter test function is used to simulate the impact of different parameter combinations on quality. The formula is: ; in is the set of quality indicators for simulation, is the parameter combination to be tested, is the process constraint, Test function for dummy parameters, To establish a digital twin model.

[0018] A remote control system based on a gold bonding wire device, comprising: In the first module, the gold wire processing parameters are processed through a distributed multimodal sensor network to obtain a high-precision multidimensional detection data set; The second module processes the multi-modal raw monitoring data through a multi-level feature extraction algorithm to obtain the gold wire quality feature vector matrix; The third module processes the multi-dimensional feature vector set through the attention-enhanced deep fusion network to obtain a unified quality indicator feature map; The fourth module processes the unified quality indicator feature mapping through a hierarchical deep classification network to obtain a multi-level defect detection report; The fifth module processes historical quality data and process parameters through a time series prediction network to obtain a parameter optimization recommendation report; The sixth module processes multi-source monitoring data through a spatial mapping engine to obtain an augmented reality interaction system.

[0019] The beneficial effects of the present invention are: The multi-modal gold wire processing quality monitoring system can capture microscopic defects and evaluate processing quality in real time, with detection accuracy reaching micron level; it can warn of potential quality problems in advance with improved accuracy; The augmented reality interactive interface enables remote operators to intuitively understand complex micro-quality data, improve decision-making efficiency, and reduce operational errors; The gold wire processing quality management system with full traceability supports the quality tracking of the entire life cycle from raw materials to finished products, and improves the accuracy of defect attribution; Intelligent process parameter optimization suggestion system can automatically generate optimization parameters based on historical data and current status, and improve the qualification rate of gold wire products; The high-precision remote control system achieves millisecond-level response control of gold wire processing equipment, improves instruction execution accuracy, reduces operation delays, and maintains stable control effects even under network fluctuations, effectively solving delay and accuracy issues in remote operations.

[0020] Virtual process verification platform, which supports testing new process parameters in a digital environment, reduces physical testing, and accelerates process optimization cycles; The design-manufacturing collaborative optimization system helps consider manufacturing factors during the product design phase, shortens the product design optimization cycle, and improves the first-time success rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 is a flow chart of a remote control method based on a gold bonding wire device of the present invention; Figure 2 It is an example of the deployment of sensor equipment on the gold wire processing production line of the present invention; Figure 3 is an example of data processing of the present invention; Figure 4 is an example of the defect detection result of the present invention; Figure 5 It is an example of process parameter recording of the present invention. DETAILED DESCRIPTION

[0022] Embodiment 1, at least one embodiment of the present invention discloses a remote control method based on a bonding wire device, such as Figure 1 As shown, the following steps are included: Step 1: Process the gold wire processing parameters through a distributed multimodal sensor network to obtain a high-precision multidimensional detection data set; A multimodal sensor network including high-resolution optical imaging, acoustic sensing and thermal imaging is constructed. These sensors are installed at key process nodes of gold wire processing, such as Figure 2 As shown: High-resolution optical imaging system: using microscopic lens array Indicates Microscopic lens with a resolution of 10μm to obtain surface topography images ,in Indicates the spatial position, Indicates a point in time; Acoustic Sensing Systems: Using Piezoelectric Sensor Arrays Acoustic signature of the machining process in Indicates frequency, with a sampling rate of 1MHz, capable of capturing acoustic emission signals generated by tiny cracks; Thermal imaging system: using infrared array sensor Obtaining the temperature distribution during wire processing The temperature resolution reaches 0.1°C and the spatial resolution is 50μm; Each sensor module uses a real-time data transmission protocol to send the collected data according to a unified timestamp. and space markers Labeling is performed to form a high-precision multidimensional detection dataset.

[0023] Step 2: Process the multi-modal raw monitoring data through a multi-level feature extraction algorithm to obtain the gold wire quality feature vector matrix; Preprocess and extract features of the high-precision multidimensional detection data set obtained in step 1, such as Figure 3 As shown: Optical image processing: The image obtained in step 1 Apply adaptive threshold segmentation and edge detection algorithm to extract surface morphology feature vector

[0024] ; in Represents the optical feature extraction function, which is implemented as a multi-scale convolutional neural network, including convolutional layers, pooling layers, and feature descriptor calculation operations. The input is the original image and the output is a 128-dimensional feature vector.

[0025] Acoustic signal processing: The acoustic signal obtained in step 1 Apply short-time Fourier transform (STFT) and wavelet analysis to extract acoustic feature vectors : ; in It is an acoustic feature extraction function, which is implemented as follows: First, STFT is applied to obtain the time-frequency spectrum in is the window function; then the features of different frequency bands are extracted through wavelet transform; finally, a 64-dimensional feature vector is generated using an adaptive feature selection algorithm.

[0026] Thermal image processing: The thermal image obtained in step 1 Apply temperature gradient analysis and thermal anomaly detection algorithms to extract thermal feature vectors

[0027] ; in It is a thermal feature extraction function, which is implemented as follows: first calculate the spatial temperature gradient Then identify the abnormal temperature gradient area; finally combine the time series temperature change rate Construct a 96-dimensional feature vector.

[0028] All feature vectors are organized according to unified temporal and spatial labels to form a multimodal feature set: ; Step 3: Process the multi-dimensional feature vector set through the attention-enhanced deep fusion network to obtain a unified quality indicator feature map; Using the gold wire quality feature vector matrix obtained in step 2 Building an attention-enhanced deep fusion network: Designing a multimodal deep fusion network Its structure includes: Modality specific encoders All use a fully connected neural network structure, each encoder contains 3 hidden layers (dimensions are 64, 32, and 16 respectively), and uses the ReLU activation function; Cross-modal attention module Use scaled dot product attention calculation method: ; in, are query, key, and value matrices respectively; Fusion Decoder A two-layer fully connected neural network (hidden layer dimension is 32) is used to map the attention-weighted features into a unified feature space; Encode each modal feature independently: ; in Indicates that from the modal To Modal The attention weight matrix.

[0029] Fusion of feature representations from all modalities: ; Feature representation after fusion It contains complementary information of all modes, providing a more comprehensive data basis for defect detection and quality assessment.

[0030] Step 4: Process the unified quality indicator feature map through a hierarchical deep classification network to obtain a multi-level defect detection report; Based on the unified quality indicator feature mapping obtained in step 3, a defect detection and quality assessment system is constructed: Multi-class defect classifier construction: ; in is the defect category label, is the confidence level, is the unified feature representation output from step 3, is the model parameter.

[0031] Defect area location: ; in is the defect area mask, It is the region localization function based on Gradient Class Activation Mapping (Grad-CAM).

[0032] Quality score calculation: ; in Represents the overall quality score, It is a quality scoring function that takes into account factors such as defect type, severity, and distribution.

[0033] ; in They are the critical, warning and attention thresholds, which can be adjusted according to specific application scenarios.

[0034] Through the above content, the defect detection results can be obtained, such as Figure 4 As shown, it includes sample ID, detected defect type, confidence level, defect area coordinates, quality score, and risk score.

[0035] Step 5: Process historical quality data and process parameters through the time series prediction network to obtain a parameter optimization recommendation report; Build a prediction system based on LSTM to achieve real-time prediction and parameter optimization of gold wire processing quality issues: Time series preprocessing: Perform time series preprocessing on quality scores and process parameters in the production process: ; in are the quality score sequence and process parameter sequence after preprocessing, is a preprocessing function, such as Figure 5 As shown in the figure, it specifically includes time points, high-frequency induction furnace temperature, homogenization treatment time, rolling pressure, wire drawing speed and quality score, including data alignment, outlier processing, missing value interpolation and standardization. The sliding window Z-score method is used for time series data standardization to maintain the time dependence of the sequence.

[0036] Train a Long Short-Term Memory (LSTM) prediction model: ; Predict future quality trends: ; in Represents a sequence of predicted future quality scores.

[0037] Generate parameter optimization suggestions: ; in is the recommended optimal process parameter set, is the parameter search function based on Bayesian optimization, Rate the quality of the target.

[0038] Step 6: Process the multi-source monitoring data through the spatial mapping engine to obtain an augmented reality interaction system; Based on the multi-level defect detection report in step 4 and the parameter optimization recommendation report in step 5, an augmented reality interaction system is constructed: 3D space mapping: construct the actual wire processing environment into a 3D space model And establish a mapping relationship with the physical space: ; Interactive control interface: designing gesture recognition functions and voice command recognition functions Realize natural interaction between operators and systems: ; in Represent the user's gesture and voice input respectively, Interactive commands are explained.

[0039] Feedback response system: Execute corresponding operations according to interactive commands, such as zooming in on a specific area, adjusting parameters, recording problems, remotely controlling gold wire processing equipment, etc.: ; in For system response, The response function is implemented as a response system based on the event-state mechanism, which receives interactive commands, performs corresponding operations and updates the AR interface according to the current system status and available data.

[0040] Step 7: Process the remote operator's operation decision and remotely control the gold wire processing equipment; Device control interface mapping: Construct the mapping relationship between device parameters and interactive control interfaces: ; in It is a set of process parameters, including key parameters such as temperature, pressure, speed, tension, etc. is the corresponding device control command set. This mapping is implemented through the industrial control protocol and supports bidirectional data transmission.

[0041] Intelligent execution scheduling: Intelligent scheduling of control instructions based on the current status of the equipment and production plan: ; in is the instruction sequence that is finally executed, is the scheduling function, is the current status of the device, Intelligent scheduling ensures that control instructions are executed at the most appropriate time to avoid interfering with the normal production rhythm, while ensuring the real-time nature of operation instructions.

[0042] The technical effects of this embodiment are as follows: Multi-modal gold wire processing quality monitoring system, which can capture microscopic defects and evaluate processing quality in real time, with detection accuracy reaching micron level; The predictive quality analysis system based on deep learning can provide early warning of potential quality problems with improved accuracy; The augmented reality interactive interface enables remote operators to intuitively understand complex micro-quality data, improve decision-making efficiency, and reduce operational errors; The gold wire processing quality management system with full traceability supports the quality tracking of the entire life cycle from raw materials to finished products, and improves the accuracy of defect attribution; Intelligent process parameter optimization suggestion system can automatically generate optimization parameters based on historical data and current status, and improve the qualification rate of gold wire products; The high-precision remote control system achieves millisecond-level response control of gold wire processing equipment, improves instruction execution accuracy, reduces operation delays, and maintains stable control effects even under network fluctuations, effectively solving delay and accuracy issues in remote operations. Example 2

[0043] It is difficult to quickly trace the specific cause of gold wire quality problems after they occur. They may come from multiple links such as raw materials, processing technology or equipment status; Traditional quality control is mainly based on testing, lacking accurate modeling of the relationship between material properties, process parameters and final performance; It is difficult to accurately predict the impact of different design parameters on gold wire performance during the product design phase, resulting in long cycles of repeated trials and optimization; In complex multi-process manufacturing processes, the quality transfer relationship between each process is unclear, making it difficult to achieve full process optimization; The method of this embodiment includes the following steps: Step 2-1: Processing gold wire processing parameters through a distributed multimodal sensor network to obtain a high-precision multidimensional detection data set; Same as Example 1, except for step contents omitted. (Reason for omission: this step is exactly the same as Example 1 and does not need to be modified); Step 2-2: Process the data of the entire material processing process through the full-process IoT system to obtain a multi-dimensional process tracking database; Based on Example 1, the full-process data collection capability is added: Raw material characteristic data collection: deploying raw material detection system Collect the characteristic data of gold wire raw materials: ; in It is a data set of raw material characteristics, including key indicators such as chemical composition, lattice structure, impurity content, initial hardness, etc. For raw material samples.

[0044] The detection system includes X-ray fluorescence analyzer (XRF), scanning electron microscope (SEM), energy dispersive spectrometer (EDS) and hardness tester, etc., to form a multi-dimensional material property description.

[0045] Full-process collection of process parameters: Building a process parameter collection network Covering the entire process of gold wire processing: ; in is the process parameter data set, It is a set of parameters for all processes, including: Alloying parameters, homogenization parameters, rolling parameters, wire drawing parameters, heat treatment parameters; Environmental status monitoring: recording environmental status data : Equipment Condition Monitoring: Building an Equipment Condition Monitoring System Tracking equipment operating status: ; in is the device status dataset, The equipment status includes key component temperature, vibration characteristics, power consumption, wear status and other indicators.

[0046] Material tracking system: realize material tracking of the entire process of wire processing and record material trajectory.

[0047] Step 2-3: Process multi-dimensional process tracking data through a multi-physics field coupling model to obtain a gold wire processing digital twin system; Based on the multimodal sensor network deployed in step 1 and the raw material characteristic data collected in step 2 Environmental status data and device status data Building a digital twin model of wire machining: Material microstructure evolution model: Constructing material microstructure evolution function Simulate the microstructural changes of gold wire during processing: ; in For time The microstructural state (including grain size, orientation, phase distribution, dislocation density, etc.), process parameters (temperature, stress, strain rate, etc.), is the time step (usually milliseconds to seconds).

[0048] The microstructure evolution model is based on the phase field method and cellular automaton to simulate processes such as grain growth, phase change and dislocation evolution. The phase field method is a computational method for simulating the evolution of material microstructure, and its core equation is: ; in is the order parameter (indicating the distribution of different phases or grains in the material), is the atomic mobility (a temperature-dependent parameter that usually follows the Arrhenius relation in is the activation energy, is the gas constant, is absolute temperature), is the system free energy (including chemical free energy, elastic free energy and interface energy), is the variational derivative (representing the driving force for the system to minimize free energy).

[0049] Multiphysics coupling simulation: Building multiphysics coupling functions Realize interactive simulation of thermal field, force field and material field: ; in For time The physical field states, including temperature field, stress field, deformation field, etc., are directly affected by the physical field states in step 2-2. impact.

[0050] The multi-physics field coupling uses the finite element method (FEM) to build a thermal-mechanical-material multi-field coupling model to simulate physical phenomena such as temperature distribution, stress distribution, and plastic deformation during the processing process. Gold wire macro performance prediction model: Construct performance prediction function Mapping microstructure and physical state to gold wire properties: ; in are gold wire performance indicators, including tensile strength (MPa), elongation (%), conductivity (MS / m), welding performance score, etc. This function is trained using the real performance data collected in step 2.

[0051] The performance prediction uses a structure-performance relationship model based on the obtained microstructural characteristics. and macroscopic physical states To predict the final performance of the gold wire, the model structure is a hierarchical neural network, which contains multiple professional sub-networks: Grain structure-mechanical properties subnetwork: 4-layer fully connected network (512-256-128-64 neurons), input is grain size, orientation, boundary characteristics, etc., output is yield strength, tensile strength, elongation, etc.; Phase distribution-electrical performance subnetwork: 3-layer fully connected network (256-128-64 neurons), input is phase composition, distribution, content, etc., output is resistivity, temperature coefficient, etc.; Defect distribution-reliability subnetwork: 3 layers of fully connected networks (256-128-64 neurons) plus 1 LSTM layer (128 units), input is defect type, density, distribution, etc., as well as the timing information of steps 1 and 2, and output is fatigue life, thermal cycle stability, etc.; Process-quality mapping model: Constructing process-quality mapping function Directly link process parameters and quality evaluation: ; in is the quality evaluation index (qualified rate, defect rate, consistency score, etc.). This function directly uses all the parameters collected in step 2-2 as input.

[0052] The mapping model adopts a hybrid method of deep learning combined with a mechanism model, which is specifically implemented as follows: Feature engineering layer: preprocessing and feature extraction of the raw data from step 2-2, including statistical features, frequency domain features, and expert-defined features; Deep network layer: It adopts the deep residual network (ResNet) structure, which contains 20 residual blocks, each of which contains two convolutional layers and one skip connection; Mechanism constraint layer: Integrates physical constraints (such as material laws, thermodynamic principles, etc.) to ensure that model predictions conform to physical laws; Quality prediction layer: multi-task learning head, predicting multiple quality indicators simultaneously; Digital twin model integration and calibration: Design model integration and calibration functions Integrate all the above sub-models and calibrate them: ; in For the integrated digital twin model, It is the actual production data collected in step 1 and step 2, which is used for model calibration.

[0053] The model calibration adopts Bayesian optimization method to adjust the model parameters by minimizing the following objective function: ; in is the model parameter set, is the model prediction value, is the measured value, is the distance function (such as mean square error, relative error, etc.), is the weight coefficient (reflecting the importance of different outputs).

[0054] The calibration process is automated and consists of the following steps: Parameter sensitivity analysis: Identify the subset of parameters that most significantly affect model output (usually 10% to 20% of key parameters); Initial space sampling: Use Latin hypercube sampling method to select initial points in parameter space; Bayesian optimization iteration: Establish a parameter-error proxy model based on Gaussian process regression to guide the sampling of new parameter points; Incremental update: When new production data is generated, the model parameters are automatically updated to ensure the synchronization of the digital twin model with the actual production process; The digital twin model constructed in this step can accurately reproduce the entire process of cash flow machining in a virtual environment and realize the material characteristics. Environmental conditions The correlation mapping with the final performance provides basic model support for subsequent quality prediction.

[0055] Step 2-4: Process quality anomaly information through the causal inference engine to obtain a multi-dimensional defect traceability report; Digital twin system built based on steps 2-3 Design a quality traceability engine to quickly trace quality defects to their root causes: Quality anomaly detection and localization: Building anomaly detection functions Real-time identification and location of gold wire quality anomalies: ; in Indicates the type of anomaly (such as broken wires, cracks, holes, etc., represented by integer codes), represents the abnormal location (expressed in three-dimensional coordinates), is the real-time data collected by the multimodal sensor in step 1, Set the anomaly detection threshold (configurable according to different quality level requirements).

[0056] Anomaly detection uses a combination of algorithms, including: Statistical process control (SPC): Calculate control limits based on a piecewise steady-state process, using the formula in is the mean, is the standard deviation, is the control coefficient (usually 3), which is suitable for stable process parameter monitoring; Deep anomaly detection network: It uses a Transformer-based sequence anomaly detection model with an encoder-decoder architecture (8 attention heads, 6 encoder and decoder layers), identifies anomalies through reconstruction errors, and is suitable for high-dimensional sensor data. Transformer is a deep learning model based on the self-attention module that can effectively process sequence data and capture long-distance dependencies.

[0057] Expert rule system: A decision tree rule base built based on domain knowledge, containing about 200 judgment rules for different abnormal patterns, suitable for rapid identification of known abnormal types; Reverse reasoning engine: design reverse reasoning function From abnormal phenomena to possible causes: ; in is a set of possible causes (including cause category, location, time and credibility), The digital twin model built for steps 2-3, It is historical data (including all historical parameter records in step 2-2).

[0058] Reverse reasoning is based on Bayesian networks and causal inference models to construct a model based on the detected quality anomaly types. Probabilistic mapping to potential causes. Key implementations include: Cause-and-effect diagram construction: Using the mechanism knowledge of the digital twin model in steps 2-3, combined with expert experience and data mining, a process-quality cause-and-effect diagram with 1,000 nodes and 5,000 edges is constructed; Bayesian inference: Calculate the posterior probability based on the causal graph, the formula is: ; in For a given exception Conditional reason The posterior probability of is the likelihood term (obtained through the digital twin model in steps 2-3), is the prior probability (based on historical data statistics); Time series causal analysis: Considering the time series relationship of parameter changes, Granger causality test and dynamic time warping (DTW) algorithm are used to identify the key time points and parameter changes that cause anomalies. Granger causality test is a statistical method used to determine whether one time series is helpful in predicting another time series; DTW algorithm is a method for measuring the similarity of two time series, which can handle sequences of different lengths and different rates.

[0059] Reverse simulation verification: Use the digital twin model in steps 2-3 to perform reverse simulation to verify whether the inferred cause can reproduce the observed anomaly in the model; Multidimensional cause analysis and sorting: Building a cause analysis function Comprehensively analyze and sort the possible reasons: ; in is a ranked list of reasons (with details and confidence scores), is the set of inferred causes, is the weight vector of different factors (configurable, reflecting the focus of the enterprise).

[0060] The analysis process uses a combination of decision trees and rule engines, including: Multi-dimensional impact assessment: Evaluate the impact of each possible cause in terms of quality, cost, time, and repairability, using a quantitative score of 0-10; Confidence calculation: Based on the completeness of the evidence chain, historical similar cases and model verification results, the confidence score of each cause is calculated, ranging from 0 to 1; Comprehensive ranking: The reasons are ranked based on weighted scores. The scoring formula is: ; in For the reason The confidence level, For the dimension The impact score on is the weight of this dimension; Cause correlation analysis: Identify the correlation between causes, build a cause map, and avoid repeated processing of multiple related causes; Tracing path visualization: Design tracing path visualization function Visually display the traceability path from anomalies to root causes: ; in To visualize the results, is the sorted list of reasons, For material tracking information, Rebuild the path.

[0061] Visualizations use interactive cause-and-effect diagrams and sequential flow charts, including: Causal chain visualization: Displays the propagation path from the root cause to the observed anomaly in a directed graph, with nodes representing physical states and edges representing causal relationships; Time evolution animation: Reconstruct the anomaly formation process based on the digital twin model, and show in chronological order how changes in key parameters lead to the final anomaly; Multi-level interaction: supports drilling down from macro process flow to micro physical process and then to material microstructure, providing a full-level traceability view; Evidence chain presentation: For each inferred reason, the evidence chain supporting the conclusion is presented, including sensor data, model predictions, and historical cases; Accumulation of traceability knowledge base: Constructing traceability knowledge base function Continue to accumulate traceability experience and knowledge: ; in For the updated traceability knowledge base, is the previous knowledge base, For verified actual reasons (confirmed by the Engineer).

[0062] The knowledge base uses a graph database structure (such as Neo4j). The specific implementation includes: Case vectorization: Each anomaly-cause pair is converted into a feature vector to facilitate similar case retrieval. The bi-encoder model is used for vectorization. Knowledge graph construction: Store entities such as anomalies, causes, process parameters, and environmental status and their relationships in a graph format, including attributes and timestamps; Incremental learning mechanism: based on new cases and user feedback, continuously update the reverse reasoning model and optimize the causal graph structure; Knowledge generalization: extract general rules from specific cases, enrich the expert rule base, and improve the ability to handle new anomalies; The quality traceability engine implemented in this step directly uses the digital twin model of steps 2-3 Perform reverse reasoning and simulation verification, using the sensor data from step 2-1 Perform anomaly detection, combined with the material tracking information in step 2-2 Path reconstruction can quickly trace possible causes when gold wire quality anomalies are detected, greatly shortening the time for problem location and resolution, and providing a verified causal mechanism for subsequent quality predictions.

[0063] Step 2-5: Gold wire quality prediction and virtual verification system; Using the digital twin model established in steps 2-3 And the traceability knowledge base accumulated in steps 2-4 , build a gold wire quality prediction and virtual verification system: Quality trend prediction: Constructing quality trend prediction function Predict the quality change trend in the future: ; in For the predicted quality trend (including multiple quality indicators and their fluctuation ranges in the time series), The digital twin model built for steps 2-3, is the current process parameter set, For the accumulated traceability knowledge base, is the forecast time range (configurable, usually 8 hours to 7 days).

[0064] The prediction algorithm uses a hybrid model approach, which is implemented as follows: Physical Model Prediction: Leveraging Digital Twins Simulate physical processes and calculate future quality change trends, suitable for short-term prediction (within 8 hours); Time series deep learning prediction: The LSTM network (Long Short-Term Memory, a recursive neural network that specializes in dealing with long sequence dependency problems, containing 128 hidden units and a 4-layer stacked structure) enhanced by the attention module is used to capture the long-term dependency of historical quality data: ; ; in The hidden state of the moment, is the unit state at the previous moment, Input at the moment (including current process parameters and environmental data), is the predicted output.

[0065] Uncertainty quantification: Monte Carlo random dropout (MCDropout) technology is used to keep the random dropout layer of the neural network turned on during the inference phase, and the confidence interval of the prediction is calculated through multiple sampling (usually 100-200 times): ; in is the sampling number, For the The predicted value of the subsample, Setting the warning level for the model's inherent noise variance anomaly: Setting warning thresholds based on the uncertainty interval of the prediction, divided into three risk levels (low, medium, and high), where the risk level is determined by the probability of exceeding the quality control limit (low: 5%-25%, medium: 25%-50%, high: >50%); Virtual parameter testing: Design virtual parameter testing functions Simulate the impact of different parameter combinations on quality: ; in is the set of quality indicators for simulation, is the parameter combination to be tested, is the process constraint.

[0066] The virtual test system implementation details include: Parameter combination generation: Latin Hypercube Sampling (LHS) is used to evenly generate test points in the parameter space, which reduces the number of sampling points by 60% to 80% compared to traditional grid search while maintaining coverage. Latin Hypercube sampling is a statistical method that ensures that the sample points in the parameter space are evenly distributed and representative.

[0067] Parallel simulation execution: Based on the Ray distributed computing framework, parallel simulation of large-scale parameter combinations is realized, supporting testing of >10,000 parameter combinations per hour, with a speedup ratio of 20-40 times that of a single thread; Adaptive sampling: Build a proxy model based on Gaussian Process Regression, dynamically adjust parameter sampling strategies, and increase sampling density in quality-sensitive areas: ; in is the mean function, is the kernel function (using the Matérn 5 / 2 kernel, which can handle non-smooth characteristics in parameter space). Gaussian process regression is a non-parametric Bayesian method that can model the probability of a function and quantify the uncertainty of the prediction.

[0068] Optimal parameter recommendation: building parameter optimization function : ; in is the recommended optimal parameter set, is the target quality indicator, is a constraint condition.

[0069] The optimization algorithm uses Bayesian optimization combined with particle swarm optimization to search for the best parameter combination that meets the target quality requirements in a virtual environment, taking into account constraints such as equipment capacity range and production cost.

[0070] Design-Manufacturing Co-Optimization: Design Co-Optimization Function

[0071] ; in are the optimized design parameters, To match the process parameters, For performance requirements, A historical knowledge base.

[0072] Collaborative optimization supports the consideration of manufacturing factors in the design phase, achieving joint optimization of design parameters and process parameters, improving gold wire performance while ensuring manufacturability.

[0073] Virtual experiment interactive system: designing virtual experiment interface : ; in For the experimental interface, For the experimental setup, For the simulation results, For comparison options.

[0074] The virtual experiment system supports experimental design, parameter adjustment, result preview and scheme comparison, allowing operators to test the effects of new process parameters without interrupting production.

[0075] Step 2-6: Build an augmented reality interactive interface; Full life cycle data visualization: Design full life cycle visualization function : ; in To visualize the results, For material tracking information, The whole process data collected and generated for steps 2-1 to 2-5, is the view type.

[0076] Visualization supports timeline browsing, process switching, and multi-dimensional data linkage analysis, intuitively displaying the complete state evolution process of gold wire from raw materials to finished products.

[0077] Digital Twin Model Interaction: Building Model Interaction Functions : ; in For interactive response, For operation instructions, is the current view.

[0078] The interactive system supports direct manipulation of digital twin models, such as parameter adjustment, cross-section viewing, microstructure magnification, etc., to achieve multi-scale exploration from macro to micro.

[0079] Multidimensional Quality Maps: Designing Quality Map Functions : ; in For quality map view, For all quality data, For filtering conditions, for the level of detail.

[0080] The quality map intuitively displays the quality distribution under different process parameter combinations through heat maps, contour lines and color coding, helping operators understand the complex relationship between parameters and quality.

[0081] Exception tracing interactive interface: Build a tracing interactive interface : ; in For the traceability interface, is the selected exception.

[0082] The traceability interface supports reverse exploration along the causal chain starting from abnormal phenomena, unfolding possible causes layer by layer, providing evidence support and case comparison, and assisting operators in conducting systematic analysis.

[0083] Virtual experiment interactive system: designing virtual experiment interface : ; in For the experimental interface, For the experimental setup, For the simulation results, For comparison options.

[0084] The virtual experiment system supports experimental design, parameter adjustment, result preview and scheme comparison, allowing operators to test the effects of new process parameters without interrupting production.

[0085] The technical effects of this embodiment are as follows: The digital twin system for the entire process of gold wire processing, based on the multi-physics field coupling model constructed in steps 2-3, realizes virtual simulation of the entire process from raw materials to finished products, improves simulation accuracy, and meets industrial-grade application requirements; The intelligent traceability system for quality anomalies shortens the root cause analysis time of quality problems from the traditional hours to minutes, and improves the accuracy of root cause identification; Virtual process verification platform, which supports testing new process parameters in a digital environment, reduces physical testing, and accelerates process optimization cycles; Design-manufacturing collaborative optimization system helps consider manufacturing factors during the product design phase, shortens the product design optimization cycle, and improves the first-time success rate; High-precision remote control system achieves millisecond-level response control of gold wire processing equipment, improves instruction execution accuracy, reduces operation delay, and maintains stable control effect even under network fluctuation conditions, effectively solving the delay and accuracy problems in remote operation; This embodiment builds a digital twin system for gold wire processing, realizes the rapid tracing and accurate prediction of quality problems, provides enterprises with full-process quality assurance capabilities from design to manufacturing, and significantly improves the reliability and consistency of gold wire products.

[0086] A remote control system based on a gold bonding wire device, comprising: In the first module, the gold wire processing parameters are processed through a distributed multimodal sensor network to obtain a high-precision multidimensional detection data set; The second module processes the multi-modal raw monitoring data through a multi-level feature extraction algorithm to obtain the gold wire quality feature vector matrix; The third module processes the multi-dimensional feature vector set through the attention-enhanced deep fusion network to obtain a unified quality indicator feature map; The fourth module processes the unified quality indicator feature mapping through a hierarchical deep classification network to obtain a multi-level defect detection report; The fifth module processes historical quality data and process parameters through a time series prediction network to obtain a parameter optimization recommendation report; The sixth module processes multi-source monitoring data through a spatial mapping engine to obtain an augmented reality interaction system.

[0087] The above describes an embodiment of the present invention, but this embodiment is not limited to the above-mentioned specific implementation mode. The above-mentioned specific implementation mode is merely illustrative and not restrictive. Under the guidance of this embodiment, ordinary technicians in this field can also make more forms of equivalent embodiments, all of which are within the protection of this embodiment.

Claims

1. A remote control method based on a gold bonding wire device, characterized in that: The following steps are involved: Processing gold wire machining process parameters through distributed multimodal sensor networks to obtain high-precision multidimensional inspection data sets; The multi-modal raw monitoring data is processed by a multi-level feature extraction algorithm to obtain the gold wire quality feature vector matrix; The multi-dimensional feature vector set is processed by an attention-enhanced deep fusion network to obtain a unified quality indicator feature map; The unified quality indicator feature map is processed by a hierarchical deep classification network to obtain a multi-level defect detection report; Process historical quality data and process parameters through the time series prediction network to obtain parameter optimization recommendation reports; Processing multi-source monitoring data through a spatial mapping engine to obtain an augmented reality interactive system; The augmented reality interactive system receives the remote operator's operational decisions and remotely controls the gold wire processing equipment.

2. A remote control method based on a gold bonding wire device according to claim 1, characterized in that: In the step of processing the gold wire processing parameters through a distributed multimodal sensor network to obtain a high-precision multidimensional detection data set: the multimodal sensor network includes a high-resolution optical imaging system, an acoustic sensing system, and a thermal imaging system; The high-resolution optical imaging system uses a microscopic lens array to acquire surface topography images. in Indicates the spatial position, Indicates a point in time; The acoustic sensing system uses a piezoelectric sensor array to obtain the acoustic characteristics of the processing process in Indicates frequency; The thermal imaging system uses an infrared array sensor to obtain the temperature distribution during the gold wire processing process. .

3. A remote control method based on a gold bonding wire device according to claim 1, characterized in that: The step of processing the multi-modal original monitoring data by a multi-level feature extraction algorithm to obtain a gold wire quality feature vector matrix comprises: Apply adaptive threshold segmentation and edge detection algorithms to optical images to extract surface morphology feature vectors The formula is: ； represents the adaptive threshold segmentation and edge detection algorithm; Apply short-time Fourier transform and wavelet analysis to the acoustic signal to extract the acoustic feature vector The formula is: ； represents short-time Fourier transform and wavelet analysis; Apply temperature gradient analysis and thermal anomaly detection algorithms to thermal images to extract thermal feature vectors The formula is: ； represents the temperature gradient analysis and thermal anomaly detection algorithm; Forming a multimodal feature set: ； They represent surface morphology feature vector, acoustic feature vector, thermal feature vector and multimodal feature set respectively.

4. A remote control method based on a gold bonding wire device according to claim 1, characterized in that: The step of processing a multidimensional feature vector set through an attention-enhanced deep fusion network to obtain a unified quality indicator feature map comprises: Using the gold wire quality eigenvector matrix Building an Attention-Enhanced Deep Fusion Network Its structure consists of a modality-specific encoder, a cross-modal attention module, and a fusion decoder connected in series: Modality specific encoders Both use a fully connected neural network structure, each encoder contains 3 hidden layers and uses the ReLU activation function; Cross-modal attention module Use scaled dot product attention calculation method: are query, key, and value matrices respectively; Fusion Decoder A two-layer fully connected neural network is used to map the attention-weighted features into a unified feature space; Encode each modal feature independently: ; ； ； Apply the cross-modal attention module: ； in Indicates that from the modal To Modal The attention weight matrix of Fusion of feature representations from all modalities: ； Feature representation after fusion Contains complementary information of all modes.

5. A remote control method based on a gold bonding wire device according to claim 1, characterized in that: The steps of obtaining a multi-level defect detection report by processing a unified quality indicator feature map through a hierarchical deep classification network include: Build a multi-class defect classifier: ； in is the defect category label, is the confidence level, is the unified feature representation output from step 3, are model parameters; Defect area location: ； in is the defect area mask, It is a region localization function based on gradient class activation mapping; Quality score calculation: ； in Represents the overall quality score, is the quality scoring function; Risk level classification: ； in The thresholds are critical, warning, and caution, respectively.

6. A remote control method based on a gold bonding wire device according to claim 1, characterized in that: The step of processing historical quality data and process parameters through a time series prediction network to obtain a parameter optimization recommendation report includes: performing time series preprocessing on quality scores and process parameters in the production process; Training long short-term memory network prediction models; Predict future quality trends; Generate parameter optimization suggestions through a parameter search function based on Bayesian optimization.

7. A remote control method based on a gold bonding wire device according to claim 1, characterized in that: The following steps are also included: Process the data of the entire material processing process through the full-process Internet of Things system to obtain a multi-dimensional process tracking database; The multi-dimensional process tracking data is processed through a multi-physics field coupling model to obtain a digital twin system for gold wire processing.

8. A remote control method based on a gold bonding wire device according to claim 7, characterized in that: The steps of processing multi-dimensional process tracking data through a multi-physics field coupling model to obtain a gold wire processing digital twin system include: Construct the material microstructure evolution function to simulate the microstructure changes of the gold wire during processing. The formula is: ； in For time The microstructural state of The process parameters, is the time step; Construct a multi-physics field coupling function to realize the interactive simulation of thermal field, force field and material field. The formula is: ； in For time The physical field state, including temperature field, stress field, and deformation field; A performance prediction function is constructed to map the microstructure and physical state to the gold wire performance. The formula is: ； in It is the performance index of gold wire, including tensile strength, elongation, conductivity, and welding performance score; Construct a process-quality mapping function to directly link process parameters and quality evaluation. The formula is: ； in It is a quality evaluation indicator.

9. A remote control method based on a gold bonding wire device according to claim 8, characterized in that: The following steps are also included: Process quality anomaly information through the causal inference engine to obtain multi-dimensional defect traceability reports; The virtual parameter test function is used to simulate the impact of different parameter combinations on quality. The formula is: ； in is the set of quality indicators for simulation, is the parameter combination to be tested, is the process constraint, Test function for dummy parameters, To establish a digital twin model.

10. A remote control system based on a gold bonding wire device, characterized in that: include: In the first module, the gold wire processing parameters are processed through a distributed multimodal sensor network to obtain a high-precision multidimensional detection data set; The second module processes the multi-modal raw monitoring data through a multi-level feature extraction algorithm to obtain the gold wire quality feature vector matrix; The third module processes the multi-dimensional feature vector set through the attention-enhanced deep fusion network to obtain a unified quality indicator feature map; The fourth module processes the unified quality indicator feature mapping through a hierarchical deep classification network to obtain a multi-level defect detection report; The fifth module processes historical quality data and process parameters through a time series prediction network to obtain a parameter optimization recommendation report; The sixth module processes multi-source monitoring data through a spatial mapping engine to obtain an augmented reality interaction system.

Images