A remote control method and system based on a gold wire bonding equipment

Abstract

The present invention relates to the technical field of remote control, and discloses a remote control method and system based on a bonding wire device. A remote control method based on a bonding wire device includes: obtaining a high-precision multi-dimensional detection data set; processing multi-modal original monitoring data to obtain a gold wire quality feature vector matrix; processing a multi-dimensional feature vector set to obtain a unified quality index feature mapping; processing the unified quality index feature mapping to obtain a multi-level defect detection report; processing historical quality data and process parameters to obtain a parameter optimization suggestion report; processing multi-source monitoring data to obtain an augmented reality interaction system; the augmented reality interaction system receives operation decisions of a remote operator to remotely control a gold wire processing device. Through a remote control method based on a bonding wire device, a high-precision remote control system realizes millisecond-level response control of a gold wire processing device, improves the instruction execution accuracy rate, reduces the operation delay, and early warns of potential quality problems.

Description

Technical Field

[0001] The present invention relates to the technical field of remote control, and more specifically, it relates to a remote control method and system based on a wire bonding wire device. Background Art

[0002] With the continuous miniaturization and high integration of integrated circuits and semiconductor devices, the gold wire used for chip bonding, as a key interconnection material, its quality control has a decisive impact on the reliability and performance of electronic products. The diameter of the gold wire is usually between 15 - 25 μm, and even finer diameters are required in high-frequency electronic products, which poses extremely high requirements for quality control during the processing.

[0003] Traditional quality control of gold wire processing mainly relies on the following methods:

[0004] Offline sampling inspection: By extracting samples for destructive or non-destructive tests, such as tensile tests, metallographic analysis, etc. This method cannot achieve 100% product inspection, is prone to missing local or random defects, and the inspection results lag behind the production process, making it difficult to intervene in a timely manner.

[0005] Single-modal online detection: Such as solely relying on optical imaging technology for surface defect detection, or only using resistance measurement to evaluate the electrical conductivity of the gold wire. Although these methods can monitor in real time, they have obvious limitations.

[0006] Empirical parameter control: Relying on experienced operators to adjust process parameters based on experience. This method is greatly affected by human factors, is difficult to standardize, and with the increase in process complexity and personnel replacement, it is impossible to ensure a continuous and stable quality control level. Summary of the Invention

[0007] The present invention provides a remote control method and system based on a wire bonding wire device to solve the above technical problems.

[0008] The present invention provides a remote control method based on a wire bonding wire device, including the following steps:

[0009] Processing the gold wire processing process parameters through a distributed multi-modal sensor network to obtain a high-precision multi-dimensional detection data set;

[0010] Processing the multi-modal original monitoring data through a multi-level feature extraction algorithm to obtain a gold wire quality feature vector matrix;

[0011] Processing the multi-dimensional feature vector set through an attention-enhanced deep fusion network to obtain a unified quality index feature map;

[0012] Processing the unified quality index feature map through a hierarchical deep classification network to obtain a multi-level defect detection report;

[0013] Process historical quality data and process parameters through a time series prediction network to obtain a parameter optimization suggestion report;

[0014] Process multi-source monitoring data through a spatial mapping engine to obtain an augmented reality interaction system;

[0015] The augmented reality interaction system receives the operation decisions of remote operators and remotely controls the gold wire processing equipment.

[0016] Furthermore, in the step of processing the gold wire processing process parameters through a distributed multi-modal sensor network to obtain a high-precision multi-dimensional detection data set: the multi-modal sensor network includes a high-resolution optical imaging system, an acoustic sensing system, and a thermal imaging system;

[0017] The high-resolution optical imaging system uses a microscopic lens array to obtain a surface topography image , where represents the spatial position, represents the time point;

[0018] The acoustic sensing system uses a piezoelectric sensor array to obtain acoustic characteristics during the processing , where represents the frequency;

[0019] The thermal imaging system uses an infrared array sensor to obtain the temperature distribution during the gold wire processing .

[0020] Furthermore, the steps of processing the multi-modal original monitoring data through a multi-level feature extraction algorithm to obtain a gold wire quality feature vector matrix include:

[0021] Apply an adaptive threshold segmentation and edge detection algorithm to the optical image to extract the surface topography feature vector , the formula is:

[0022] ;

[0023] represents the adaptive threshold segmentation and edge detection algorithm;

[0024] Apply a short-time Fourier transform and wavelet analysis to the acoustic signal to extract the acoustic feature vector , the formula is:

[0025] ;

[0026] represents the short-time Fourier transform and wavelet analysis;

[0027] Apply temperature gradient analysis and thermal anomaly detection algorithms to the thermal image to extract thermal feature vectors , the formula is:

[0028] ;

[0029] represents the temperature gradient analysis and thermal anomaly detection algorithm;

[0030] Form a gold wire quality feature vector matrix:

[0031] ;

[0032] , , and represent the surface topography feature vector, acoustic feature vector, thermal feature vector, and gold wire quality feature vector matrix respectively.

[0033] Furthermore, the steps of processing the multi-dimensional feature vector set through the attention-enhanced deep fusion network to obtain the unified quality index feature map include:

[0034] Utilize the gold wire quality feature vector matrix to construct an attention-enhanced deep fusion network , whose structure includes a modality-specific encoder, a cross-modal attention module, and a fusion decoder connected in series in sequence:

[0035] Modality-specific encoder , and , all adopt a fully connected neural network structure. Each encoder contains 3 hidden layers and uses the ReLU activation function;

[0036] Cross-modal attention module , adopts the scaled dot-product attention calculation method: , where , , are the query, key, and value matrices respectively;

[0037] Fusion decoder , adopts a two-layer fully connected neural network to map the attention-weighted features to a unified feature space;

[0038] Independently encode each modality feature: ;

[0039] ;

[0040] ;

[0041] Apply cross-modal attention module:

[0042] ;

[0043] Among them , represents the attention weight matrix from modality to modality ; Fuse the feature representations of all modalities:

[0044] ;

[0045] The fused feature representation contains complementary information of all modalities.

[0046] Furthermore, the steps of obtaining a multi-level defect detection report by processing the unified quality index feature map through a hierarchical depth classification network include:

[0047] Construct a multi-level defect classifier:

[0048] ;

[0049] Among them is the defect category label, is the confidence level, is the fused feature representation, are the model parameters;

[0050] Defect area localization:

[0051] ;

[0052] Among them is the defect area mask, is the area localization function based on gradient class activation mapping;

[0053] Quality score calculation:

[0054] ;

[0055] Among them represents the comprehensive quality score, is the quality score function;

[0056] Risk level classification:

[0057] ;

[0058] Among them , and are the thresholds for critical, warning, and attention respectively.

[0059] Further, the step of processing historical quality data and process parameters through the time series prediction network to obtain a parameter optimization suggestion report includes: performing time series preprocessing on the quality scores and process parameters in the production process;

[0060] Training a long short-term memory network prediction model;

[0061] Predicting the future quality trend;

[0062] Generating parameter optimization suggestions through a parameter search function based on Bayesian optimization.

[0063] Further, a remote control method based on a wire bonding equipment further includes the following steps:

[0064] Processing the whole-process data of material processing through a full-process Internet of Things system to obtain a multi-dimensional process tracking database;

[0065] Processing the multi-dimensional process tracking data through a multi-physical field coupling model to obtain a digital twin system for gold wire processing.

[0066] Further, the step of processing the multi-dimensional process tracking data through the multi-physical field coupling model to obtain a digital twin system for gold wire processing includes:

[0067] Constructing a material microstructure evolution function to simulate the microstructure changes of the gold wire during the processing, and the formula is:

[0068] ;

[0069] where is the microstructure state at time , is the process parameter from , is the time step;

[0070] Constructing a multi-physical field coupling function to realize the interactive simulation of the thermal field, force field and material field, and the formula is:

[0071] ;

[0072] where is the physical field state at time , including the temperature field, stress field, deformation field;

[0073] Constructing a performance prediction function to map the microstructure and physical state to the performance of the gold wire, and the formula is:

[0074] ;

[0075] where Performance indicators of gold wire, including tensile strength, elongation, electrical conductivity, and welding performance score;

[0076] Construct a process-quality mapping function to directly associate process parameters with quality evaluation. The formula is:

[0077] ;

[0078] where is the quality evaluation index.

[0079] Furthermore, it also includes the following steps:

[0080] Process the quality anomaly information through a causal inference engine to obtain a multi-dimensional defect traceability report;

[0081] Simulate the impact of different parameter combinations on quality through a virtual parameter test function. The formula is:

[0082] ;

[0083] where is the set of simulated quality indicators, is the parameter combination to be tested, is the process constraint condition, is the virtual parameter test function, is the established digital twin model.

[0084] A remote control system based on a gold wire bonding equipment, including:

[0085] The first module processes the gold wire processing process parameters through a distributed multi-modal sensor network to obtain a high-precision multi-dimensional detection data set;

[0086] The second module processes the multi-modal raw monitoring data through a multi-level feature extraction algorithm to obtain a gold wire quality feature vector matrix;

[0087] The third module processes the multi-dimensional feature vector set through an attention-enhanced deep fusion network to obtain a unified quality index feature mapping;

[0088] The fourth module processes the unified quality index feature mapping through a hierarchical deep classification network to obtain a multi-level defect detection report;

[0089] The fifth module processes the historical quality data and process parameters through a time series prediction network to obtain a parameter optimization suggestion report;

[0090] The sixth module processes the multi-source monitoring data through a space mapping engine to obtain an augmented reality interaction system.

[0091] The beneficial effects of the present invention are as follows:

[0092] The multi-modal gold wire processing quality monitoring system can capture microscopic defects in real time and evaluate the processing quality, with the detection accuracy reaching the micron level; it can give early warnings of potential quality problems and improve the accuracy rate.

[0093] The augmented reality interaction interface enables remote operators to intuitively understand complex microscopic quality data, improving the decision-making efficiency and reducing the operation error rate.

[0094] The fully traceable gold wire processing quality management system supports the full life cycle quality tracking from raw materials to finished products and improves the defect attribution accuracy rate.

[0095] The intelligent process parameter optimization recommendation system can automatically generate optimized parameters based on historical data and the current state, improving the qualified rate of gold wire products.

[0096] The high-precision remote control system realizes the millisecond-level response control of gold wire processing equipment, improves the instruction execution accuracy rate, reduces the operation delay, and can maintain a stable control effect even under network fluctuation conditions, effectively solving the time delay and accuracy problems in remote operation.

[0097] The virtual process verification platform supports testing new process parameters in a digital environment, reduces physical experiments, and accelerates the process optimization cycle.

[0098] The design-manufacturing collaborative optimization system helps to consider manufacturing factors in the product design stage, shortens the product design optimization cycle, and improves the first-pass success rate. Brief Description of the Drawings

[0099] Figure 1 is a flowchart of a remote control method based on a gold wire bonding equipment according to the present invention;

[0100] Figure 2 is an example of the deployment of sensor devices on a gold wire processing production line according to the present invention;

[0101] Figure 3 is an example of data processing according to the present invention;

[0102] Figure 4 is an example of the defect detection result according to the present invention;

[0103] Figure 5 is an example of the process parameter record according to the present invention. Detailed Description of the Embodiments

[0104] In Embodiment 1, at least one embodiment of the present invention discloses a remote control method based on a gold wire bonding equipment, as Figure 1 shown, which includes the following steps:

[0105] Step 1: Process the gold wire processing process parameters through a distributed multi-modal sensor network to obtain a high-precision multi-dimensional detection data set;

[0106] Construct a multi-modal sensor network that includes high-resolution optical imaging, acoustic sensing, and thermal imaging. These sensors are respectively installed at key process nodes of gold wire processing, such as Figure 2 shown:

[0107] High-resolution optical imaging system: Adopt a microscopic lens array , where represents the th microscopic lens, with a resolution of 10μm, to obtain a surface topography image , where represents the spatial position, represents the time point;

[0108] Acoustic sensing system: Use a piezoelectric sensor array to obtain the acoustic characteristics during the processing , where represents the frequency, with a sampling rate of 1MHz, capable of capturing the acoustic emission signals generated by micro-cracks;

[0109] Thermal imaging system: Adopt an infrared array sensor to obtain the temperature distribution during the gold wire processing , with a temperature resolution of 0.1°C and a spatial resolution of 50μm;

[0110] Each sensor module adopts a real-time data transmission protocol to label the collected data according to a unified time stamp and spatial marking to form a high-precision multi-dimensional detection data set.

[0111] Step 2: Process the multi-modal original monitoring data through a multi-level feature extraction algorithm to obtain a gold wire quality feature vector matrix;

[0112] Preprocess and extract features from the high-precision multi-dimensional detection data set obtained in Step 1, as Figure 3 shown:

[0113] Optical image processing: Apply an adaptive threshold segmentation and edge detection algorithm to the obtained in Step 1 to extract the surface topography feature vector :

[0114] ;

[0115] where Denote the optical feature extraction function, which is specifically implemented as a multi-scale convolutional neural network, including convolutional layers, pooling layers, and feature descriptor calculation operations. The input is the original image, and the output is a 128-dimensional feature vector.

[0116] Acoustic signal processing: For the one obtained in step 1 Apply short-time Fourier transform (STFT) and wavelet analysis to extract the acoustic feature vector :

[0117] ;

[0118] where is the acoustic feature extraction function, and is implemented as: First, apply STFT to obtain the time-frequency spectrum , where is the window function; then extract the features of different frequency bands through wavelet transform; finally, use an adaptive feature selection algorithm to generate a 64-dimensional feature vector.

[0119] Thermal image processing: For the one obtained in step 1 Apply temperature gradient analysis and thermal anomaly detection algorithms to extract the thermal feature vector :

[0120] ;

[0121] where is the thermal feature extraction function, and the implementation method is: First, calculate the spatial temperature gradient ; then identify the regions with abnormal temperature gradients; finally, combine the temporal temperature change rate to construct a 96-dimensional feature vector.

[0122] All feature vectors are organized according to unified time and space markers to form a gold wire quality feature vector matrix:

[0123] ;

[0124] Step 3: Process the multi-dimensional feature vector set through an attention-enhanced deep fusion network to obtain a unified quality index feature map;

[0125] Use the gold wire quality feature vector matrix obtained in step 2 , and construct an attention-enhanced deep fusion network:

[0126] Design a multi-modal deep fusion network , whose structure includes:

[0127] Modal-specific encoders , and , all adopt a fully connected neural network structure. Each encoder contains 3 hidden layers (with dimensions of 64, 32, and 16 respectively), and the ReLU activation function is used;

[0128] Cross-modal attention module , adopt the scaled dot-product attention calculation method:

[0129] ;

[0130] Among them 、 、 are the query, key, and value matrices respectively;

[0131] Fusion decoder , adopt a two-layer fully connected neural network (the hidden layer dimension is 32), and map the attention-weighted features to a unified feature space;

[0132] Independently encode each modal feature:

[0133] ;

[0134] ;

[0135] ;

[0136] Apply the cross-modal attention module: ;

[0137] Among them , represents the attention weight matrix from modality to modality .

[0138] Fuse the feature representations of all modalities:

[0139] ;

[0140] The fused feature representation contains the complementary information of all modalities, providing a more comprehensive data basis for defect detection and quality assessment.

[0141] Step 4: Process the unified quality index feature map through a hierarchical deep classification network to obtain a multi-level defect detection report;

[0142] Based on the unified quality index feature map obtained in Step 3, construct a defect detection and quality assessment system:

[0143] Construction of a multi-level defect classifier:

[0144] ;

[0145] wherein is the defect category label, is the confidence level, is the fused feature representation output by step 3, are the model parameters.

[0146] Defect area localization:

[0147] ;

[0148] wherein is the defect area mask, is the area localization function based on Gradient-weighted Class Activation Mapping (Grad-CAM).

[0149] Quality score calculation:

[0150] ;

[0151] wherein represents the comprehensive quality score, is the quality score function, which considers factors such as defect type, severity, distribution, etc.

[0152] ;

[0153] wherein , and are the thresholds for critical, warning, and attention respectively, which can be adjusted according to specific application scenarios. Through the above content, the defect detection results can be obtained, as shown in Figure 4 , including sample ID, defect type, confidence level, defect area coordinates, quality score, and risk score.

[0154] Step 5: Process the historical quality data and process parameters through a time series prediction network to obtain a parameter optimization suggestion report;

[0155] Construct an LSTM-based prediction system to achieve real-time prediction of wire bonding quality problems and parameter optimization:

[0156] Time series preprocessing: Perform time series preprocessing on the quality scores and process parameters in the production process:

[0157] ;

[0158] wherein and are the preprocessed quality score sequence and process parameter sequence respectively, as shown in Figure 5 , specifically including time points, high-frequency induction furnace temperature, homogenization treatment time, rolling pressure, wire drawing speed, and quality score, It is a preprocessing function, including operations such as data alignment, outlier handling, missing value imputation, and normalization. The time series data normalization adopts the sliding window Z-score method to maintain the time dependence of the sequence.

[0159] Train a long short-term memory network (LSTM) prediction model:

[0160] ;

[0161] Predict the future quality trend:

[0162] ;

[0163] Among them represents the predicted future quality score sequence.

[0164] Generate parameter optimization suggestions:

[0165] ;

[0166] Among them is the recommended optimal process parameter set, is the parameter search function based on Bayesian optimization, is the target quality score.

[0167] Step 6: Process multi-source monitoring data through a spatial mapping engine to obtain an augmented reality interaction system

[0168] Based on the multi-level defect detection report in Step 4 and the parameter optimization suggestion report in Step 5, construct an augmented reality interaction system:

[0169] Three-dimensional spatial mapping: Construct the actual gold wire processing environment as a three-dimensional space model and establish a mapping relationship with the physical space:

[0170] ;

[0171] Interaction control interface: Design a gesture recognition function and a voice command recognition function to achieve natural interaction between the operator and the system:

[0172] ;

[0173] Among them and respectively represent the user's gesture and voice input, is the interpreted interaction command.

[0174] Feedback response system: Perform corresponding operations according to interaction commands, such as magnifying a specific area, adjusting parameters, recording problems, remotely controlling the gold wire processing equipment, etc.:

[0175] ;

[0176] where is the system response, is the response function, implemented as a response system based on the event-state mechanism, receiving interaction commands, and performing corresponding operations and updating the AR interface according to the current system state and available data.

[0177] Step 7: Process the operation decision of the remote operator and remotely control the gold wire processing equipment

[0178] Device control interface mapping: Construct the mapping relationship between device parameters and interaction control interfaces:

[0179] ;

[0180] where is the set of process parameters, including key parameters such as temperature, pressure, speed, and tension, is the corresponding set of device control commands. This mapping is implemented through an industrial control protocol and supports bidirectional data transmission.

[0181] Intelligent execution scheduling: Intelligently schedule control instructions according to the current state of the device and the production plan:

[0182] ;

[0183] where is the instruction sequence finally executed, is the scheduling function, is the current state of the device, is the production plan. Intelligent scheduling ensures that control instructions are executed at the most appropriate time, avoiding interfering with the normal production rhythm and ensuring the real-time nature of operation instructions.

[0184] Technical effects of this embodiment:

[0185] The multi-modal gold wire processing quality monitoring system can capture microscopic defects in real time and evaluate the processing quality, with a detection accuracy reaching the micron level;

[0186] The predictive quality analysis system based on deep learning can give early warnings of potential quality problems and improve the accuracy;

[0187] The augmented reality interaction interface enables remote operators to intuitively understand complex microscopic quality data, improving the decision-making efficiency and reducing the operation error rate;

[0188] A gold wire processing quality management system with full-process traceability, supporting quality tracking throughout the entire life cycle from raw materials to finished products, and improving the accuracy of defect attribution;

[0189] An intelligent process parameter optimization recommendation system that can automatically generate optimized parameters based on historical data and current status, improving the qualification rate of gold wire products;

[0190] A high-precision remote control system that enables millisecond-level response control of gold wire processing equipment, improves the accuracy of instruction execution, reduces operation latency, and can maintain a stable control effect even under network fluctuation conditions, effectively solving the problems of latency and accuracy in remote operation.

[0191] Example 2:

[0192] After a quality problem occurs in the gold wire, it is difficult to quickly trace back to the specific cause, which may stem from multiple links such as raw materials, processing technology, or equipment status;

[0193] Traditional quality control mainly focuses on inspection and lacks an accurate model of the relationship between material properties, process parameters, and final performance;

[0194] It is difficult to accurately predict the impact of different design parameters on the performance of gold wires during the product design stage, resulting in long cycles of repeated experiments and optimization;

[0195] In a complex multi-process processing flow, the quality transfer relationship between processes is not clear, making it difficult to achieve full-process optimization;

[0196] The method of this example includes the following steps:

[0197] Step 2-1: Process the gold wire processing process parameters through a distributed multi-modal sensor network to obtain a high-precision multi-dimensional detection data set;

[0198] Same as Example 1, the step content is omitted. (Reason for omission: This step is exactly the same as Example 1 and does not need to be modified);

[0199] Step 2-2: Process the data of the entire material processing process through a full-process Internet of Things system to obtain a multi-dimensional process tracking database; On the basis of Example 1, the full-process data acquisition ability is increased:

[0200] Raw material characteristic data acquisition: Deploy a raw material detection system , and collect the characteristic data of the gold wire raw materials:

[0201] ;

[0202] Among them is the raw material characteristic data set, including key indicators such as chemical composition, crystal structure, impurity content, and initial hardness, is the raw material sample.

[0203] The detection system includes devices such as X-ray fluorescence analyzer (XRF), scanning electron microscope (SEM), energy dispersive spectrometer (EDS), and hardness tester, forming a multi-dimensional material property description.

[0204] Full-process acquisition of process parameters: Construct a process parameter acquisition network , covering the full process of gold wire processing:

[0205] ;

[0206] Among them is the process parameter data set, is the parameter set of all processes, including:

[0207] alloying parameters, homogenization treatment parameters, rolling parameters, wire drawing parameters, heat treatment parameters;

[0208] Environmental status monitoring: Record environmental status data :

[0209] Equipment status monitoring: Construct an equipment status monitoring system , tracking the operating status of the equipment:

[0210] ;

[0211] Among them is the equipment status data set, is the equipment status, including indicators such as the temperature of key components, vibration characteristics, power consumption, and wear status.

[0212] Material tracking system: Realize the material tracking of the full process of gold wire processing and record the material trajectory.

[0213] Step 2-3: Process the multi-dimensional process tracking data through a multi-physics field coupling model to obtain a digital twin system for gold wire processing;

[0214] Based on the multi-modal sensor network deployed in Step 1 and the raw material property data collected in Step 2 ( ), process parameters ( ), environmental status data ( ), and equipment status data ( ), construct a digital twin model for gold wire processing:

[0215] Material microstructure evolution model: Construct a material microstructure evolution function , simulating the microstructure changes of the gold wire during the processing:

[0216] ;

[0217] where is the time of the microstructure state (including grain size, orientation, phase distribution, dislocation density, etc.), is from of the process parameters (temperature, stress, strain rate, etc.), is the time step (usually in milliseconds to seconds).

[0218] The microstructure evolution model is based on the phase field method (Phase Field Method) and cellular automaton, simulating processes such as grain growth, phase transformation, and dislocation evolution. The phase field method is a computational method for simulating the evolution of material microstructure, and its core equation is:

[0219] ;

[0220] where is the order parameter (representing the distribution of different phases or grains in the material), is the atomic mobility (a temperature-dependent parameter, usually following the Arrhenius relationship , where is the activation energy, is the gas constant, is the absolute temperature), is the system free energy (including chemical free energy, elastic free energy, and interfacial energy), is the variational derivative (representing the driving force for the system to tend to minimize the free energy).

[0221] Multi-physics coupling simulation: Construct a multi-physics coupling function , to achieve the interactive simulation of the thermal field, force field, and material field:

[0222] ;

[0223] where is the physical field state at time , including the temperature field, stress field, deformation field, etc., and these states are directly affected by those in step 2-2 and .

[0224] The multi-physics coupling adopts the finite element method (FEM) to construct a thermal-mechanical-material multi-field coupling model to simulate physical phenomena such as temperature distribution, stress distribution, and plastic deformation during the processing. Gold wire macroscopic property prediction model: Construct a property prediction function , mapping the microstructure and physical state to the properties of the gold wire:

[0225] ;

[0226] wherein are the performance indicators of the gold wire, including tensile strength (MPa), elongation rate (%), conductivity (MS / m), welding performance score, etc. This function is trained using the real performance data collected in Step 2.

[0227] Performance prediction adopts a structure-performance relationship model, and based on the obtained microstructural features ( ) and macroscopic physical states ( ), the final performance of the gold wire is predicted. The model structure is a hierarchical neural network, which includes multiple professional sub-networks:

[0228] Grain structure-mechanical property sub-network: A 4-layer fully connected network (512-256-128-64 neurons), with the input being grain size, orientation, boundary characteristics, etc., and the output being yield strength, tensile strength, elongation rate, etc.

[0229] Phase distribution-electrical property sub-network: A 3-layer fully connected network (256-128-64 neurons), with the input being phase composition, distribution, content, etc., and the output being resistivity, temperature coefficient, etc.

[0230] Defect distribution-reliability sub-network: A 3-layer fully connected network (256-128-64 neurons) plus 1 layer of LSTM layer (128 units), with the input being defect type, density, distribution, etc., and the time series information of Step 1 and Step 2, and the output being fatigue life, thermal cycle stability, etc.

[0231] Process-quality mapping model: Construct a process-quality mapping function , directly associating process parameters with quality evaluation:

[0232] ;

[0233] wherein are quality evaluation indicators (qualified rate, defect rate, consistency score, etc.), and this function directly uses all the parameters collected in Step 2-2 as the input.

[0234] The mapping model adopts a hybrid method of deep learning combined with a mechanism model, and the specific implementation is as follows:

[0235] Feature engineering layer: Preprocess and extract features from the original data from Step 2-2, including statistical features, frequency domain features, and expert-defined features;

[0236] Deep network layer: Adopt the structure of a deep residual network (ResNet), including 20 residual blocks, and each residual block contains two convolutional layers and a skip connection;

[0237] Mechanism Constraint Layer: Integrate physics-based constraints (such as material laws, thermodynamics principles, etc.) to ensure that the model predictions conform to physical laws;

[0238] Quality Prediction Layer: A multi-task learning head that predicts multiple quality indicators simultaneously;

[0239] Digital Twin Model Integration and Calibration: Design model integration and calibration functions , integrate all the above sub-models and perform calibration:

[0240] ;

[0241] where is the integrated digital twin model, is the actual production data collected in Step 1 and Step 2, which is used for model calibration.

[0242] The model calibration adopts the Bayesian optimization method, and adjusts the model parameters by minimizing the following objective function:

[0243] ;

[0244] where is the set of model parameters, is the model prediction value, is the measured value, is the distance function (such as mean square error, relative error, etc.), is the weight coefficient (reflecting the importance of different outputs).

[0245] The calibration process is automated and includes the following steps:

[0246] Parameter Sensitivity Analysis: Identify the subset of parameters that have the most significant impact on the model output (usually 10% - 20% of the key parameters);

[0247] Initial Space Sampling: Use the Latin Hypercube Sampling method to select initial points in the parameter space;

[0248] Bayesian Optimization Iteration: Establish a parameter-error surrogate model based on Gaussian process regression to guide the sampling of new parameter points;

[0249] Incremental Update: When new production data is generated, automatically update the model parameters to ensure the synchronization of the digital twin model with the actual production process;

[0250] The digital twin model constructed through this step can accurately reproduce the entire process of gold wire processing in a virtual environment, realizing material properties ( ), process parameters ( ), environmental conditions ( ), and equipment status ( The correlation mapping with the final performance provides a basic model support for subsequent quality prediction.

[0251] Step 2-4: Process the quality anomaly information through the causal inference engine to obtain a multi-dimensional defect traceability report;

[0252] Based on the digital twin system constructed in Step 2-3 ( ), design a quality traceability engine to achieve rapid traceability from quality defects to root causes:

[0253] Quality anomaly detection and location: Construct an anomaly detection function to identify and locate the quality anomalies of the gold wire in real time:

[0254] ;

[0255] where represents the anomaly type (such as wire breakage, crack, cavity, etc., represented by integer encoding), represents the anomaly location (represented by three-dimensional coordinates), is the real-time data collected by the multi-modal sensor in Step 1, is the anomaly detection threshold setting (configurable according to different quality level requirements).

[0256] A combination of multiple algorithms is used for anomaly detection, including:

[0257] Statistical Process Control (SPC): Calculate the control limits based on the segmented steady-state process, and the formula is where is the mean value, is the standard deviation, is the control coefficient (usually taken as 3), which is applicable to the monitoring of stable process parameters;

[0258] Deep anomaly detection network: Adopt a sequence anomaly detection model based on Transformer, with the structure of encoder-decoder architecture (8 attention heads, 6 encoder and decoder layers), and identify anomalies through the reconstruction error, which is applicable to high-dimensional sensor data. Transformer is a deep learning model based on the self-attention module, which can effectively process sequence data and capture long-distance dependencies.

[0259] Expert rule system: A decision tree rule base constructed based on domain knowledge, containing about 200 determination rules for different anomaly patterns, which is applicable to the rapid identification of known anomaly types;

[0260] Reverse inference engine: Design a reverse inference function to trace back from the anomaly phenomenon to the possible causes:

[0261] ​ ;

[0262] where is the set of possible causes (including cause category, location, time, and confidence level), is the digital twin model constructed in Step 2-3, is historical data (including all historical parameter records in Step 2-2).

[0263] Backward inference is based on the Bayesian network and causal inference model to construct a probability mapping from the detected quality anomaly type ( ) to potential causes. The key implementations include:

[0264] Causal graph construction: Utilize the mechanism knowledge of the digital twin model in Step 2-3, combine expert experience and data mining to construct a process-quality causal graph with 1000 nodes and 5000 edges

[0265] Bayesian inference: Calculate the posterior probability based on the causal graph, and the formula is:

[0266] ;

[0267] where is the posterior probability of cause under the given anomaly , is the likelihood term (obtained through the digital twin model in Step 2-3), is the prior probability (based on historical data statistics);

[0268] Time-series causal analysis: Consider the time-series relationship of parameter changes, and adopt the Granger causality test and dynamic time warping (DTW) algorithm to identify the key time points and parameter changes that lead to anomalies. The Granger causality test is a statistical method used to determine whether one time series helps predict another time series; the DTW algorithm is a method for measuring the similarity of two time series, which can handle sequences of different lengths and different rates.

[0269] Backward simulation verification: Use the digital twin model in Step 2-3 for backward simulation to verify whether the inferred cause can reproduce the observed anomaly in the model;

[0270] Multidimensional cause analysis and ranking: Construct a cause analysis function , comprehensively analyze and rank possible causes:

[0271] ;

[0272] where is the sorted list of causes (including detailed information and confidence scores), is the set of inferred reasons, is the weight vector of different factors (configurable, reflecting the key concerns of the enterprise).

[0273] The analysis process adopts a method combining decision tree and rule engine, specifically including:

[0274] Multi-dimensional impact assessment: Evaluate the impact degree of each possible reason in dimensions such as quality, cost, time, and repairability, using a quantitative score of 0-10;

[0275] Confidence calculation: Calculate the confidence score of each reason based on the integrity of the evidence chain, historical similar cases, and model verification results, with a range of 0-1;

[0276] Comprehensive ranking: Rank the reasons based on the weighted score, and the scoring formula is:

[0277] ;

[0278] where is the reason 's confidence, is the impact score in dimension and is the weight of this dimension;

[0279] Cause correlation analysis: Identify the correlation relationships between reasons, construct a cause map, and avoid repeated processing of multiple related reasons;

[0280] Traceability path visualization: Design a traceability path visualization function , and intuitively display the traceability path from the anomaly to the root cause:

[0281] ;

[0282] where is the visualization result, is the sorted list of reasons, is the material tracking information, and path reconstruction is performed through .

[0283] The visualization adopts interactive causal diagrams and time-series flowcharts, including:

[0284] Causal chain visualization: Display the propagation path from the root cause to the observed anomaly in the form of a directed graph, with nodes being physical states and edges being causal relationships;

[0285] Time-series evolution animation: Reconstruct the anomaly formation process based on the digital twin model, and display how the changes in key parameters lead to the final anomaly in chronological order;

[0286] Multi-level interaction: Support drilling down from the macroscopic process flow to the microscopic physical process and then to the microscopic material structure, providing a full-level traceability view;

[0287] Evidence chain display: For each inferred reason, display the evidence chain supporting the conclusion, including sensing data, model predictions, and historical cases, etc.;

[0288] Accumulation of traceability knowledge base: Build a traceability knowledge base function , continuously accumulate traceability experience knowledge:

[0289] ;

[0290] Among them is the updated traceability knowledge base, is the previous knowledge base, is the verified actual reason (confirmed by engineers).

[0291] The knowledge base adopts a graph database structure (such as Neo4j), and the specific implementation includes:

[0292] Case vectorization: Convert each anomaly - cause pair into a feature vector for easy retrieval of similar cases, and use a dual-tower model (Bi-Encoder) for vectorization;

[0293] Knowledge graph construction: Store entities such as anomalies, causes, process parameters, and environmental states and their relationships in the form of a graph, including attributes and timestamps;

[0294] Incremental learning mechanism: Continuously update the reverse inference model and optimize the causal graph structure based on new cases and user feedback;

[0295] Knowledge generalization: Extract general rules from specific cases to enrich the expert rule base and improve the ability to handle new types of anomalies;

[0296] The quality traceability engine implemented in this step directly uses the digital twin model in steps 2 - 3 ( ) for reverse inference and simulation verification, uses the sensor data in step 2 - 1 ( ) for anomaly detection, and combines the material tracking information in step 2 - 2 ( ) for path reconstruction. When detecting a gold wire quality anomaly, it can quickly trace the possible causes, greatly shortening the problem location and solution time, and providing a verified causal mechanism for subsequent quality prediction.

[0297] Step 2 - 5: Gold wire quality prediction and virtual verification system;

[0298] Utilize the digital twin model established in step 2 - 3 ( ) and the traceability knowledge base accumulated in step 2 - 4 ( ), build a gold wire quality prediction and virtual verification system:

[0299] Quality trend prediction: Build a quality trend prediction function , predict the quality change trend in the next period of time:

[0300] ;

[0301] where is the predicted quality trend (including multiple quality indicators and their fluctuation ranges in the time series), is the digital twin model built in step 2-3, is the current process parameter set, is the accumulated traceability knowledge base, is the prediction time range (configurable, generally 8 hours to 7 days).

[0302] The prediction algorithm adopts a hybrid model method and is implemented as follows:

[0303] Physical model prediction: Use the digital twin model ( ) to simulate the physical process and calculate the future quality change trend, which is applicable to short-term prediction (within 8 hours);

[0304] Time series deep learning prediction: Adopt an LSTM network enhanced by an attention module (Long Short-Term Memory, a recursive neural network specialized in dealing with long sequence dependency problems, containing 128 hidden units and a 4-layer stacked structure) to capture the long-term dependency relationship of historical quality data:

[0305] ;

[0306] ;

[0307] where is the hidden state at time t, is the cell state at the previous time, is the input at time t (including the current process parameters and environmental data), is the prediction output.

[0308] Uncertainty quantification: Adopt the Monte Carlo Dropout (MCDropout) technique to keep the dropout layer of the neural network open during the inference stage, and calculate the prediction confidence interval through multiple samplings (usually 100-200 times):

[0309] ;

[0310] ;

[0311] where is the number of sampling times, is the predicted value of the th sampling, and

[0312] is the noise variance inherent in the model.

[0313] Anomaly warning level setting: Set the warning threshold based on the prediction uncertainty interval, which is divided into three levels (low, medium, high) of risk levels. The risk level is determined by the probability of exceeding the quality control limit (low: 5% - 25%, medium: 25% - 50%, high: >50%); Virtual parameter testing: Design a virtual parameter testing function

[0314] to simulate the influence of different parameter combinations on quality:

[0315] where is the set of simulated quality indicators, is the parameter combination to be tested, and are the process constraint conditions.

[0316] The implementation details of the virtual test system include:

[0317] Parameter combination generation: Use the Latin Hypercube Sampling (LHS) method to uniformly generate test points in the parameter space, reducing the number of sampling points by 60% - 80% compared to the traditional grid search while maintaining coverage. Latin Hypercube Sampling is a statistical method that ensures uniform and representative sample points in the parameter space.

[0318] Parallel simulation execution: Based on the Ray distributed computing framework, implement parallel simulation of large-scale parameter combinations, supporting testing >10,000 groups of parameter combinations per hour, with an acceleration ratio of 20 - 40 times that of a single thread

[0319] Adaptive sampling: Build a surrogate model based on Gaussian Process Regression (GaussianProcessRegression) to dynamically adjust the parameter sampling strategy and increase the sampling density in the quality-sensitive area:

[0320] ;

[0321] where is the mean function, and is the kernel function (using the Matérn5 / 2 kernel, which can handle non-smooth characteristics in the parameter space). Gaussian Process Regression is a non-parametric Bayesian method that can perform probabilistic modeling of functions and quantify the uncertainty of predictions.

[0322] Optimal Parameter Recommendation: Construct a Parameter Optimization Function :

[0323] ;

[0324] where is the recommended optimal parameter set, is the target quality index, are the constraint conditions.

[0325] The optimization algorithm adopts Bayesian optimization combined with the particle swarm algorithm to search for the optimal parameter combination that meets the target quality requirements in a virtual environment, considering constraint conditions such as equipment capacity range and production cost.

[0326] Design-Manufacturing Collaborative Optimization: Design a Collaborative Optimization Function :

[0327] ;

[0328] where are the optimized design parameters, are the matching process parameters, is the performance requirement, is the historical knowledge base.

[0329] Collaborative optimization supports considering manufacturing factors in the design stage, realizing the joint optimization of design parameters and process parameters, improving the performance of gold wires while ensuring manufacturability.

[0330] Virtual Experiment Interaction System: Design a Virtual Experiment Interface :

[0331] ;

[0332] where is the experiment interface, is the experiment setting, is the simulation result, is the comparison option.

[0333] The virtual experiment system supports experiment design, parameter adjustment, result preview and scheme comparison, enabling operators to test the effects of new process parameters without interrupting production.

[0334] Step 2-6: Construct an augmented reality interaction interface;

[0335] Full Lifecycle Data Visualization: Design a Full Lifecycle Visualization Function :

[0336] ;

[0337] where For the visualization result, For the material tracking information, For the whole process data collected and generated from Step 2-1 to Step 2-5, For the view type.

[0338] The visualization supports timeline browsing, process switching, and multi-dimensional data linkage analysis, and intuitively displays the complete state evolution process of the gold wire from raw material to finished product.

[0339] Digital twin model interaction: Construct the model interaction function :

[0340] ;

[0341] Wherein is the interaction response, is the operation instruction, is the current view.

[0342] The interaction system supports direct operation of the digital twin model, such as parameter adjustment, cross-section viewing, microstructure magnification, etc., to achieve multi-scale exploration from macro to micro.

[0343] Multi-dimensional quality map: Design the quality map function :

[0344] ;

[0345] Wherein is the quality map view, is all the quality data, is the filtering condition, is the level of detail.

[0346] The quality map intuitively displays the quality distribution under different combinations of process parameters in the form of heat maps, contour lines, and color coding, helping the operator understand the complex relationship between parameters and quality.

[0347] Abnormal origin tracing interaction interface: Construct the origin tracing interaction interface :

[0348] ;

[0349] Wherein is the origin tracing interface, is the selected abnormality.

[0350] The origin tracing interface supports starting from the abnormal phenomenon and exploring backward along the causal chain, gradually unfolding the possible causes, providing evidence support and case comparison, and assisting the operator in systematic analysis.

[0351] Virtual experiment interaction system: Design the virtual experiment interface :

[0352] ;

[0353] Among them is the experiment interface, is the experiment setting, is the simulation result, is the comparison option.

[0354] The virtual experiment system supports experiment design, parameter adjustment, result preview and scheme comparison, enabling the operator to test the effects of new process parameters without interrupting production.

[0355] Technical effects of this embodiment:

[0356] The full-process digital twin system for gold wire processing, based on the multi-physical field coupling model constructed in steps 2-3, realizes the full-process virtual simulation from raw materials to finished products, improves the simulation accuracy, and meets the requirements of industrial-level applications;

[0357] The intelligent quality anomaly traceability system shortens the root cause analysis time of quality problems from traditional hours to minutes and improves the root cause identification accuracy;

[0358] The virtual process verification platform supports testing new process parameters in a digital environment, reduces physical experiments, and accelerates the process optimization cycle;

[0359] The design-manufacturing collaborative optimization system helps to consider manufacturing factors during the product design stage, shortens the product design optimization cycle, and improves the first-pass success rate;

[0360] The high-precision remote control system realizes the millisecond-level response control of gold wire processing equipment, improves the instruction execution accuracy, reduces the operation delay, and can maintain a stable control effect even under network fluctuation conditions, effectively solving the problems of time delay and accuracy in remote operation;

[0361] This embodiment constructs a digital twin system for gold wire processing, realizes the rapid traceability and accurate prediction of quality problems, provides the enterprise with the full-process quality assurance ability from design to manufacturing, and significantly improves the reliability and consistency of gold wire products.

[0362] A remote control system based on gold wire bonding equipment, comprising:

[0363] The first module processes the gold wire processing process parameters through a distributed multi-modal sensor network to obtain a high-precision multi-dimensional detection data set;

[0364] The second module processes the multi-modal original monitoring data through a multi-level feature extraction algorithm to obtain a gold wire quality feature vector matrix;

[0365] The third module processes the multi-dimensional feature vector set through an attention-enhanced deep fusion network to obtain a unified quality index feature map;

[0366] The fourth module processes the unified quality index feature map through a hierarchical deep classification network to obtain a multi-level defect detection report;

[0367] The fifth module processes the historical quality data and process parameters through a time series prediction network to obtain a parameter optimization suggestion report;

[0368] The sixth module processes the multi-source monitoring data through a spatial mapping engine to obtain an augmented reality interaction system.

[0369] The above describes the embodiments of the present invention, but these embodiments are not limited to the above specific implementation manners. The above specific implementation manners are merely illustrative rather than restrictive. Under the inspiration of this embodiment, those of ordinary skill in the art can also make more equivalent embodiments in various forms, all of which fall within the protection scope of this embodiment.

Claims

1. A remote control method based on a bonding wire equipment, characterized in that, Including the following steps: Processing the gold wire processing process parameters through a distributed multi-modal sensor network to obtain a high-precision multi-dimensional detection data set; Processing the multi-modal original monitoring data through a multi-level feature extraction algorithm to obtain a gold wire quality feature vector matrix; Processing the multi-dimensional feature vector set through an attention-enhanced deep fusion network to obtain a unified quality index feature map; Processing the unified quality index feature map through a hierarchical deep classification network to obtain a multi-level defect detection report; Processing the historical quality data and process parameters through a time series prediction network to obtain a parameter optimization suggestion report; Processing the multi-source monitoring data through a space mapping engine to obtain an augmented reality interaction system; The augmented reality interaction system receives the operation decisions of remote operators and remotely controls the gold wire processing equipment; Processing the whole process data of material processing through a full-process Internet of Things system to obtain a multi-dimensional process tracking database; Processing the multi-dimensional process tracking data through a multi-physical field coupling model to obtain a gold wire processing digital twin system; The steps of processing the multi-dimensional process tracking data through the multi-physical field coupling model to obtain the gold wire processing digital twin system include: Constructing a material microstructure evolution function to simulate the microstructure changes of the gold wire during the processing, the formula is: ； wherein is the time of the microstructure state, is from the process parameters, is the time step; Constructing a multi-physical field coupling function to realize the interactive simulation of the thermal field, force field and material field, the formula is: ； wherein is the physical field state of time , including temperature field, stress field, and deformation field; Constructing a performance prediction function to map the microstructure and physical state to the performance of the gold wire, the formula is: ； Among them are the performance indicators of the gold wire, including tensile strength, elongation, electrical conductivity, and welding performance score; Constructing a process-quality mapping function to directly correlate the process parameters with the quality evaluation, the formula is: ； Among them is a quality evaluation index; Processing the quality anomaly information through a causal inference engine to obtain a multi-dimensional defect traceability report; Simulating the influence of different parameter combinations on the quality through a virtual parameter test function, the formula is: ； wherein is a set of simulated quality indicators, is a parameter combination to be tested, is a process constraint condition, is a virtual parameter test function, is the established digital twin model.

2. The remote control method based on the bonding wire equipment according to claim 1, characterized in that, In the step of processing the gold wire processing process parameters through the distributed multi-modal sensor network to obtain a high-precision multi-dimensional detection data set: the multi-modal sensor network includes a high-resolution optical imaging system, an acoustic sensing system and a thermal imaging system; The high-resolution optical imaging system uses a microscopic lens array to obtain a surface topography image , where represents the spatial position, represents the time point; The acoustic sensing system uses a piezoelectric sensor array to obtain acoustic features during the machining process , where represents the frequency; The thermal imaging system uses an infrared array sensor to obtain the temperature distribution during the gold wire processing .

3. The remote control method based on a bonding wire device according to claim 2, wherein The steps of processing the multi-modal original monitoring data through the multi-level feature extraction algorithm to obtain the gold wire quality feature vector matrix include: Apply the adaptive threshold segmentation and edge detection algorithms to the optical image to extract the surface topography feature vector , and the formula is: ； Represents an adaptive threshold segmentation and edge detection algorithm; Apply short-time Fourier transform and wavelet analysis to the acoustic signal to extract the acoustic feature vector , and the formula is: ； represent short-time Fourier transform and wavelet analysis; Apply temperature gradient analysis and thermal anomaly detection algorithms to the thermal image to extract thermal feature vectors , and the formula is: ； Represents a temperature gradient analysis and thermal anomaly detection algorithm; Forming a gold wire quality feature vector matrix: ； , , and respectively represent the surface topography feature vector, the acoustic feature vector, the thermal feature vector, and the gold wire quality feature vector matrix.

4. A remote control method for a wire bonding equipment according to claim 3, wherein The steps of processing the multi-dimensional feature vector set through the attention-enhanced deep fusion network to obtain the unified quality index feature map include: Using the gold wire quality feature vector matrix , construct an attention-enhanced deep fusion network , the structure of which includes a modality-specific encoder, a cross-modal attention module, and a fusion decoder connected in series in turn: Modal-specific encoder , and , all adopt a fully connected neural network structure. Each encoder contains 3 hidden layers and uses the ReLU activation function; Cross-modal attention module , using the scaled dot-product attention calculation method: , where , , are query, key, and value matrices respectively; Fusion decoder , which uses a two-layer fully connected neural network to map the attention-weighted features to a unified feature space; Encode each modal feature independently: ; ； ； Applying a cross-modal attention module: ； Among them , represents the attention weight matrix from modality to modality ; Fusing the feature representations of all modalities: ； Fused feature representation It contains complementary information of all modalities.

5. A remote control method for a wire bonding equipment according to claim 1, characterized in that, The steps of processing the unified quality index feature map through the hierarchical deep classification network to obtain the multi-level defect detection report include: Constructing a multi-level defect classifier: ； wherein is the defect category label, is the confidence level, is the fused feature representation, are the model parameters; Defect area localization: ； wherein is a defect area mask, is a region localization function based on gradient class activation mapping; Quality score calculation: ； Among them represents the comprehensive quality score is the quality scoring function Risk level division: ； Among them , and are the thresholds for critical, warning, and attention respectively.

6. The remote control method based on a gold wire bonding equipment according to claim 1, wherein The steps of processing the historical quality data and process parameters through the time series prediction network to obtain the parameter optimization suggestion report include: performing time series preprocessing on the quality scores and process parameters during the production process; Training a long short-term memory network prediction model; Predicting the future quality trend; Generating parameter optimization suggestions through a parameter search function based on Bayesian optimization.

7. A remote control system based on a gold wire bonding equipment, which is used to execute a remote control method based on a gold wire bonding equipment according to any one of claims 1-6, characterized in that, Including: The first module, processing the gold wire processing process parameters through a distributed multi-modal sensor network to obtain a high-precision multi-dimensional detection data set; The second module processes the multi-modal original monitoring data through a multi-level feature extraction algorithm to obtain a gold wire quality feature vector matrix; The third module processes the multi-dimensional feature vector set through an attention-enhanced deep fusion network to obtain a unified quality index feature map; The fourth module processes the unified quality index feature map through a hierarchical deep classification network to obtain a multi-level defect detection report; The fifth module processes the historical quality data and process parameters through a time series prediction network to obtain a parameter optimization suggestion report; The sixth module processes the multi-source monitoring data through a spatial mapping engine to obtain an augmented reality interaction system.

Images