A welding seam quality real-time prediction method and control system based on multi-source sensor information fusion

Abstract

The application discloses a kind of based on multi-source sensor information fusion's weld quality real-time prediction method and control system, the system includes multi-source perception, feature fusion, digital twin, intelligent decision control and edge cloud collaborative evolution subsystem, by synchronous acquisition short wave infrared, high-frequency electric parameter and acoustic emission signal;With cross attention mechanism, deep physical coupling is analyzed, and cross-modal high-dimensional fusion features are extracted;Based on three-dimensional deconvolution network, a digital twin model is constructed, a weld three-dimensional voxel matrix is output, and internal defects and stress are predicted;Using the Actor-Critic architecture based on deep reinforcement learning, combined with the active output of twin state Continuous process parameter fine-tuning instruction is replaced by traditional passive feedback control;At the same time, using federated learning aggregation policy gradient realizes the swarm intelligence evolution of edge node and cloud, the application realizes the three-dimensional prospective evaluation of weld quality from surface to inside, completely solves the control lag problem, and greatly enhances the generalization ability of the system under the premise of ensuring data privacy.

Description

Technical Field

[0001] This invention relates to the field of welding technology, and in particular to a method and control system for real-time prediction of weld quality based on multi-source sensor information fusion. Background Technology

[0002] With the continuous upgrading of modern manufacturing, automated welding robots have been widely used in industrial production; however, the welding process is a complex physical process with highly nonlinear and strong coupling of multiple variables. How to predict and control the weld quality in real time and accurately has always been a pain point in the industry.

[0003] Currently, most existing real-time weld quality prediction methods rely on single visual sensors or simply splice and process visual and electrical signal features in parallel. This superficial data fusion approach cannot effectively uncover the deep physical coupling between acoustic, electrical, and molten pool temperature field evolution. Furthermore, most existing prediction models can only output one-dimensional features such as weld width and depth, or perform discrete classification of surface defects, lacking a three-dimensional, forward-looking assessment of the weld's three-dimensional spatial morphology and internal residual stress.

[0004] In terms of welding control strategies, traditional systems often employ passive feedback control based on fixed thresholds or conventional optimization algorithms (such as ant colony optimization) for parameter adjustment. These control methods exhibit significant lag when dealing with complex and variable welding conditions, making it difficult to achieve dynamic and proactive compensation of welding parameters. Furthermore, iterative updates to existing models typically require uploading large amounts of raw field data to a server for retraining. This not only places high demands on communication bandwidth but also poses a significant risk of data privacy breaches for enterprises, resulting in extremely poor cross-condition collaborative intelligence and model generalization capabilities. Summary of the Invention

[0005] The purpose of this invention is to overcome the above-mentioned technical deficiencies. The technical solution of this invention provides a method for real-time prediction of weld quality based on multi-source sensor information fusion, including the following steps: Step S1: Simultaneously acquire short-wave infrared images, high-frequency voltage / current timing signals, and high-frequency acoustic emission signals during the welding process through a multi-source sensing subsystem, and extract the corresponding visual feature vectors and timing feature vectors.

[0006] Step S2: Using the feature fusion subsystem, visual feature vectors are used as the query benchmark. The attention weights between the feature vectors are calculated using the cross-attention mechanism to extract cross-modal high-dimensional fusion features.

[0007] Step S3: Input the high-dimensional fusion features into the digital twin subsystem, construct a three-dimensional voxel matrix to predict the weld morphology at the next moment through a three-dimensional deconvolution network, and simultaneously analyze the three-dimensional probability distribution matrix of internal defects and the residual stress gradient.

[0008] Step S4: Construct an environmental state vector from the current 3D twin features and the historical action trajectory of the previous cycle, and input it into the intelligent decision control subsystem based on the Actor-Critic architecture; calculate the comprehensive reward value and output continuous process parameter fine-tuning actions, and send them to the physical machine tool to perform dynamic compensation.

[0009] Step S5: Each edge node updates its policy gradient locally based on the actual forming quality and uploads the policy gradient without the original underlying data; the federated learning aggregation algorithm is executed through the edge-cloud collaborative evolution subsystem to update the global reinforcement learning model and distribute it periodically.

[0010] Furthermore, this invention also provides a real-time prediction and control system for weld quality based on multi-source sensor information fusion, used to implement the above method. The system includes: Multi-source sensing subsystem: used to simultaneously acquire short-wave infrared images, high-frequency electrical parameter signals, and acoustic emission signals in complex welding physical environments.

[0011] Feature fusion subsystem: used to perform dimensionality reduction encoding on heterogeneous data and extract cross-modal high-dimensional fusion features through the cross-attention module.

[0012] Digital twin subsystem: used to output a three-dimensional voxel matrix based on the high-dimensional fusion features, and predict the probability distribution of porosity, incomplete fusion, and residual stress gradient.

[0013] Intelligent decision control subsystem: It is used to construct the environmental state based on the current digital twin characteristics and historical action trajectories, and output continuous process parameter fine-tuning instructions through the Actor-Critic network structure.

[0014] Edge-Cloud Co-evolution Subsystem: Used to update and distribute the global reinforcement learning model by aggregating the policy gradients of each edge node without uploading the original underlying data.

[0015] Compared with the prior art, the beneficial effects of the present invention include: 1. This invention abandons the traditional feature-level shallow splicing by using a multimodal cross-attention mechanism, and realizes the autonomous analysis of the deep physical coupling law between acoustic, electrical and temperature field signals, which greatly improves the robustness of feature extraction under complex welding conditions.

[0016] 2. This invention constructs a three-dimensional deconvolution (3D-Deconv) digital twin model and outputs a complete three-dimensional voxel matrix, thereby expanding the prediction target from one-dimensional apparent parameters to three-dimensional spatial morphology and internal porosity and residual stress, and completing a forward-looking three-dimensional evaluation of weld quality from the surface to the interior.

[0017] 3. This invention transforms highly nonlinear welding control into a Markov decision process by employing a deep reinforcement learning (DRL) algorithm based on the Actor-Critic architecture. This enables proactive prediction of unstable conditions such as spatter and hump and active compensation of continuous process parameters, thus completely solving the overshoot and lag problems that are prone to occur in traditional passive feedback control. Attached Figure Description

[0018] Figure 1 This is a diagram of the architecture of the real-time prediction and adaptive control system for weld quality provided by the present invention. Figure 2 This is a diagram of the cross-attention fusion network structure based on multimodal Transformer; Figure 3 This is a flowchart illustrating the generation of three-dimensional twin states of welds and the prediction of defects and stresses. Figure 4 Schematic diagram of a deep reinforcement learning adaptive control network based on Markov decision-making; Figure 5 Flowchart of a multi-source fusion prediction and edge-cloud collaborative control method for weld quality. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0020] This invention provides a real-time predictive control system for weld quality based on multi-source sensor information fusion, referring to... Figure 1 The system includes a multi-source sensing subsystem 100, a feature fusion subsystem 200, a digital twin subsystem 300, an intelligent decision-making and control subsystem 400, and an edge-cloud collaborative evolution subsystem 500.

[0021] The multi-source sensing subsystem 100 is responsible for acquiring the underlying multi-dimensional heterogeneous signals, and includes: Shortwave infrared imaging module: Acquires two-dimensional temperature gradient field images of the molten pool surface through penetrating arc light and spatter. .

[0022] High-frequency electrical parameter module: Acquires high-frequency voltage / current timing signals reflecting arc stability ( ).

[0023] Acoustic emission array module: captures high-frequency elastic wave signals at the moment of microcrack and pore formation inside the material. ).

[0024] The feature fusion subsystem 200 connects to the multi-source sensing subsystem 100 and is responsible for dimensionality reduction encoding and parsing of deep physical coupling relationships in heterogeneous data; it includes: Visual feature encoder: Employs two-dimensional convolutional block slicing (2DPatch) to convert infrared images into visual feature vectors. ).

[0025] Temporal Feature Encoder: Employs one-dimensional convolution (1DCNN) to transform acoustic and electrical waveforms into temporal feature vectors. ).

[0026] Cross-Attention Module: Utilizing the Transformer's Cross-Attention mechanism, dynamic attention weights are calculated to extract cross-modal high-dimensional fusion features between acoustic, optical, and electrical signals. ).

[0027] Digital twin subsystem 300 based on cross-modal high-dimensional fusion features ( This enables advanced three-dimensional reconstruction of the future state of the weld; it includes: 3D Voxel Generation Module: Employs a 3D deconvolutional network (3D-Deconv) to output the 3D voxel matrix of the weld seam at the next time step. ).

[0028] Defect evolution prediction module: Based on voxel morphology, predicts the three-dimensional probability distribution matrix of internal defects such as porosity and lack of fusion. ).

[0029] Residual stress assessment module: Combines temperature field time gradient to estimate the residual stress gradient inside the weld. ).

[0030] The intelligent decision control subsystem 400, based on the Actor-Critic architecture, outputs continuous process parameter fine-tuning instructions, which include: Markov State Building Module: Integrates current digital twin features with historical action trajectories to construct an environment state vector for reinforcement learning. ).

[0031] Action strategy execution module (Actor network): Directly outputs continuous compensation actions including current, voltage, and speed based on the environmental state. ).

[0032] Multidimensional reward calculation module (Critic network): Dynamically calculates the comprehensive reward value based on actual forming quality, defect probability, and stress trend. ).

[0033] The edge-cloud collaborative evolution subsystem 500 is responsible for control response and cross-machine model swarm intelligence evolution, which includes: Edge inference computing nodes: Deployed on a single machine, they carry lightweight model inference for the aforementioned fusion, twin, and decision subsystems.

[0034] Cloud-based shared experience pool: aggregates "status-action-reward" experience tuples generated by each node, and encrypts and isolates the original data.

[0035] Experience fusion engine: It uses federated learning aggregation algorithms to update the global large model and distribute it periodically.

[0036] Based on the above system architecture, the real-time prediction method of the present invention is as follows: Figure 5 As shown, the specific implementation steps and algorithm logic of this method are as follows: Step S1: Synchronous sensing and feature encoding of multi-source heterogeneous signals The multi-source sensing subsystem 100 synchronously collects data during the control cycle. Subsequently, the data is input into the feature fusion subsystem 200, whose visual feature encoder... Feature extraction Temporal feature encoder , Feature extraction .

[0037] Step S2: Cross-modal feature fusion and physical coupling computation Reference Figure 2 In the cross-attention module, visual features are used as the basis for... Define a query vector to capture acoustic and electrical abrupt changes in the query benchmark. Key vector Sum value vector for: , , , in , , As learnable weights, multimodal fusion features are obtained through cross-attention calculation. : , Scaling factor Key vector The transpose of .

[0038] Step S3: Real-time prediction of the three-dimensional twin state of the weld. Combination Figure 3 ,feature The input is fed into the digital twin subsystem, where the 3D voxel generation module maps the high-dimensional features into a 3D voxel matrix: , based on The defect evolution prediction module and the residual stress assessment module simultaneously parse the internal defect probability matrix. With stress gradient This enables comprehensive prediction of weld formation quality from the surface to the core, and from one-dimensional to three-dimensional.

[0039] Step S4: Adaptive Decision Making and Physical Control Based on Reinforcement Learning Combination Figure 4 The intelligent decision control subsystem 400 transforms control into a Markov decision process.

[0040] Markov state construction extracts the digital twin features output in step S3 and compares them with the actual actions executed in the previous control cycle. To stitch the images together and construct the current moment... Environment state vector : , in, Provide spatial topographic context, and Provide quality alert context, Ensure the smoothness and consistency of the strategy in terms of time sequence.

[0041] The multi-dimensional reward calculation module (Critic) calculates the overall reward value. To guide the model to output high-quality welding processes, the reward function is designed as a linear combination of positive forming incentives and negative defect penalties: , In the formula: The crossover ratio is a three-dimensional ratio used to evaluate the generated twin voxels. With the system's preset ideal weld voxel The degree of overlap between the shapes; the higher the degree of overlap, the greater the positive reward. This is the maximum probability value in the internal defect matrix. If there is a high probability of porosity or incomplete fusion, a negative penalty is applied. The L2 norm of the residual stress gradient is used to prevent excessive local thermal stress from causing cracking. The weighting coefficients are dynamically adjusted.

[0042] The Action Policy Execution Module (Actor) receives... Through inference via a fully connected layer network, the initial values ​​of the continuous control actions at the current moment are output. To ensure the safety of the physical machine tool and the welding process, the system introduces boundary clipping constraints on the initial values ​​to generate the final execution actions. : , The final action vector (Representing compensation amounts for voltage, current, wire feeding, and speed), and then sent down to the underlying physical machine tool for execution.

[0043] After the action is performed, the Critic network calculates the state value function. The dominance function is calculated based on the temporal difference error. : , In the formula, This is the discount factor.

[0044] Subsequently, using the Proximal Policy Optimization (PPO) algorithm, the Actor network optimizes the policy gradient. Update its own network weight The Critic network minimizes the mean square error. Update its own network weight This process enables the system to autonomously learn optimal parameter adjustment strategies to avoid unstable operating conditions such as splashing and humps through continuous trial and error in localized areas.

[0045] Step S5: Edge-Cloud Experience Fusion and Model Co-evolution. At the end of this control cycle, the edge inference computing node will merge the experience tuples of the current cycle. The system stores the policy gradient locally without uploading the original image and acoustic / electric signals. It then extracts the policy gradient of the local model and uploads it to the cloud-shared experience pool. The experience fusion engine executes the FedAvg algorithm to fuse the global policy gradient, updates and distributes the latest global reinforcement learning model. The system determines whether the current welding task has ended. If it has not ended, it returns to step S1 to perform prediction and control for the next cycle, thereby achieving closed-loop adaptive adjustment and collective intelligent collaborative evolution of the entire system.

[0046] The specific embodiments of the present invention described above do not constitute a limitation on the scope of protection of the present invention. Any other corresponding changes and modifications made in accordance with the technical concept of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. A method for real-time prediction of weld quality based on multi-source sensor information fusion, characterized in that, Includes the following steps: Step S1: Synchronously acquire multi-source heterogeneous signals during the welding process through the multi-source sensing subsystem and extract the corresponding feature vectors; Step S2: Through the feature fusion subsystem, the attention weights between the feature vectors are calculated using the cross-attention mechanism to extract cross-modal high-dimensional fusion features; Step S3: Input the high-dimensional fusion features into the digital twin subsystem to construct a three-dimensional voxel matrix that predicts the three-dimensional morphology and internal state of the weld at the next moment. Step S4: Construct the three-dimensional voxel matrix and historical action trajectory into an environmental state vector, input it into the intelligent decision control subsystem based on deep reinforcement learning, output continuous process parameter fine-tuning actions, and send them to the physical machine tool for execution. Step S5: Calculate the policy gradient locally using the empirical tuples generated by each edge node, and execute the federated learning aggregation algorithm through the edge-cloud co-evolution subsystem to update the global reinforcement learning model.

2. The real-time prediction method for weld quality based on multi-source sensor information fusion according to claim 1, characterized in that, The extraction process of multi-source heterogeneous signals and feature vectors in step S1 includes: acquiring short-wave infrared images of the molten pool surface through penetrating arc light and spatter. Two-dimensional convolution blocks are used to slice the data and transform it into visual feature vectors. ; Acquisition of high-frequency voltage / current timing signals and high-frequency acoustic emission signals generated by microcracks and pores inside the material. One-dimensional convolution is used to extract temporal features and transform them into temporal feature vectors. .

3. The real-time prediction method for weld quality based on multi-source sensor information fusion according to claim 2, characterized in that, The calculation process for extracting cross-modal high-dimensional fusion features in step S2 is as follows: Visual feature vectors Define a query vector as the query baseline. Key vector Sum value vector for: ， ， ， in, , , The weight matrix is ​​a learnable weight matrix; Cross-modal high-dimensional fusion features are obtained by calculating through a cross-attention mechanism. : ， In the formula, Scaling factor Key vector The transpose of .

4. The real-time prediction method for weld quality based on multi-source sensor information fusion according to claim 3, characterized in that, The specific process of step S3 includes: The high-dimensional fusion features Input a 3D deconvolutional network to generate a 3D voxel matrix representing the weld morphology at the next time step. : ， Based on the three-dimensional voxel matrix Simultaneously analyze the three-dimensional probability distribution matrix for predicting internal defects. And to estimate the residual stress gradient inside the weld. .

5. The real-time prediction method for weld quality based on multi-source sensor information fusion according to claim 4, characterized in that, In step S4, the intelligent decision-making and control subsystem adopts an Actor-Critic architecture, and the specific decision-making process includes: Combine the current digital twin features with the actual actions executed in the previous control cycle. To stitch the images together and construct the current moment... Environment state vector ; The overall reward value for the current environmental state is calculated using the Critic network. : ， In the formula, , represents the three-dimensional crossover ratio between the generated twin voxel and the ideal weld voxel. This represents the maximum probability value in the internal defect matrix. The L2 norm of the residual stress gradient. To dynamically adjust the weighting coefficients; The environmental state vector is received via the Actor network. The system outputs initial values ​​for continuous compensation actions, which are then used to generate the final execution actions after boundary truncation constraints. These represent the dynamic compensation amounts for real-time output voltage fine-tuning, current fine-tuning, wire feed speed, and welding torch movement speed, respectively.

6. The real-time prediction method for weld quality based on multi-source sensor information fusion according to claim 5, characterized in that, The specific process of federated learning aggregation update in step S5 includes: At the end of the edge control cycle, the original images and acoustic signals are not uploaded. Only the updated policy gradients of the local reinforcement learning model are extracted and uploaded to the cloud-shared experience pool. The cloud-based experience fusion engine executes the FedAvg aggregation algorithm to fuse the policy gradients of each physical machine, updates the global reinforcement learning model, and periodically distributes it to each edge node.

7. A real-time prediction and control system for weld quality based on multi-source sensor information fusion, used to implement the method described in any one of claims 1-6, characterized in that, include: A multi-source sensing subsystem is used to simultaneously acquire short-wave infrared images, high-frequency electrical parameter signals, and acoustic emission signals in complex welding physical environments; The feature fusion subsystem is used to reduce the dimensionality of heterogeneous data and extract cross-modal high-dimensional fusion features through the cross-attention module; A digital twin subsystem is used to output a three-dimensional voxel matrix based on the high-dimensional fusion features to predict the probability distribution of porosity, incomplete fusion, and residual stress gradient. The intelligent decision control subsystem is used to construct the environmental state based on the current digital twin characteristics and historical action trajectories, and output continuous process parameter fine-tuning instructions through the Actor-Critic network structure. The edge-cloud collaborative evolution subsystem is used to update and distribute the global reinforcement learning model by aggregating the policy gradients of each edge node without uploading the original underlying data.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the real-time prediction method for weld quality based on multi-source sensor information fusion as described in any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the real-time prediction method for weld quality based on multi-source sensor information fusion as described in any one of claims 1 to 6.

Images