A Smart Control Method for Welding Process under Multi-Field Coupling Conditions

By combining reinforcement learning and digital twin virtual-real mapping technology with cross-domain transfer learning, the problem of insufficient parameter adjustment under multi-field coupling conditions in the welding process was solved, realizing real-time optimization and adaptive control of welding parameters, and improving weld quality and workpiece deformation stability.

CN123113350APending Publication Date: 2026-11-17LIUZHOU YINRUI VEHICLE IND CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611338377.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-08-31
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing intelligent welding control methods struggle to achieve real-time dynamic optimization when faced with complex working conditions under multi-field coupling, resulting in insufficient adjustment of welding parameters and an inability to guarantee the adaptive capability of weld quality and workpiece deformation.

Method used

A reinforcement learning-based dynamic optimization framework for welding parameters is adopted, which combines a digital twin virtual-real mapping model and a cross-domain transfer learning mechanism. The policy network is trained by the soft Actor-Critic algorithm, and Kalman filtering is used to realize the synchronous update of real-time sensing data and virtual model. A safety constraint mechanism is designed to achieve online fine-tuning.

Benefits of technology

It significantly improves the adaptability and control precision of the welding process under unexpected working conditions, reduces trial and error costs and safety risks, shortens the debugging cycle of new tasks, and improves the efficiency of real-time adjustment of welding parameters.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123113350A_ABST
    Figure CN123113350A_ABST
Patent Text Reader

Abstract

This application discloses an intelligent control method for welding processes under multi-field coupling conditions, belonging to the field of welding manufacturing and intelligent control technology. The method includes: establishing a dynamic optimization framework for welding parameters based on reinforcement learning, constructing a state space, action space, and weighted reward function, and training the policy network using a soft Actor-Critic algorithm; establishing a digital twin virtual-real mapping model, fusing the physical model and the data-driven model, and achieving virtual-real state synchronization and error correction through Kalman filtering; offline pre-training the policy network in a virtual environment, designing a safety constraint mechanism, deploying the policy to the real equipment, and using real-time feedback data for online fine-tuning and continuous iteration. This application also constructs a process knowledge graph and a cross-domain transfer learning mechanism to achieve structured representation of process knowledge and rapid adaptation between tasks. This method can effectively improve the adaptive capability and control accuracy of the welding process under complex working conditions.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application belongs to the field of welding manufacturing and intelligent control technology, specifically relating to an intelligent control method for welding processes under multi-field coupling conditions. Background Technology

[0002] Welding processes under multi-field coupling conditions are affected by multiple factors such as material type, workpiece geometry, interaction of thermal, force and electromagnetic fields, and environmental interference. Welding parameters need to be dynamically adjusted according to real-time working conditions to ensure weld quality. However, the parameter adjustment strategies of existing methods are difficult to adapt to such complex and ever-changing working conditions.

[0003] Common intelligent control methods for welding typically employ strategies such as expert rule bases, lookup table-based parameter configuration, or feedback adjustment based on fixed algorithms to optimize welding process parameters. However, when faced with complex changes in operating conditions such as material changes, workpiece variations, and environmental disturbances, preset rules or fixed algorithms lack the ability to perceive and respond to real-time operating conditions. They cannot autonomously adjust control strategies based on dynamic feedback information during the welding process, resulting in a severe deficiency in the adaptive capability of the control system when facing unexpected operating conditions, making it difficult to achieve real-time dynamic optimization of welding parameters under different operating conditions. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides an intelligent control method for welding processes under multi-field coupling conditions, which can effectively solve the problems mentioned in the background technology.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] In a first aspect, the present invention provides an intelligent control method for welding processes under multi-field coupling conditions, the method comprising the following steps:

[0007] A dynamic optimization framework for welding parameters based on reinforcement learning is established, including constructing a state space, defining an action space and action constraints, designing a weighted reward function, and using a soft Actor-Critic algorithm to train the policy network and value network.

[0008] Establish a digital twin virtual-real mapping model, including constructing a heat conduction model, a fluid dynamics model, and a mechanical model as physical models, constructing a neural network model and a Gaussian process model as data-driven models, establishing a weighted fusion framework of physical models and data-driven models, and using Kalman filtering to realize real-time synchronous update and error correction of the state of sensor data and virtual models;

[0009] In the virtual environment formed by the digital twin virtual-real mapping model, the policy network is pre-trained offline using reinforcement learning; a security constraint mechanism including parameter safety thresholds and anomaly detection processing is designed, and the pre-trained policy network is deployed to a real welding device; using real-time feedback data from the real welding process, the policy network is fine-tuned online using a policy gradient algorithm.

[0010] In summary, this application includes at least one of the following beneficial technical effects:

[0011] 1. This invention constructs a state space encompassing material properties, workpiece geometry, and multi-dimensional sensor information. It then combines this with a soft Actor-Critic algorithm to jointly train the policy network and value network. This enables the control policy to autonomously output continuous adjustments to multi-dimensional parameters such as welding current, voltage, and speed based on real-time operating conditions. The weighted reward function is designed to simultaneously consider weld quality, workpiece deformation, energy consumption, and safety constraints, guiding the policy network to achieve a comprehensive balance among multiple optimization objectives. This significantly improves the adaptive capability and control accuracy of the welding process under unexpected conditions.

[0012] 2. This invention establishes a weighted fusion framework by integrating physical models of heat conduction, fluid dynamics, and mechanics with neural networks and Gaussian process data-driven models. Kalman filtering is then used to achieve synchronous updates of sensor data and the virtual model, enabling the digital twin to reflect the real welding process with high fidelity. Based on this virtual environment, the policy network can complete offline pre-training without occupying actual equipment, significantly reducing trial-and-error costs and safety risks. Safety constraint mechanisms and online fine-tuning strategies ensure a smooth transition from virtual to actual deployment, allowing the control strategy to be continuously optimized in actual production, effectively solving the discrepancy between physical simulation and the real environment.

[0013] 3. This invention constructs a heterogeneous graph containing multiple entities such as materials, weld types, equipment, process parameters, and quality indicators. It then utilizes knowledge embedding technology to transform symbolic knowledge into vectorized representations, achieving unified organization and efficient retrieval of historical experience. Based on a source-target domain matching algorithm using process similarity calculation and a domain adversarial transfer learning mechanism, the control knowledge accumulated in existing welding tasks can be quickly transferred to new materials, new weld types, or new equipment conditions, significantly shortening the debugging cycle for new tasks. The introduction of a meta-learning framework further endows the policy network with the ability to quickly adapt to new tasks with a small number of samples, effectively solving the time and resource consumption problems caused by repeated training in welding task switching scenarios. Attached Figure Description

[0014] Figure 1 This is an overall flowchart of the intelligent control method for welding process under multi-field coupling conditions according to the present invention;

[0015] Figure 2 A flowchart for constructing the reinforcement learning-based dynamic optimization framework for welding parameters in step S1;

[0016] Figure 3 This is a flowchart for the construction and characterization of the process knowledge graph in step S2;

[0017] Figure 4 Here is a flowchart of the cross-domain transfer learning mechanism in step S3;

[0018] Figure 5 Establish a flowchart for the digital twin virtual-real mapping model in step S4;

[0019] Figure 6 This is a flowchart of the offline pre-training and online fine-tuning collaborative mechanism in step S5. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this invention clearer, the following description is provided in conjunction with the appendix. Figure 1 To be continued Figure 6 The present invention will be further described in detail below with reference to specific embodiments.

[0021] See Figure 1 The intelligent control method for welding processes under multi-field coupling conditions provided by this invention includes the following core steps: establishing a dynamic optimization framework for welding parameters based on reinforcement learning, constructing a process knowledge graph and representation mechanism, establishing a cross-domain transfer learning mechanism, establishing a digital twin virtual-real mapping model, and establishing an offline pre-training-online fine-tuning collaborative mechanism. The specific implementation methods of each step are described in detail below.

[0022] First, for step S1, a dynamic optimization framework for welding parameters based on reinforcement learning is established. (See [link to relevant documentation]). Figure 2 Step S1 transforms the welding parameter optimization problem into a sequential decision problem. By constructing and training a reinforcement learning model, the model can autonomously output welding parameter adjustment strategies based on the real-time perceived welding status. The specific construction process of this step is as follows.

[0023] Step S101: Construct the state space of the Markov decision process, formally defining the multidimensional information affecting the welding process, and building a spatial set describing the real-time state of the welding process. The state space comprehensively considers multi-source heterogeneous data such as material properties, workpiece geometric features, and real-time sensor feedback.

[0024] Before welding begins, material property characteristics are extracted from the workpiece material information. These characteristics include the coded values ​​mapped to the material grade, the mass percentage of various elements in the material's chemical composition, mechanical properties such as yield strength and tensile strength, and thermophysical properties such as thermal conductivity and specific heat capacity.

[0025] Extract workpiece geometric features from the workpiece's 3D model or scanned data. These features include workpiece thickness, bevel angle, 3D coordinates of the weld center in the workpiece coordinate system, and geometric descriptions of adjacent structures at the weld site.

[0026] During the welding process, real-time sensing features are collected and constructed. These features include real-time sampled values ​​of welding current and their arithmetic mean and standard deviation within a preset time window, real-time sampled values ​​of welding voltage and their arithmetic mean and standard deviation within a preset time window, real-time wire feed speed, real-time welding speed, and real-time shielding gas flow rate.

[0027] The geometric features of the molten pool are extracted after preprocessing the images of the molten pool region acquired by the optical sensor. Key temperature feature values ​​are obtained after processing the temperature field distribution data acquired by the thermal imaging sensor. Time-domain energy features and frequency-domain dominant frequency features are extracted from the arc sound signal.

[0028] Combine all the above features in a preset order. State vector at time step The set of all possible states constitutes the state space. state space The mathematical form is:

[0029]

[0030] in, This represents the control time sequence number in the discretization, and its value is a non-negative integer. express Moment A dimensional column vector of states. Symbols It represents the set of real numbers. This represents the total dimension of the state vector, taking a positive integer value determined by the actual number of features selected. State vector The order of features in each dimension is fixed during system initialization.

[0031] Each feature dimension is normalized before being input into the neural network, mapping the value range of each dimension to an interval. The normalization process uses a minimum-maximum scaling method, with the normalization parameters determined by the global minimum and maximum values ​​of each dimension in the historical data.

[0032] Step S102: Construct the action space and action constraints of the Markov decision process, and define the set of actions that the reinforcement learning agent can execute. The action space covers all adjustable process parameters during the welding process. Action vector at time step for The vector consists of four columns, each corresponding to an adjustment for welding current, welding voltage, welding speed, wire feed speed, and shielding gas flow rate. Each adjustment is a signed real value; positive values ​​increase the corresponding parameter, and negative values ​​decrease it. The set of all possible actions constitutes the action space. Action space The mathematical form is:

[0033]

[0034] in, Indicates the control time sequence number in the discretization. express Moment A column vector of actions. Symbols It represents the set of real numbers. This represents the total dimension of the action vector, in this embodiment. The value is 5. Action vector The dimensions are arranged in a fixed order: current adjustment, voltage adjustment, speed adjustment, wire feed speed adjustment, and gas flow rate adjustment.

[0035] Motion constraints are determined based on the physical limits of the welding equipment and process safety specifications. Hard constraints are implemented by independently limiting each dimension of the motion vector: for the motion vector The Dimensions The output value is restricted to a range Inside, and These represent the lower and upper limits of the adjustment amount for that dimension, respectively. If the original action value output by the policy network is lower than... Then take If higher Then take Otherwise, keep the original value.

[0036] Soft constraints are implemented through a safety constraint penalty component in the subsequent reward function. For actions exceeding the rated working range, the reward function imposes a penalty proportional to the degree of exceeding the limit, guiding the agent to gradually tend to choose actions within the safety margin during the learning process.

[0037] Step S103: Construct a weighted reward function that comprehensively considers multiple objectives, design a reward function to evaluate the welding effect, and drive the agent to learn a process parameter optimization strategy that comprehensively balances multiple objectives. The reward function at each control time step... Feed back a scalar reward value to the agent Scalar reward value The form is a weighted sum of the objective components:

[0038]

[0039] in, Indicates the control time sequence number in the discretization. express The total reward value at any given moment is a scalar real number. , , , These are the weighting coefficients for the four components, all of which are positive real numbers. The initial values ​​of the weighting coefficients are determined by process experts based on the quality priority of the welding scenario. This indicates the bonus for weld quality. This indicates the workpiece deformation penalty component. This indicates the energy consumption bonus component. This indicates the penalty component for safety constraints.

[0040] Weld quality bonus The weld appearance quality is calculated based on the quality inspection results after a single weld pass is completed. The weld appearance quality is divided into four levels: excellent, good, medium, and poor, corresponding to base scores of 10, 5, 0, and -5 respectively. The weld depth score is calculated based on the relative deviation between the measured weld depth and the target weld depth. A relative deviation of less than 5% earns 5 points, between 5% and 15% earns 0 points, and exceeding 15% earns -5 points.

[0041] The weld width score is calculated based on the relative deviation between the measured weld width and the target weld width value. The scoring rules for weld width are the same as those for weld depth. Defect points are calculated based on the number and size of defects such as porosity, inclusions, and cracks detected by internal non-destructive testing. Each defect deducts 1 to 5 points depending on its severity. The weld quality bonus is obtained by adding the base score, weld depth score, and weld width score, and then subtracting the defect points. The value of .

[0042] Workpiece deformation penalty component Calculations are based on the three-dimensional deformation of the workpiece at key measurement points after welding. Pre-set parameters are placed on the workpiece. One deformation measurement point, The value is a positive integer. After welding, the spatial displacement of each measurement point is obtained using a three-dimensional measurement device. The displacement of each measurement point is denoted as The unit is millimeters, where Values ​​range from 1 to Integer values. Workpiece deformation penalty component. The calculation formula is the negative of the sum of the displacements at each measurement point:

[0043]

[0044] in, This indicates the total number of preset deformation measurement points. Indicates the first Spatial displacement of each measurement point, in millimeters. Workpiece deformation penalty component. The calculation formula comes from the basic definition of deformation measurement, that is, the total deformation is the sum of the displacements of each measuring point. In this embodiment, the opposite number is taken to convert the total deformation into a penalty signal.

[0045] Energy consumption bonus Based on the total energy consumption calculation for a single welding process, the total energy consumption includes electrical energy consumption. and gas consumption Electricity consumption The gas consumption is obtained by integrating the real-time sampled values ​​of welding current and welding voltage over time. The energy consumption is obtained by integrating the protective gas flow rate over time. and gas consumption The weighted sum of the unit costs and the opposite of the sum:

[0046]

[0047] in, This indicates the energy consumption bonus component. This represents the unit cost coefficient for electricity consumption, expressed in monetary units per kilowatt-hour. This represents the unit cost coefficient for gas consumption, expressed in monetary units per liter. This indicates the total electrical energy consumption of a single welding process, expressed in kilowatt-hours. This represents the total gas consumption for a single welding process, expressed in liters. Energy consumption bonus component. The calculation formula is derived from the basic accounting method of energy consumption cost, that is, the total cost is the sum of the products of various types of energy consumption and the corresponding unit cost. In this embodiment, the opposite number is taken to convert the total cost into a reward signal.

[0048] Safety constraint penalty component This is an accumulation of multiple penalties. When the welding current exceeds 110% of the rated current limit, an overload penalty is triggered, with a penalty value of -20. When the welding voltage deviates from the preset target voltage by more than 15%, a voltage anomaly penalty is triggered, with a penalty value of -10. When the temperature sensor detects that the local temperature of the workpiece exceeds 80% of the material's solidus temperature, a temperature over-limit penalty is triggered, with a penalty value of -15. The above three penalties are triggered independently; if multiple penalties are triggered simultaneously, they are accumulated; if none are triggered, a safety constraint penalty component is applied. Take 0.

[0049] Weighting coefficient , , , During system operation, dynamic adjustments are made during cross-task migration through a meta-learning mechanism. For specific adjustment methods, please refer to step S3.

[0050] Step S104: Based on the three elements of state, action, and reward, determine the value and policy function of the optimal welding parameter strategy. Establish a mapping relationship from the welding state to the optimal welding parameter adjustment amount by solving the optimal policy. The solution process is implemented by training the policy network and the value network.

[0051] Policy network representation of conditional probability distribution That is, in the state The conditional probability density of each action is selected. The value network represents the state-value function. That is, in the state The expected cumulative reward that can be obtained by acting according to the current strategy. This embodiment uses the soft Actor-Critic algorithm as the core solution algorithm, introducing an entropy regularization term on the basis of the standard Actor-Critic framework. The training objective is to maximize the expected cumulative reward while maximizing the entropy of the strategy.

[0052]

[0053] in, Let represent the objective function, which is a scalar real number representing the overall goal of the algorithm optimization. This represents the strategy to be optimized, i.e., the mapping rule from state to action. Indicates the control time sequence number in the discretization. The mathematical expectation operator. express The state vector at time t. express Action vector at any given moment. Representation Strategy The induced state-action joint distribution.

[0054] Indicates the state Next action The instant reward value obtained afterwards, instant reward value It is calculated using the reward function in step S103. This represents the entropy regularization coefficient, a non-negative real number, which controls the balance between exploration and utilization. The entropy value represents the policy's entropy in a given state. It is a scalar real number, and the larger the entropy value, the more dispersed the policy's action choices are in a given state.

[0055] Both the policy network and the value network are implemented using multi-layer fully connected neural networks. The number of input layer nodes in the policy network is equal to the number of state vectors. Dimensions The number of output layer nodes in the policy network is equal to the number of action vectors. Dimensions The output layer of the policy network adopts The activation function maps the original output value to The range is then converted to the actual adjustment range corresponding to each dimension through an inverse scaling operation.

[0056] The number of nodes in the input layer of a value network is equal to the number of state vectors. Dimensions The value network has a single output layer node, which directly outputs a scalar value estimate. Both the policy network and the value network's hidden layers employ a three-layer fully connected structure, and all hidden layers in both networks use the ReLU activation function.

[0057] The loss function of a value network is defined as the expected value of the mean squared error between the predicted value and the target value. The loss function of a policy network maximizes the objective function. The gradient is calculated using reparameterization techniques.

[0058] The training data is constructed as follows: In the digital twin virtual environment, the current policy network generates and executes actions, and records the quadruplets of each interaction. The quadruplets are stored in the experience replay buffer. During training, mini-batch samples are randomly sampled from the experience replay buffer to update the network parameters. The training convergence objective is that within a preset number of training rounds, the average cumulative reward value of each complete welding task no longer increases, and the loss value of the value network remains below a preset threshold.

[0059] Taking the welding scenario of aluminum alloy thin sheets for automotive body as an example, aluminum alloys have high thermal conductivity and a large tendency to deform. The state space mainly includes thermal conductivity and workpiece thickness characteristics. The action constraint sets an upper limit on the current to prevent burn-through, and the reward function includes a workpiece deformation penalty component. Weighting coefficients The values ​​are set to relatively high to prioritize deformation control. The actions output by the trained strategy network tend to select lower combinations of thermal input parameters while ensuring melt depth.

[0060] In summary, step S1 completes the construction of the reinforcement learning welding parameter optimization framework. This framework transforms the multi-physics coupled welding process into a sequential decision problem with computable three elements: state, action, and reward. It outputs a continuous policy mapping through a soft Actor-Critic algorithm, providing a direct decision interface for subsequent process knowledge graph fusion, cross-domain transfer learning, and digital twin virtual-real mapping.

[0061] For step S2 in the method of this embodiment, refer to... Figure 3 The process involves constructing a process knowledge graph and representation mechanism. The welding process involves multiple elements such as materials, equipment, process parameters, and quality requirements, and these elements have complex process relationships. Simultaneously, historical welding process data contains a wealth of experiential knowledge for reference. Step S2 constructs a structured process knowledge graph to uniformly organize and represent welding knowledge scattered across different data sources. Furthermore, knowledge embedding technology transforms the symbolic knowledge in the graph into vectorized representations that can be directly used by machine learning models, enabling subsequent reinforcement learning policy networks and transfer learning mechanisms to effectively utilize this prior knowledge. The specific construction process of step S2 is as follows.

[0062] Step S201: Construct a heterogeneous graph structure containing multiple types of entities and multiple types of relationships. Extract multiple types of entities from historical welding process data and establish multiple types of process association relationships between entities to form a heterogeneous graph structure.

[0063] The extracted entity types include material entities, weld type entities, equipment entities, process parameter entities, quality index entities, welding method entities, operator entities, and workpiece type entities.

[0064] Material entity nodes store information on material grade, chemical composition, mechanical properties, and thermophysical properties.

[0065] The weld type entity node stores weld type and weld size information. Weld type information includes types such as butt welds, fillet welds, and T-welds. Weld size information includes values ​​such as weld width and weld height.

[0066] The equipment entity node stores information such as welding machine model, welding torch model, wire feeding device model, and gas shielding equipment model.

[0067] The process parameter entity node stores information on current parameters, voltage parameters, speed parameters, and gas parameters.

[0068] The quality indicator entity node stores information on melt depth, melt width, excess height, forming grade, and internal defect grade.

[0069] Edge types between entities are used to represent the process relationships between different entities. Edge types include material-weld relationships, equipment-weld relationships, process-quality relationships, material-process relationships, and equipment-process relationships. Each edge type corresponds to a fixed relationship semantic. A material-weld relationship indicates that a specific material type is suitable for a specific weld type. A material-process relationship indicates the recommended range of process parameters for a specific material type.

[0070] In step S202, knowledge embedding technology is used to map entities and relations in the graph to a low-dimensional vector space. Based on the heterogeneous graph constructed in step S201, knowledge graph embedding technology is used to map each entity and each relation in the graph to a low-dimensional real vector, thereby realizing the distributed representation of knowledge.

[0071] For the head entity in the map and relationships The head entity embedding vector is denoted as The relation embedding vector is denoted as For the triples in the graph ,in For the tail entity, the tail entity embedding vector is denoted as . Knowledge embedding models learn to embed head entities into vectors. With relation embedding vector The sum of the tail entity embedding vectors In vector space, they should be as close as possible. The above addition operation is an element-wise addition where each dimension is independent, i.e. The Wei and The Adding the dimensions together yields the first dimension of the resulting vector. Dimension, among which Take all dimensions of the embedded vector.

[0072] The training objective of the knowledge embedding model is to minimize the marginal loss function. The loss function is derived from the classic definition of the TransE series of knowledge graph embedding models, and this embodiment directly adopts its standard form. The mathematical form of the loss function is:

[0073]

[0074] in, The loss function value is a scalar real number used to measure the degree of deviation between the current embedding vector representation and the ideal representation. This represents the set of positive samples, which are derived from actual triples in the graph. This represents the set of negative samples, which are generated by replacing entities or relations in the positive samples: [The text abruptly ends here, likely due to an incomplete sentence or a formatting error.] Head entity Randomly replace with other entities in the graph to obtain Or positive samples tail entity Randomly replace with other entities in the graph to obtain Or positive samples Relationship Replace with other relation types to get . This represents the marginal hyperparameter, which takes the value of a positive real number and is used to control the minimum interval between the scores of positive and negative samples. This represents the vector distance metric function. In this embodiment, Euclidean distance is used, which is the square root of the sum of the squares of the differences between corresponding elements of two vectors. The embedding vector of the head entity is denoted as . A dimensional real vector, where For the embedded dimension. The embedding vector representing the relation is: A 3D real vector. The embedding vector representing the tail entity is: A 3D real vector. Let represent the embedding vector of the head entity in the negative sample, which is . A 3D real vector. The embedding vector representing the relation in the negative sample is: A 3D real vector. Let the embedding vector of the tail entity in the negative sample be denoted as . A 3D real vector. Symbol This indicates the operation of taking the positive part, when The time value is ,when The value is 0 at that time.

[0075] The convergence objective of training a knowledge embedding model is the loss function value. Training terminates when the loss function value falls below a preset threshold within a predetermined number of consecutive training epochs, indicating that the decline no longer occurs. The knowledge embedding model can be implemented using TransE, TransH, TransR, DistMult, or ConvE models.

[0076] Step S201 establishes a structured knowledge framework, clarifying the key entity types and their process relationships in the welding field. Step S202 transforms the symbolic knowledge in the above framework into embedding vectors that can be directly used by downstream machine learning tasks. Subsequent reinforcement learning policy networks and transfer learning mechanisms can call upon historical process experience in a unified vectorized form, such as retrieving historical process parameters under similar materials or similar working conditions from the process knowledge graph and representation mechanism as a reference for policy initialization, thereby reducing dependence on trial and error data and improving the initial performance of the policy.

[0077] Step S203: Based on the embedded vector representation, establish a causal relationship chain between material properties, process parameters and quality results. Based on the entity embedded vector and relation embedded vector obtained in step S202, establish a causal relationship chain between material properties, process parameters and final quality to support the reasoning and application of process knowledge.

[0078] The causal relationship chain is represented as a directed graph structure. The nodes in the causal relationship chain graph include material nodes, process parameter nodes, and quality result nodes. The directed edges in the causal relationship chain graph represent the direction of the causal relationship, i.e., from the cause node to the result node. The construction process of the causal relationship chain is as follows:

[0079] First, using a combination of specific material nodes and specific process parameter nodes as query conditions, the corresponding historical quality data records are retrieved from the knowledge graph constructed in step S201 to obtain the quality indicator entity nodes associated with the above material-process combination and their stored quality indicator values.

[0080] Second, causal inference algorithms are used to analyze the causal relationships between various process parameters and quality results. These algorithms include causal discovery algorithms based on intervention variables and causal inference algorithms based on observational data. Algorithms based on intervention variables determine the strength of causality by fixing other variables and changing the values ​​of the target process parameters, then observing the changes in quality results. Algorithms based on observational data calculate the strength of causality by statistically analyzing the covariance between process parameters and quality results in historical data.

[0081] The causal strength is calculated as follows:

[0082] For a given process parameter and quality results Collect process parameters from historical data Quality results corresponding to different values The measured values. Calculate the process parameters. Standard deviation and quality results Standard deviation and process parameters With quality results Pearson correlation coefficient between Then the strength of the causal relationship Defined as:

[0083]

[0084] in, Indicate process parameters For quality results The causal strength value is a scalar non-negative real number; the larger the value, the stronger the process parameter. For quality results The stronger the causal influence, the better. Indicate process parameters With quality results The Pearson correlation coefficient between them ranges from [-1, 1], and the absolute value is used to measure the degree of correlation. Indicate process parameters Historical data standard deviation Indicates quality results Historical data standard deviation, ratio This is used to convert correlation coefficients into a causal strength dimension with physical units. The formula for calculating the strength of causality is derived from the standardized measure of the influence of independent variables on dependent variables in linear regression, namely the absolute value of the standardized regression coefficient. In this embodiment, it is directly applied to quantify the causal relationship between welding process parameters and quality results.

[0085] Third, the causal relationships obtained from the analysis are added to the causal relationship chain in the form of directed edges. Each directed edge points from the process parameter node to the quality result node, and the edge is accompanied by a causal relationship strength value. .

[0086] The constructed causal chain is used for the following functions:

[0087] The process recommendation function searches along the causal chain based on the input material type to find the optimal combination of process parameters that yields the best quality results. The optimal criterion is to normalize four quality indicators—melt depth, melt width, residual height, and forming grade—and then take the weighted sum of these indicators. The process parameter combination that results in the maximum weighted sum is then selected as the recommended result.

[0088] The quality prediction function infers the corresponding quality result prediction value by reasoning along the causal chain based on the input process parameters.

[0089] The parameter adjustment suggestion function identifies the types of parameters that need adjustment by tracing the causal chain backwards from the deviation between the current process parameters and the target quality.

[0090] The loss function formula in step S202 is used to train the knowledge embedding model, and the resulting entity embedding vectors provide the semantic similarity calculation basis for the causal relationship chain construction in step S203. The causal strength formula in step S203 is independent of the embedding vectors and is directly calculated based on the statistics of historical data. The two serve two different purposes: knowledge representation and causal reasoning, and there is no mutual derivation relationship between the formulas.

[0091] Step S2 ultimately outputs a knowledge system that can be directly used by subsequent modules. The process knowledge graph organizes discrete experiences from historical data into an explicit, structured network of entity relationships. Knowledge embedding transforms this into numerical representations, and causal chains further refine the response relationships between process parameters and quality. Subsequent reinforcement learning decisions and cross-domain transfer learning can directly obtain prior constraints and similarity references from this knowledge system.

[0092] See Figure 4 Step S3 involves establishing a cross-domain transfer learning mechanism. In actual production, welding tasks frequently switch. When encountering new materials, new weld types, or new workpieces, training the control strategy from scratch each time would consume a significant amount of time and trial welding costs. Step S3 establishes a cross-domain transfer learning mechanism to enable the accumulated welding process knowledge to be transferred from old tasks to new tasks, thereby reducing the strategy learning cost under new working conditions and shortening the debugging cycle. The specific implementation process of step S3 is as follows.

[0093] Step S301 involves designing a source-target domain matching algorithm based on process similarity. To select the most suitable knowledge source from existing welding tasks for transfer to the current new task, it is necessary to quantify the similarity between different welding tasks. Step S301 achieves source-target domain matching by calculating multi-dimensional process similarity.

[0094] The calculation of process similarity comprehensively considers four dimensions: material similarity, weld type similarity, equipment similarity, and operating condition similarity.

[0095] Material similarity is calculated based on the vector distance of material property features. These features include material grade, chemical composition, mechanical properties, and thermophysical properties, which have been converted into embedding vectors by the process knowledge graph and characterization mechanism in step S2. The formula for calculating material similarity is:

[0096]

[0097] in, This represents the material similarity between two materials. It is a scalar real number with a value ranging from 0 to 1. The larger the value, the more similar the two materials are. Let the embedding vector of the first material be denoted as . The dimensional real vectors are derived from the entity embedding representation of the process knowledge graph trained in step S2. Let the embedding vector of the second material be denoted as . The dimensional real vector also comes from the process knowledge graph embedding result of step S2. This represents the vector distance metric function; in this embodiment, Euclidean distance is used. The material similarity calculation formula is derived from a conventional transformation method that maps the distance metric to the 0-1 interval using an inverse proportional function.

[0098] The calculation of weld type similarity considers both weld form characteristics and weld size characteristics. Weld form characteristics include butt welds, fillet welds, and T-welds. Weld size characteristics include weld width and weld height. Weld form characteristics are represented using unique thermal coding.

[0099] The weld seam form similarity is calculated as follows: if the unique thermal encoding vectors of two weld seam types are completely identical, the form similarity is 1; otherwise, it is 0. The weld seam size similarity is calculated as follows: The weld width and height values ​​are combined into a two-dimensional numerical vector. The Euclidean distance between the two weld seam size vectors is calculated, and the distance value is converted into a size similarity score between 0 and 1 through an inverse proportional mapping. The weld seam type similarity is taken as the arithmetic mean of the weld seam form similarity and the weld seam size similarity.

[0100]

[0101] in, This represents the weld type similarity; it is a scalar real number with a value ranging from 0 to 1. This indicates the similarity of weld seam types, with a value of 0 or 1. The similarity of weld dimensions is represented by a real number between 0 and 1 obtained by converting the Euclidean distance of the dimensions through an inverse proportional mapping.

[0102] Equipment similarity is calculated based on a distance metric derived from equipment performance parameters. These parameters include rated current, rated voltage, and adjustable speed range. A three-dimensional numerical vector is constructed from these three performance parameters. The Euclidean distance between two equipment performance vectors is calculated, and the distance value is converted into a similarity score between 0 and 1 using an inverse proportional mapping.

[0103]

[0104] in, This represents the device similarity; it is a scalar real number with a value ranging from 0 to 1. The first device represents a performance parameter vector, which is a three-dimensional real number vector, with each dimension corresponding to the rated current, rated voltage, and adjustable speed range, respectively. This represents the performance parameter vector of the second device, a three-dimensional real number vector, with each dimension having the same meaning as... same. This represents the Euclidean distance metric function.

[0105] The calculation of working condition similarity comprehensively considers three sub-factors: welding method, welding position, and environmental conditions. Welding methods include gas shielded welding, laser welding, and resistance spot welding. Welding positions include flat welding, horizontal welding, vertical welding, and overhead welding. The similarity of welding methods is calculated as follows: if two welding methods are completely identical, the similarity is 1; otherwise, it is 0. The similarity of welding positions is calculated in the same way: if two welding positions are completely identical, the similarity is 1; otherwise, it is 0. Environmental conditions include two continuous numerical parameters: ambient temperature and ambient humidity. These are combined into a two-dimensional numerical vector, and the Euclidean distance is calculated. This vector is then converted into an environmental similarity score between 0 and 1 through an inverse proportional mapping. The working condition similarity is a weighted fusion of welding method similarity, welding position similarity, and environmental similarity.

[0106]

[0107] in, This represents the similarity of operating conditions. It is a scalar real number with a value ranging from 0 to 1. This indicates the similarity of welding methods, with a value of 0 or 1. This indicates the similarity of the welding positions, with a value of 0 or 1. This represents the environmental similarity, and is a real number between 0 and 1. , , These are the weighting coefficients for welding method similarity, welding position similarity, and environmental similarity, respectively. They are all positive real numbers and their sum is 1.

[0108] The final process similarity is obtained by weighted fusion of the similarity scores from the four dimensions mentioned above:

[0109]

[0110] in, The final overall process similarity score is represented by a scalar real number, ranging from 0 to 1. The larger the value, the higher the overall process similarity between the two welding tasks. , , , These are the weight coefficients for the four dimensions mentioned above, all of which are positive real numbers and sum to 1. The initial values ​​of the weight coefficients are determined by process experts based on the degree of influence of each dimension on welding quality, and are fine-tuned through feedback adjustment based on the actual transfer effect during subsequent transfer learning.

[0111] Based on the above final process similarity Based on the calculation results, the source domain task with the highest similarity to the target domain task process is selected as the source of knowledge transfer.

[0112] Step S302 involves adjusting the network parameters for domain adaptation based on the process similarity calculation results. After determining the most suitable source domain task for transfer in step S301, the policy network parameters trained on the source domain task need to be adjusted to adapt to the characteristics of the target domain task, thereby reducing the distribution difference between the source and target domains. Domain adaptation adjustment is achieved through domain discriminator constraints and domain adversarial training.

[0113] The domain discriminator is a neural network classifier whose input is the hidden layer feature representation of a deep reinforcement learning policy network. The output of the domain discriminator is the domain label prediction value. The domain label prediction value identifies whether the input sample comes from the source domain or the target domain.

[0114] Domain adversarial training comprises two opposing optimization objectives. The first objective is to minimize the domain relevance of the policy network, making it so that the hidden layer feature representations of the policy network cannot be accurately distinguished by the domain discriminator. This objective is achieved by backpropagating the classification loss of the domain discriminator as a negative gradient to the policy network after the gradient reversal layer.

[0115] The second optimization objective is to maximize the classification accuracy of the domain discriminator, that is, to ensure that the domain discriminator can distinguish the source of features as accurately as possible. This second optimization objective is achieved by updating the network parameters of the domain discriminator normally. The gradient reversal layer keeps the input unchanged and outputs directly during forward propagation, and inverts the input gradient before continuing to propagate during backward propagation.

[0116] Domain adaptation adjustment also employs the maximum mean difference alignment method as a supplement. Maximum mean difference alignment calculates the mean difference between the feature distributions of the source domain and the target domain in the regenerable kernel Hilbert space as a distribution distance metric, and achieves distribution alignment between the two domains in the feature space by minimizing this distribution distance metric. For a given set of source domain features… and target domain feature set The maximum mean difference is calculated as follows: calculate the mean vectors of the two sets of features in the regenerable kernel Hilbert space, and then calculate the Hilbert space distance between the two mean vectors. The value of the Hilbert space distance is the maximum mean difference.

[0117] The training convergence target of domain adaptation is: after a preset number of rounds of domain adversarial training, the classification accuracy of the domain discriminator is close to 50%, that is, the feature representation output by the policy network can no longer be effectively distinguished by the domain discriminator. At this point, it is considered that the distribution difference between the source domain and the target domain has been effectively reduced.

[0118] Step S303 involves constructing a few-sample fast adaptation mechanism based on a model-independent meta-learning framework. After domain adaptation and adjustment, the strategy for transferring to the target domain already possesses certain basic performance, but it usually requires further fine-tuning with a small amount of target domain data. Step S303, by constructing a model-independent meta-learning framework, enables the model to achieve good performance when facing new tasks with only a small number of samples and a small number of parameter update steps.

[0119] The training data for the meta-learning framework is organized into multiple welding tasks, each containing a support set and a query set. The support set is used for rapid parameter tuning within each welding task, while the query set is used to evaluate generalization performance across welding tasks. The meta-learning framework employs a two-layer optimization loop: an inner update loop and an outer update loop.

[0120] The inner layer update is performed on a single welding task. For each welding task sampled from the task distribution... With current initialization parameters As a starting point, in the welding task Calculate the loss function value on the support set Then, several steps of parameter updates are performed using the gradient descent algorithm to obtain temporary parameters specific to the welding task. The specific rules for inner layer updates are as follows:

[0121]

[0122] in, Indicates welding task The temporary parameters are obtained after the inner layer update. This represents the current initialization parameters, indicating the initial model parameters that the meta-learning framework is optimizing. The learning rate for inner layer updates is a positive real number hyperparameter. Represents the loss function Regarding parameters The gradient.

[0123] The outer update is performed on all welding tasks. After the inner updates of all welding tasks are completed, the loss function value for each welding task on its respective query set is calculated. The sum of the query set losses for all welding tasks is used as the outer layer total loss. The goal of the outer layer update is to minimize the outer layer total loss with respect to the initialization parameters. Functions:

[0124]

[0125] in, This represents the total number of welding tasks sampled in the current batch, and is a positive integer. The outer layer update uses the gradient descent algorithm, calculating the total outer layer loss with respect to the initialization parameters through backpropagation. The gradient is calculated, and the initialization parameters are updated accordingly. :

[0126]

[0127] in, The learning rate, representing the outer layer update, is a positive real number hyperparameter. In the gradient calculation of the outer layer update, It is itself The function is called, therefore it needs to be considered during backpropagation. right The impact.

[0128] The training convergence objective of the meta-learning framework is: after a preset number of outer iterations, the average loss value of each welding task on the query set no longer decreases, and training terminates when the average loss value falls below a preset threshold. The initialization parameters output after training are... This is the output of the meta-learning framework.

[0129] In summary, step S3 establishes a complete knowledge transfer mechanism, enabling the welding control strategy to be transferable and rapidly adaptable across different tasks. This mechanism identifies reusable knowledge sources through similarity matching, eliminates distributional differences between tasks through domain adaptation, and endows the model with the ability to quickly adapt to limited samples through meta-learning. The combined effect of these steps allows this method to benefit from existing experience when facing new working conditions, rather than starting from scratch each time.

[0130] See Figure 5 Step S4 involves establishing a digital twin virtual-real mapping model. Directly attempting trial and error on actual welding equipment is costly and risky. Step S4, by establishing a digital twin virtual-real mapping model, reproduces the welding process in a virtual environment, enabling the control strategy to be pre-trained and tested in the virtual environment, thereby reducing actual trial and error costs and improving safety. The specific implementation process of step S4 is as follows.

[0131] Step S401: Construct a physical model of the welding process. The physical model includes a heat conduction model, a fluid dynamics model, and a mechanical model.

[0132] The heat conduction model describes the heat transfer behavior during the welding process, and its partial differential equation form is:

[0133]

[0134] in, This indicates the density of the material. This indicates specific heat capacity. Temperature is a function of time and spatial coordinates. Indicates time. This indicates thermal conductivity. The term representing the heat source indicates the energy input from the welding heat source. The partial differential equation for heat conduction originates from the law of conservation of energy in heat transfer.

[0135] The fluid dynamics model describes the flow behavior of metal inside the molten pool. It is based on the Navier-Stokes equations and introduces the arc pressure term, surface tension term, and electromagnetic force term as volume force source terms to characterize the special driving force on the molten pool during the welding process.

[0136] The magnitude of the arc pressure is proportional to the square of the welding current. Surface tension drives the liquid metal near the surface of the molten pool to flow from the high-temperature zone to the low-temperature zone. The electromagnetic force originates from the effect of the magnetic field generated by the welding current on the conductive fluid. Solving the fluid dynamics model yields the velocity vector field and pressure scalar field distribution inside the molten pool.

[0137] The mechanical model describes the stress-strain behavior during welding. It is based on the elastoplastic mechanics equations and introduces a thermal strain term to characterize expansion or contraction caused by temperature changes, a phase transformation strain term to characterize the volume change accompanying solid-state phase transformation, and a residual stress term to characterize the stress remaining in the workpiece after cooling to room temperature. Solving the mechanical model yields the stress tensor distribution inside the workpiece during welding and the residual deformation after welding.

[0138] The physical model is solved numerically using the finite element method, with a finer mesh applied in the region near the heat source to improve computational accuracy.

[0139] Step S402: Construct a data-driven model for the welding process. The data-driven model includes a neural network model and a Gaussian process model, both of which are learned based on historical welding process data.

[0140] The neural network model takes into account the state and process parameters of the welding process as inputs. State parameters include workpiece thickness, bevel angle, and ambient temperature. Process parameters include welding current, welding voltage, welding speed, wire feed speed, and shielding gas flow rate. The output of the neural network model is the weld quality prediction, including predicted weld penetration, weld width, and weld bead formation grade. The neural network model employs a fully connected neural network architecture.

[0141] Preprocessing of training data includes data cleaning, outlier handling, and feature standardization. The loss function of the neural network model is the mean squared error between the predicted values ​​and the actual measured values ​​of the output layer, and the training convergence objective is that the mean squared error value on the validation set no longer decreases after multiple consecutive training epochs.

[0142] The input to the Gaussian process model is the same as that of the neural network model. The output of the Gaussian process model is the predicted mean and the predicted variance. The predicted mean is the central estimate of the quality index, and the predicted variance quantifies the degree of uncertainty in the prediction results. The training data preprocessing method is the same as that of the neural network model; training determines the hyperparameters in the kernel function by maximizing the marginal likelihood function.

[0143] Step S403 establishes a weighted fusion framework for the physical model and the data-driven model. The physical model, based on physical laws, has advantages such as strong interpretability and good extrapolation ability, but its accuracy is limited under complex boundary conditions. The data-driven model, based on data learning, has advantages such as strong fitting ability and fast computation speed, but it depends on the coverage of training data. This step achieves complementary advantages of the two through a weighted fusion method.

[0144] The mathematical representation of the fusion framework is as follows:

[0145]

[0146] in, This represents the fused prediction result vector, with each dimension corresponding to the fused prediction values ​​of melt depth, melt width, and forming grade, respectively. This represents the vector of prediction results from the physical model. This represents the vector of prediction results from the data-driven model. This represents the fusion weight coefficient, with a value ranging from 0 to 1. The fusion operation is a weighted sum of independent scalar values ​​for each dimension.

[0147] Fusion weighting coefficient An adaptive adjustment strategy is employed to determine this. The recent prediction errors of the physical model are compared. and recent prediction errors of data-driven models Size: when Increase when smaller ,when Decrease when smaller . The value of is restricted to the interval [0.1, 0.9]. The error measurement method is as follows: for multiple welding tasks recently completed, the mean absolute error of each quality index between the predicted values ​​of the two models and the actual measured values ​​is calculated.

[0148] The fusion framework also employs a switching strategy as a supplement: when welding parameters are within the coverage of historical training data, the physical model is used first; when welding parameters are close to equipment limits or material combinations are rarely seen in historical data, the model is switched to a data-driven model. The switching decision is based on whether the minimum Euclidean distance between the current input parameter vector and each sample in the historical training dataset exceeds a preset threshold.

[0149] Steps S401 to S403 together complete the construction of the simulation capability for the virtual welding process. The physical model provides prior knowledge constraints, the data-driven model provides experience correction capabilities, and the fusion framework unifies the output of both, thereby obtaining more accurate prediction results than a single model.

[0150] Step S404 enables the synchronous update of real-time sensor data and virtual model. The synchronous update mechanism establishes a real-time data channel from the physical welding equipment to the virtual model, including three stages: sensor data acquisition, data transmission, and data preprocessing.

[0151] The sensor data acquisition module is responsible for acquiring real-time data from current sensors, voltage sensors, speed sensors, temperature sensors, and vision sensors. The current and voltage sensors acquire data at a frequency of 1000Hz, the speed and temperature sensors at a frequency of 100Hz, and the vision sensor at a frequency of 30 frames per second.

[0152] The data transmission module adopts the industrial Ethernet communication protocol, and the data transmission latency is controlled in the millisecond range.

[0153] The data preprocessing module filters, synchronizes, and converts the format of the raw sensor data. Data filtering uses a low-pass filter to remove high-frequency noise. Data synchronization aligns sensor data from different sampling frequencies to a unified time base according to timestamps. Data format conversion converts the raw sensor readings into the physical quantity units required by the virtual model.

[0154] The virtual model's state update is implemented using the Kalman filter algorithm. The input to the Kalman filter algorithm is a vector of real-time sensor measurements. and physical model predicted value vector The output is the fused optimal state estimate vector. .

[0155] State vector It includes the current temperature field characteristics, molten pool geometry, and the location of the heat source center. The Kalman filter algorithm consists of two steps: prediction and update. The prediction step calculates the prior state estimate based on the state transition matrix of the physical model, and the update step corrects the prior state estimate based on sensor measurements. The corrected state estimate is used as the current state of the virtual model.

[0156] Step S405: Construct an error correction mechanism. The error correction mechanism corrects the model parameters by comparing the deviation between the virtual model output and the actual welding results to ensure consistency between the virtual and real models.

[0157] The error correction process is as follows: First, calculate the deviation between the virtual model's predicted output and the actual welding result, and then adjust the virtual model's predicted penetration depth value. Compared with the actual measured melt depth value The difference yields the melt depth deviation, and the melt width deviation and deformation deviation are calculated in the same way. Then, based on the deviation, the correction amount of the model parameters is estimated, and an online learning algorithm is used to calculate the gradient of the deviation relative to the model parameters. The correction amount is obtained by multiplying the deviation by the correction step size coefficient and the gradient direction.

[0158] Next, update the virtual model parameters based on the correction amount. Finally, verify the predictive performance of the corrected model. If the recalculated deviation is lower than a preset threshold, the correction is confirmed to be effective; otherwise, repeat the above steps until the deviation converges.

[0159] Error estimation is achieved using an adaptive Kalman filter algorithm, which dynamically adjusts the noise covariance parameter by calculating the covariance matrix of the innovation sequence in real time.

[0160] The error correction mechanism is triggered by two conditions: timed triggering and abnormal triggering. Timed triggering occurs at fixed time intervals. Abnormal triggering occurs immediately when the prediction deviation exceeds a preset threshold.

[0161] In summary, step S4 completes the establishment of the digital twin virtual-real mapping model through the above five sub-steps. This virtual environment provides a simulation platform for the offline pre-training in step S5, enabling the control strategy to be pre-trained in the virtual environment before being deployed to the actual device.

[0162] See Figure 6 Finally, in step S5, a collaborative mechanism for offline pre-training and online fine-tuning is established. The digital twin virtual environment established in step S4 provides a low-cost training platform for the control strategy, but there are always differences between the virtual environment and the real physical environment. Step S5, by establishing a collaborative mechanism for offline pre-training and online fine-tuning, first completes the offline pre-training of the strategy in the virtual environment, and then migrates the strategy to the real device for continuous online fine-tuning, thereby achieving a safe and smooth transition from virtual to real. The specific implementation process of this step is as follows.

[0163] Step S501: Complete offline reinforcement learning pre-training of the control strategy in the digital twin virtual environment. In the digital twin virtual environment, use a virtual welding process simulator to perform offline reinforcement learning pre-training of the control strategy.

[0164] The offline pre-training data sources include two categories. The first category is historical process databases, which store experiential data from real welding processes, including state sequences, action sequences, and corresponding reward feedback. The second category is digital twin simulation-generated data, which is randomly explored and generated through a virtual welding process simulator to cover working conditions not present in historical data.

[0165] The optimization objective of offline pre-training is to maximize the expected cumulative reward in a virtual environment. The expected cumulative reward is calculated as follows: for each round of the welding process, the instantaneous reward value acquired at each control moment is accumulated along the time axis to obtain the total cumulative reward for that round. The calculation formula is as follows:

[0166]

[0167] in, Indicates the first Total cumulative reward for each training round Indicates the first Total number of time steps in each training round Indicates the first In the training round, the first The instantaneous reward value acquired at each control time point is calculated using the reward function in step S103. The expected cumulative reward is the arithmetic mean of the total cumulative rewards across all training rounds. The objective of updating the policy network parameters is to maximize this expected cumulative reward.

[0168] Offline pre-training uses - A greedy strategy balances action exploration and utilization, with a high initial exploration probability that gradually decreases to a lower value as training rounds increase. Offline pre-training employs a strategy that combines turn-based and single-step training.

[0169] Round-based training uses a complete welding task as one training round. After each round, the strategy is updated based on the total cumulative reward, making it suitable for learning global process parameter planning. Single-step training uses each control moment as the basic unit, updating the strategy in real time based on the instantaneous reward, making it suitable for learning fine-tuning of local parameters. The termination condition for offline pre-training is that the average cumulative reward value reaches a preset performance index threshold or stops increasing after several consecutive training rounds.

[0170] Step S502 involves designing a security constraint mechanism and migrating the pre-trained strategy to the real device. While the pre-trained strategy performs well in the virtual environment, direct deployment to the real device may pose security risks. Step S502 ensures the safety of the welding process during strategy migration by designing a security constraint mechanism.

[0171] The safety constraint mechanism comprises two parts: parameter safety threshold definitions and anomaly detection and handling procedures. Parameter safety thresholds are determined comprehensively based on welding process standards, equipment manuals, and material safety specifications, covering the safe ranges of key parameters such as welding current, welding voltage, welding speed, and workpiece temperature. The anomaly detection mechanism monitors key parameters in real time during the welding process, triggering the anomaly handling procedure when any parameter exceeds its corresponding safety threshold.

[0172] The anomaly handling process is divided into three levels. The first level is alarm notification, which issues a warning message to the operator. The second level is automatic degradation, where the control program automatically adjusts the welding parameters to a safe range. The third level is safe shutdown, which automatically cuts off the welding power and stops all moving mechanisms when the abnormal situation is severe or automatic degradation fails.

[0173] The safety constraint mechanism also directly affects the action output of the pre-trained strategy. The action output is processed by the safety constraint layer before being sent to the welding equipment. The safety constraint layer limits the adjustment amounts in each dimension of the action output to ensure that the parameter commands finally sent to the equipment are always within the safety margin.

[0174] Step S503 involves online fine-tuning and strategy iteration based on real-time feedback. After the pre-trained strategy is deployed to the real equipment, real-time feedback data from the actual welding process can be used for continuous optimization of the strategy. Step S503, through the online fine-tuning and strategy iteration mechanism, enables the control strategy to continuously learn and improve during the actual welding process.

[0175] Online fine-tuning utilizes feedback data from the actual welding process to subtly adjust the pre-trained strategy, adapting to the differences between real-world conditions and the virtual environment. Online fine-tuning employs a policy gradient algorithm. This algorithm calculates the gradient estimates of the policy parameters based on the reward feedback from the real welding process, and then updates the policy parameters along the gradient ascent direction. The mathematical form of the policy gradient estimate is:

[0176]

[0177] in, This represents the parameter vector of the policy network. Indicates the expected cumulative reward. Indicates the state Select action The conditional probability, The advantage function is calculated by adding the discounted next-state value estimate to the immediate reward value and then subtracting the current state value estimate. This formula is the standard form of the policy gradient theorem, derived from classic theory in reinforcement learning. Gradient calculation is a global vector operation, with all dimensions updated simultaneously.

[0178] The online fine-tuning update frequency and update magnitude are adaptively adjusted according to the degree of change in operating conditions. The degree of change in operating conditions is determined by calculating the Euclidean distance between the current state vector and the previous state vector. When the distance exceeds a preset threshold, it is determined that the operating conditions have changed significantly, and the update frequency and update step size are increased. When the operating conditions stabilize, the update frequency and update step size are restored to the baseline update frequency and update step size.

[0179] The online fine-tuning data utilizes an experience playback mechanism. This mechanism stores sample data collected during the actual welding process in an experience playback buffer, with each sample data point being a quadruple. Each parameter includes a state, action, reward, and next-time state. During policy updates, multiple sets of samples are randomly sampled from the buffer to form update batches, and the average gradient of these batches is used to update the parameters. The stability of online fine-tuning is guaranteed by trust region constraints, limiting the magnitude of parameter changes in each policy update.

[0180] Strategy iteration requires periodically collecting new data and updating strategy parameters to achieve continuous evolution of the control strategy. The strategy iteration cycle is set to a comprehensive strategy update after the completion of multiple complete welding tasks. The data collection process for strategy iteration includes: executing the current strategy for welding production, recording the status, actions, and reward feedback at each control moment during the welding process, saving the final weld quality inspection results and workpiece deformation measurement results, and storing all the above data in the strategy iteration database.

[0181] The parameter update process of policy iteration includes: sampling historical training data from the database to form training batches, calculating policy gradient estimates, executing gradient update steps to update policy network parameters, and synchronously updating the parameter estimates of the value network.

[0182] The strategy iteration also incorporates knowledge graph updates: after each strategy iteration, if a new effective process pattern is discovered, that is, a certain combination of process parameters has yielded high-quality weld results in multiple welding processes, then the process pattern is added as new knowledge to the process knowledge graph constructed in step S2.

[0183] In summary, step S5 constitutes the complete implementation process of the offline pre-training and online fine-tuning collaborative mechanism. This mechanism first obtains a control strategy with good initial performance through offline pre-training in a virtual environment, then smoothly transfers the strategy to real equipment through a safety constraint mechanism, and finally continuously optimizes the strategy in actual production through online fine-tuning and strategy iteration, enabling the control strategy to maintain good adaptability under different operating conditions.

[0184] For ease of understanding, an application example is provided below. In a specific application scenario, the intelligent control method for the welding process under multi-field coupling conditions described above can be applied to the body-in-white welding production line in the automotive manufacturing industry. The automotive body-in-white welding production line involves various materials such as low-carbon steel, high-strength steel, and aluminum alloys; various weld types such as spot welds, arc welds, and laser welds; and various welding equipment such as resistance spot welders, gas shielded welders, and laser welders.

[0185] When applying the intelligent control method provided in this embodiment, the following process is executed:

[0186] First, a process knowledge graph is constructed based on historical welding process data to establish the correlation between materials, weld types, process parameters, and quality indicators.

[0187] Then, for the body-in-white welding task of a specific vehicle model, similar historical tasks are retrieved from the process knowledge graph, and the transfer learning strategy is determined based on the process similarity calculation results.

[0188] Next, the control strategy is pre-trained offline in a digital twin virtual environment, and prior knowledge from the process knowledge graph is incorporated during the training process to accelerate strategy convergence.

[0189] After pre-training is completed, the pre-trained strategy is deployed to real welding equipment, and a safety constraint mechanism is enabled to ensure the safety of the welding process.

[0190] During actual welding, the pre-trained strategy is fine-tuned online using real-time sensor data to continuously optimize the control strategy to adapt to actual working conditions. This online fine-tuning process accumulates new experience through a strategy iteration mechanism, enabling the continuous evolution of the control strategy.

[0191] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention. Therefore, the embodiments should be regarded as exemplary and non-limiting in all respects.

[0192] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A method for intelligent control of welding processes under multi-field coupling conditions, characterized in that, include: A dynamic optimization framework for welding parameters based on reinforcement learning is established, including constructing a state space, defining an action space and action constraints, designing a weighted reward function, and using a soft Actor-Critic algorithm to train the policy network and value network. Establish a digital twin virtual-real mapping model, including constructing a heat conduction model, a fluid dynamics model, and a mechanical model as physical models, constructing a neural network model and a Gaussian process model as data-driven models, establishing a weighted fusion framework of physical models and data-driven models, and using Kalman filtering to realize real-time synchronous update and error correction of the state of sensor data and virtual models; In the virtual environment constructed by the digital twin virtual-real mapping model, the policy network is pre-trained offline using reinforcement learning. The design incorporates a safety constraint mechanism that includes parameter safety thresholds and anomaly detection handling. The pre-trained policy network is deployed to a real welding device. Real-time feedback data from the actual welding process is used to fine-tune the policy network online through a policy gradient algorithm.

2. The intelligent control method for welding process under multi-field coupling conditions according to claim 1, characterized in that, This also includes building a process knowledge graph: Extract material entities, weld type entities, equipment entities, process parameter entities, and quality index entities from historical welding process data, and establish process correlation relationships between these entities to form a heterogeneous map. By minimizing the marginal loss function, each entity and each relation in the heterogeneous graph is mapped to a low-dimensional real vector. Negative samples are generated by replacing entities or relations in positive samples, including randomly replacing the head entity of the positive sample with other entities in the heterogeneous graph, randomly replacing the tail entity of the positive sample with other entities in the heterogeneous graph, or replacing the relation of the positive sample with other relation types. The loss function is: Where L is the loss function value, γ is the marginal hyperparameter, and dist is the Euclidean distance function. , , These are the embedding vectors of the head entity, relation, and tail entity of the positive sample, respectively. , , These are the embedding vectors of the head entity, relation, and tail entity of the negative sample, respectively, and [·]_+ indicates the operation of taking the positive part.

3. The intelligent control method for welding process under multi-field coupling conditions according to claim 2, characterized in that, It also includes establishing causal relationship chains based on the aforementioned low-dimensional real vectors: Using a combination of specific material nodes and specific process parameter nodes as query conditions, the corresponding quality result measurement values ​​are retrieved from the heterogeneous spectrum; The causal strength value of the process parameters on the quality results is calculated. This causal strength value is defined as the ratio of the absolute value of the Pearson correlation coefficient between the process parameters and the quality results to the standard deviation of the quality results to the standard deviation of the process parameters. ,in This represents the causal strength value. The Pearson correlation coefficient is used. The standard deviation of historical data for process parameters. The standard deviation of historical data for quality results; The causal strength value is used as the weight of the directed edge from the process parameter node to the quality result node to form a causal relationship chain graph.

4. The intelligent control method for welding process under multi-field coupling conditions according to claim 1, characterized in that, This also includes establishing a cross-domain transfer learning mechanism: The process similarity between the new welding task and the existing welding task is calculated. The process similarity is obtained by weighted fusion of material similarity, weld type similarity, equipment similarity and working condition similarity. Among them, the weld type similarity is the arithmetic mean of weld form similarity and weld size similarity. The existing welding task with the highest process similarity is selected as the source domain, and its strategy network parameters are transferred to the new task.

5. The intelligent control method for welding process under multi-field coupling conditions according to claim 4, characterized in that, This also includes domain adaptation and meta-learning fine-tuning of the transferred policy network: The classification loss of the domain discriminator is backpropagated to the policy network in the form of negative gradient through the gradient reversal layer for adversarial training, and the maximum mean difference between the source domain and target domain features in the regenerable kernel Hilbert space is minimized to achieve domain adaptation. When performing meta-learning fine-tuning, for each sampling task, starting from the current initialization parameters, it is performed on its support set through... Perform inner layer update to obtain temporary parameters Then minimize the sum of losses of all sampling tasks on their query set with respect to the initialization parameters. The function, where This represents the inner learning rate.

6. The intelligent control method for welding process under multi-field coupling conditions according to claim 1, characterized in that, The state vector of the state space is composed of material property features, workpiece geometric features and real-time sensing features in a preset order; The material properties include the coded value mapped to the material grade, the mass percentage of each element in the chemical composition, mechanical properties such as yield strength and tensile strength, and thermophysical properties such as thermal conductivity and specific heat capacity. The workpiece geometric features include the workpiece thickness value, bevel angle value, three-dimensional coordinate value of the weld center position, and geometric description information of the adjacent structures of the part to be welded; The real-time sensing features include the real-time values ​​of welding current, welding voltage, wire feed speed, welding speed, shielding gas flow rate, as well as statistical features within a preset time window, molten pool geometry features, key temperature features, and arc acoustic signal features.

7. The intelligent control method for welding process under multi-field coupling conditions according to claim 1, characterized in that, The motion vectors in the motion space include welding current adjustment, welding voltage adjustment, welding speed adjustment, wire feed speed adjustment, and shielding gas flow rate adjustment. The action constraints include hard constraints that independently limit the adjustment amount of each dimension, and soft constraints that impose penalties on actions that exceed the rated working range through the safety constraint penalty component in the weighted reward function.

8. The intelligent control method for welding process under multi-field coupling conditions according to claim 1, characterized in that, The weighted reward function is a weighted sum of the weld quality reward component, the workpiece deformation penalty component, the energy consumption reward component, and the safety constraint penalty component. The weld quality bonus is calculated based on the weld appearance forming grade, relative deviation of penetration depth, relative deviation of penetration width, and deduction of internal defects. The workpiece deformation penalty component is calculated based on the negative of the sum of the three-dimensional deformation amounts at preset measurement points of the workpiece after welding; The energy consumption reward component is calculated by taking the opposite of the weighted sum of the unit costs of electricity consumption and gas consumption. The safety constraint penalty component triggers an independent penalty when the welding current exceeds 110% of the rated value, the welding voltage deviates from the target value by more than 15%, or the local temperature of the workpiece exceeds 80% of the solidus temperature of the material.

9. The intelligent control method for welding process under multi-field coupling conditions according to claim 2, characterized in that, The safety constraint mechanism includes three levels of abnormal handling: alarm prompt, automatic degradation, and safe shutdown, when any welding parameter is detected to exceed the corresponding safety threshold. The online fine-tuning also includes: storing quadruple samples of the actual welding process in the experience replay buffer, and randomly sampling from the buffer to update the strategy gradient; collecting data after completing multiple complete welding tasks, performing strategy iteration, and adding the discovered process parameter combinations that meet the preset quality conditions to the process knowledge graph.