Welding quality self-optimization method based on multi-source sensing and deep reinforcement learning

Abstract

The application discloses a welding quality self-optimization method based on multi-source sensing and deep reinforcement learning, relates to the technical field of intelligent welding control, and comprises the following steps: collecting time sequence signals of welding current, welding speed, wire feeding speed, welding gun height and protective gas flow in a welding process, and constructing a real-time multi-source sensing data set; performing multi-modal feature alignment and fusion processing on the multi-source sensing data to generate a fusion state feature tensor; introducing the feature tensor into an improved SAC reinforcement learning algorithm, outputting a multi-class welding parameter adjustment action sequence based on a preset welding quality reward function, correcting existing welding parameters online according to the sequence, and transmitting an optimized process parameter set to a welding execution mechanism. The method can realize self-adaptive adjustment of welding process parameters, accurately adapt to dynamic changes of welding operation conditions, and perfect an intelligent closed-loop control mode of the welding process.

Description

Technical Field

[0001] This invention belongs to the field of intelligent welding control technology, specifically a welding quality self-optimization method based on multi-source sensing and deep reinforcement learning. Background Technology

[0002] Welding process parameter control is a crucial step in ensuring the weld formation state. Currently, conventional welding production generally uses fixed preset process parameters, limiting the collection of only a single type of sensor data during the welding process. It fails to simultaneously collect and acquire welding current timing signals, welding speed timing signals, wire feed speed timing signals, welding torch height timing signals, and shielding gas flow timing signals. Welding operation conditions exhibit dynamic changes, and fixed process parameters cannot adapt to these dynamic fluctuations. The limited dimensions of single-sensor data cannot fully characterize the actual operating state of the entire welding process, making it difficult to provide comprehensive data support for refined process control.

[0003] Existing technologies lack standardized processing mechanisms for multi-source heterogeneous sensor data, making it impossible to complete multimodal feature alignment and fusion processes, and unable to generate fused state feature tensors that can fully represent working condition information. Traditional welding parameter control does not incorporate deep reinforcement learning algorithm architectures, nor does it match reward function setting logic adapted to welding quality standards. It cannot simultaneously output quantitative adjustment amounts for multiple welding parameters, making it difficult to perform online correction operations on existing welding parameters, and unable to construct a closed-loop feedback control chain for process parameters. Summary of the Invention

[0004] This invention aims to solve at least one of the technical problems existing in the prior art; To this end, this invention proposes a welding quality self-optimization method based on multi-source sensing and deep reinforcement learning, comprising: Acquire a real-time multi-source sensor data set during the welding process, the real-time multi-source sensor data set including welding current timing signal, welding speed timing signal, wire feed speed timing signal, welding torch height timing signal, and shielding gas flow rate timing signal; Perform multimodal feature alignment and fusion processing on the real-time multi-source sensor data set to generate a fused state feature tensor; The fusion state feature tensor is input into the workflow of the improved SAC reinforcement learning algorithm. The improved SAC reinforcement learning algorithm outputs a welding parameter adjustment action sequence based on the fusion state feature tensor and the preset welding quality reward function. The welding parameter adjustment action sequence includes the target welding current adjustment amount, the target welding speed adjustment amount, the target wire feed speed adjustment amount, the target welding torch height adjustment amount, and the target shielding gas flow rate adjustment amount. The current welding parameters are corrected online according to the welding parameter adjustment action sequence to generate an optimized welding process parameter set; the optimized welding process parameter set is fed back to the welding actuator to trigger the adaptive adjustment operation of the welding parameters.

[0005] Furthermore, the step of performing multimodal feature alignment and fusion processing on the real-time multi-source sensing data set to generate a fused state feature tensor specifically includes: The welding current timing signal is subjected to sliding window sampling processing to obtain multiple current time window segments; the welding speed timing signal is subjected to sliding window sampling processing to obtain multiple welding speed time window segments. Perform sliding window sampling processing on the wire feeding speed timing signal to obtain multiple wire feeding speed time window segments; Perform sliding window sampling processing on the welding torch height timing signal to obtain multiple welding torch height time window segments; Perform sliding window sampling processing on the protective gas flow time sequence signal to obtain multiple protective gas flow time window segments; The multiple current time window segments, the multiple welding speed time window segments, the multiple wire feed speed time window segments, the multiple welding torch height time window segments, and the multiple shielding gas flow time window segments are aligned and spliced ​​according to timestamps to generate a multi-channel time series data matrix. Normalization scaling is performed on each channel in the multi-channel time series data matrix to map the data value range of each channel to a unified interval, thereby generating a normalized multi-channel time series data matrix. The normalized multi-channel time-series data matrix is ​​subjected to convolutional dimensionality reduction processing to extract high-dimensional feature vectors at each time step. The high-dimensional feature vectors at each time step are stacked along the time dimension to generate the fused state feature tensor.

[0006] Furthermore, the step of inputting the fused state feature tensor into the workflow of the improved SAC reinforcement learning algorithm specifically includes: The Actor policy network in the improved SAC reinforcement learning algorithm is invoked to perform action probability distribution mapping on the fused state feature tensor to generate an action probability distribution parameter set, which includes a mean parameter vector and a log-standard deviation parameter vector. Based on the set of action probability distribution parameters, reparameterization sampling is performed to generate the original welding parameter adjustment action sequence; the first Critic evaluation network in the improved SAC reinforcement learning algorithm is called to perform joint value evaluation processing on the fused state feature tensor and the original welding parameter adjustment action sequence to generate the first action value score; The improved SAC reinforcement learning algorithm is used to call the second Critic evaluation network to perform joint value evaluation processing on the fused state feature tensor and the original welding parameter adjustment action sequence, and generate a second action value score. Perform a minimum value selection operation on the first action value score and the second action value score to generate a conservative action value score; The original welding parameter adjustment action sequence is modified based on the conservative action value score to generate the welding parameter adjustment action sequence.

[0007] Furthermore, the construction process of the preset welding quality reward function in the improved SAC reinforcement learning algorithm specifically includes: The system acquires real-time arc sound signals, electrical signals, and arc light signals during the welding process, wherein the electrical signals include arc voltage signals and welding current signals. Perform a short-time Fourier transform on the electric arc sound signal to generate a sound spectrum feature matrix; The arc light signal is subjected to spectral decomposition processing to generate a spectral intensity distribution vector; The electrical signal is subjected to time-domain feature extraction processing to generate a time-domain statistical feature set of the electrical signal, which includes mean value features, variance features and peak value features. The sound spectrum feature matrix, the spectral intensity distribution vector, and the electrical signal time-domain statistical feature set are input into a pre-trained welding stability scoring model to generate a welding stability score. The welding quality reward function is constructed based on the welding stability score and welding process constraints.

[0008] Furthermore, the welding quality reward function includes a welding stability reward component, a welding speed reward component, and a welding torch height penalty component; the value of the welding stability reward component is directly proportional to the welding stability score; the value of the welding speed reward component is directly proportional to the current value of the welding speed time-series signal; the value of the welding torch height penalty component is inversely proportional to the absolute value of the deviation of the current value of the welding torch height time-series signal from the preset reference height, and the welding torch height penalty component takes a negative value when the current value of the welding torch height time-series signal deviates from the preset reference height by more than a preset threshold; the welding stability reward component, the welding speed reward component, and the welding torch height penalty component are linearly combined according to preset weighting coefficients to generate the final output value of the welding quality reward function.

[0009] Furthermore, the working principle of the improved SAC reinforcement learning algorithm includes a policy iteration process based on the maximum entropy framework, specifically including: Collect historical state transition trajectory data generated by the welding actuator after performing historical welding parameter adjustment actions within a historical time period. The historical state transition trajectory data includes historical state tensor, historical action tensor, historical reward value and state tensor at the next moment. The historical state transition trajectory data is stored in the experience replay buffer pool; Randomly sample batches of historical state transition trajectory data from the experience replay buffer pool to form training batch data; The gradient information of the policy network is calculated based on the training batch data; the temporal difference error of the first Critic evaluation network and the second Critic evaluation network is calculated based on the training batch data. Based on the gradient information of the policy network, perform gradient descent update operation on the network parameters of the policy network; Gradient descent update operation is performed on the network parameters of the first Critic evaluation network and the second Critic evaluation network based on the temporal difference error of the first Critic evaluation network and the second Critic evaluation network.

[0010] Furthermore, the improved SAC reinforcement learning algorithm also includes an automatic temperature parameter adjustment process, specifically including: Initialize the target entropy parameter, and set the value of the target entropy parameter to the negative value of the action dimension; Initialize the logarithm of the learnable temperature parameter; Sample batch state tensors from the experience replay buffer; The batch state tensor is input into the policy network to generate batch action probability distribution parameters; Based on the batch action probability distribution parameters, a batch sampling action tensor is generated; Calculate the actual entropy value of the batch sampling action tensor under the probability distribution parameters of the batch action; The loss gradient of the temperature parameter is calculated based on the difference between the actual entropy value and the target entropy parameter; A gradient descent update operation is performed on the logarithm of the learnable temperature parameters based on the loss gradient of the temperature parameters; Perform an exponential transformation on the updated learnable temperature parameters to generate the temperature parameter values ​​for the current moment.

[0011] Furthermore, the construction and training process of the pre-trained welding stability scoring model specifically includes: Multiple sets of labeled training samples were collected during the welding process under various welding conditions. Each set of labeled training samples included arc sound spectrum feature matrix samples, spectral intensity distribution vector samples, electrical signal time-domain statistical feature samples, and manually labeled welding stability rating labels. The sample of the arc sound spectrum feature matrix is ​​input into the first branch of the convolutional neural network to generate a sound embedding feature vector; The spectral intensity distribution vector sample is input into the second convolutional neural network branch to generate a spectral embedding feature vector; The time-domain statistical feature samples of the electrical signal are input into a fully connected network branch to generate an electrical signal embedding feature vector; The sound embedding feature vector, the spectral embedding feature vector, and the electrical signal embedding feature vector are concatenated to generate a fused embedding feature vector. The fused embedded feature vector is input into the output layer network to generate a predicted welding stability score. A loss function is constructed based on the error between the predicted welding stability score and the manually labeled welding stability score. The network parameters of the first convolutional neural network branch, the second convolutional neural network branch, the fully connected network branch, and the output layer network are jointly optimized through the backpropagation algorithm until the loss function converges, thus obtaining the pre-trained welding stability scoring model.

[0012] Furthermore, the step of adjusting the current welding parameters online according to the welding parameter adjustment action sequence to generate an optimized set of welding process parameters specifically includes: Obtain the current welding parameter set, which includes the current welding current value, the current welding speed value, the current wire feed speed value, the current welding torch height value, and the current shielding gas flow rate value; The target welding current adjustment amount in the welding parameter adjustment action sequence is added to the current welding current value to generate an optimized welding current value. The target welding speed adjustment amount in the welding parameter adjustment action sequence is added to the current welding speed value to generate an optimized welding speed value. The target wire feed speed adjustment amount in the welding parameter adjustment action sequence is added to the current wire feed speed value to generate the optimized wire feed speed value. The target welding torch height adjustment amount in the welding parameter adjustment action sequence is added to the current welding torch height value to generate the optimized welding torch height value. The target shielding gas flow rate adjustment amount in the welding parameter adjustment action sequence is added to the current shielding gas flow rate value to generate the optimized shielding gas flow rate value. The optimized welding current value, the optimized welding speed value, the optimized wire feed speed value, the optimized welding torch height value, and the optimized shielding gas flow rate value are packaged into the optimized welding process parameter set.

[0013] Furthermore, the step of feeding back the optimized welding process parameter set to the welding actuator to trigger the adaptive adjustment of welding parameters specifically includes: Extract the optimized welding current value, optimized welding speed value, optimized wire feed speed value, optimized welding torch height value, and optimized shielding gas flow rate value from the optimized welding process parameter set; The optimized welding current value is converted into an analog control signal and output to the current adjustment interface of the welding power supply. The optimized welding speed value is converted into a pulse frequency control signal and output to the drive motor controller of the welding walking mechanism; The optimized wire feeding speed value is converted into a speed control command and output to the servo driver of the wire feeding mechanism; The optimized welding torch height value is converted into a displacement control command and output to the stepper motor controller of the welding torch height adjustment mechanism; The optimized protective gas flow rate value is converted into a valve opening control signal and output to the actuator of the protective gas flow rate regulating valve; After the welding actuator completes the adaptive adjustment operation, it returns to the step of acquiring the real-time multi-source sensor data set during the welding process, forming a closed-loop iterative optimization loop.

[0014] Compared with the prior art, the beneficial effects of the present invention are: A complete sensor data set is formed by collecting welding current timing signals, welding speed timing signals, wire feed speed timing signals, welding torch height timing signals, and shielding gas flow timing signals during the welding process. Multimodal feature alignment and fusion processing are performed on the heterogeneous multi-source sensor data to construct a structured fused state feature tensor. Feature alignment eliminates differences in data dimensionality and timing, while fusion processing integrates the operating condition information carried by different sensor signals, uncovers the correlations between various welding process parameters, refines the granularity of the welding condition state representation, and enriches the dimensionality of the operating condition information at the algorithm input.

[0015] The improved SAC reinforcement learning algorithm runtime chain integrates state feature tensors. Relying on a reward function aligned with welding quality standards, it synchronously generates adjustment values ​​for multiple welding parameters and assembles them into an action sequence. Following this action sequence, it performs online correction of existing welding parameters and delivers the optimized process parameters to the welding actuator. The reinforcement learning algorithm can autonomously iterate parameter control logic based on real-time operating conditions, simultaneously covering the dynamic fine-tuning of five core welding process parameters. This adapts to the irregular fluctuations in operating conditions at the welding site, forming a continuous closed-loop autonomous adjustment mode for process parameters. Attached Figure Description

[0016] Figure 1 This is a flowchart of the welding quality self-optimization method based on multi-source sensing and deep reinforcement learning described in this invention. Figure 2 A flowchart for generating fused state feature tensors for multimodal feature alignment and fusion processing; Figure 3 A flowchart for generating an optimized set of welding process parameters by online correction of current welding parameters; Figure 4 This is a graph showing the change of the sensor signal over time. Figure 5 Numerical plot of the fusion matrix; Figure 6 This is a graph showing the change in the loss function. Figure 7 To optimize the algorithm iteration path graph. Detailed Implementation

[0017] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] See Figure 1This invention discloses a welding quality self-optimization method based on multi-source sensing and deep reinforcement learning. The method acquires a real-time multi-source sensing data set during the welding process, including welding current timing signals, welding speed timing signals, wire feed speed timing signals, welding torch height timing signals, and shielding gas flow rate timing signals. Then, it performs multi-modal feature alignment and fusion processing on the real-time multi-source sensing data set to generate a fused state feature tensor. This fused state feature tensor is input into the workflow of an improved SAC reinforcement learning algorithm. The improved SAC reinforcement learning algorithm outputs a welding parameter adjustment action sequence based on the fused state feature tensor and a preset welding quality reward function. This welding parameter adjustment action sequence includes target welding current adjustment, target welding speed adjustment, target wire feed speed adjustment, target welding torch height adjustment, and target shielding gas flow rate adjustment. The method then corrects the current welding parameters online according to the welding parameter adjustment action sequence, generating an optimized welding process parameter set. Finally, the optimized welding process parameter set is fed back to the welding actuator to trigger adaptive adjustment of the welding parameters.

[0019] In one embodiment of the present invention, the specific process of performing multimodal feature alignment and fusion processing on the real-time multi-source sensing data set to generate a fused state feature tensor is as follows, see [reference]. Figure 2 The welding current timing signal, welding speed timing signal, wire feed speed timing signal, welding torch height timing signal, and shielding gas flow timing signal are sampled using a sliding window method to obtain multiple current time window segments. The multiple wire feed speed time window segments, the multiple welding torch height time window segments, and the multiple shielding gas flow rate time window segments are aligned and spliced ​​according to timestamps to generate a multi-channel time series data matrix. Normalization scaling is performed on each channel in the multi-channel time series data matrix to map the data value range of each channel to a unified interval, generating a normalized multi-channel time series data matrix. Convolution dimensionality reduction is performed on the normalized multi-channel time series data matrix to extract high-dimensional feature vectors at each time step. The high-dimensional feature vectors at each time step are stacked along the time dimension to generate the fused state feature tensor.

[0020] In practical implementation, taking the robotic arc welding station in an automotive body welding production line as an example, multimodal feature alignment and fusion processing is performed on the real-time multi-source sensor data set. During the welding process, the welding control system continuously acquires welding current timing signals, welding speed timing signals, wire feed speed timing signals, welding torch height timing signals, and shielding gas flow timing signals. For the welding current timing signal, the control system uses a sliding window with a length of 100 sampling points and a step size of 50 sampling points to perform sliding window sampling processing, obtaining multiple current time window segments. For the welding speed timing signal, the control system uses a sliding window with a length of 100 sampling points and a step size of 50 sampling points to perform sliding window sampling processing, obtaining multiple welding speed time window segments. For the wire feed speed timing signal, the control system uses a sliding window with a length of 100 sampling points and a step size of 50 sampling points to perform sliding window sampling processing, obtaining multiple wire feed speed time window segments. For the welding torch height timing signal, the control system uses a sliding window with a length of 100 sampling points and a step size of 50 sampling points to perform sliding window sampling processing on the welding torch height timing signal, obtaining multiple welding torch height time window segments. For the shielding gas flow timing signal, the control system uses a sliding window with a length of 100 sampling points and a step size of 50 sampling points to perform sliding window sampling processing on the shielding gas flow timing signal, obtaining multiple shielding gas flow time window segments.

[0021] In some embodiments, the control system aligns and splices the plurality of current time window segments, the plurality of welding speed time window segments, the plurality of wire feed speed time window segments, the plurality of welding torch height time window segments, and the plurality of shielding gas flow time window segments according to the timestamp of each window segment. Specifically, for a current time window segment, a welding speed time window segment, a wire feed speed time window segment, a welding torch height time window segment, and a shielding gas flow time window segment corresponding to the same timestamp, the control system splices them along the channel dimension to form a multi-channel timing data matrix with 5 channels. The length of each channel is equal to the sliding window length of 100, and the dimension of the multi-channel timing data matrix is ​​100×5.

[0022] See Figure 4 , Figure 4The waveforms of welding current and welding speed timing signals from a real-time multi-source sensor dataset during the welding process are displayed. The welding current timing signal is represented by a solid line, with values ​​ranging from 135 A to 165 A. The welding current timing signal exhibits periodic fluctuations accompanied by high-frequency noise on the time axis. The welding speed timing signal is represented by a dashed line, with values ​​ranging from 23 cm / min to 33 cm / min. The observation length on the time axis is set to 10 seconds, and the welding current and welding speed timing signals are strictly synchronized in the time dimension.

[0023] In some embodiments, the control system performs normalization scaling on each channel of the multi-channel timing data matrix. For the welding current channel, the control system collects the minimum and maximum current values ​​of the welding current timing signal during historical welding processes, and normalizes the value of each sampling point in the welding current channel according to the following formula: in: This represents the original sampled value in the channel. This indicates the minimum value of the channel during the historical welding process. This indicates the maximum value of the channel during the historical welding process. This represents the normalized value. For the welding speed channel, wire feed speed channel, welding torch height channel, and shielding gas flow channel, the control system collects the minimum and maximum values ​​of each channel during the historical welding process, and maps the data value range of each channel to a unified interval [0,1] according to the same normalization formula, generating a normalized multi-channel time-series data matrix.

[0024] Optionally, the control system performs convolutional dimensionality reduction processing on the normalized multi-channel time-series data matrix. The control system configures a one-dimensional convolutional neural network layer containing 32 convolutional kernels, each with a size of 3 and a stride of 1, using a linear rectified function as the activation function. The control system inputs the normalized multi-channel time-series data matrix into the one-dimensional convolutional neural network layer. The layer performs convolution operations along the time dimension on the multi-channel data, outputting a 32-dimensional high-dimensional feature vector at each time step. After the convolution operation, the control system obtains a high-dimensional feature vector sequence with a length of 98 time steps (due to the kernel size of 3 and no boundary padding, the time dimension is reduced from 100 to 98).

[0025] Optionally, the control system stacks the high-dimensional feature vectors at each time step along the time dimension to form a fused state feature tensor with a dimension of 98×32. This fused state feature tensor simultaneously preserves the temporal dynamic information of multi-source sensor data and the cross-correlation information between multimodal features. It can be understood that aligning and stitching multiple time window segments obtained through sliding window sampling according to timestamps ensures that data from different sensors at the same time are correctly correlated. It can also be understood that normalization scaling unifies sensor data with different physical dimensions into the same numerical range, eliminating the impact of dimensional differences on subsequent feature extraction.

[0026] See Figure 5 , Figure 5 This paper presents the feature fusion matrix in the fused state feature tensor generated by multimodal feature alignment and fusion processing. The feature fusion matrix adopts a 10x10 two-dimensional matrix structure. The horizontal axis of the feature fusion matrix represents the temporal feature dimension, and the vertical axis represents the sensor channel dimension. Each pixel in the feature fusion matrix represents a feature fusion weight value, and the magnitude of the feature fusion weight value is mapped by the grayscale level. The feature fusion weight values ​​range from 0.0 to 1.0. The feature fusion matrix describes the distribution of features from different sensor channels across different time dimensions.

[0027] In one embodiment of the present invention, the specific process of inputting the fused state feature tensor into the workflow of the improved SAC reinforcement learning algorithm is as follows: The Actor policy network in the improved SAC reinforcement learning algorithm is invoked to perform action probability distribution mapping on the fused state feature tensor, generating an action probability distribution parameter set, which includes a mean parameter vector and a logarithmic standard deviation parameter vector; Reparameterization sampling is performed based on the action probability distribution parameter set to generate an original welding parameter adjustment action sequence; the first Critic evaluation network in the improved SAC reinforcement learning algorithm is invoked to perform joint value evaluation on the fused state feature tensor and the original welding parameter adjustment action sequence, generating a first action value score; the second Critic evaluation network in the improved SAC reinforcement learning algorithm is invoked to perform joint value evaluation on the fused state feature tensor and the original welding parameter adjustment action sequence, generating a second action value score; a minimum value selection operation is performed on the first action value score and the second action value score to generate a conservative action value score; and action correction processing is performed on the original welding parameter adjustment action sequence based on the conservative action value score to generate the welding parameter adjustment action sequence.

[0028] In practical implementation, an automotive chassis structural component welding robot is used as the application scenario, executing the workflow of the improved SAC reinforcement learning algorithm. The welding control system inputs the fused state feature tensor into the improved SAC reinforcement learning algorithm. Internally, the improved SAC reinforcement learning algorithm maintains an Actor policy network, a first Critic evaluation network, and a second Critic evaluation network. The Actor policy network has a three-layer fully connected feedforward neural network structure. The input layer receives the flattened vector of the fused state feature tensor, the hidden layer contains 256 neurons, and the output layer outputs a set of action probability distribution parameters. The control system calls the Actor policy network in the improved SAC reinforcement learning algorithm to perform forward propagation calculations on the fused state feature tensor. The Actor policy network outputs a set of action probability distribution parameters, which includes a mean parameter vector and a log-standard deviation parameter vector. The dimension of the mean parameter vector is equal to the dimension of the action space (5), and the dimension of the log-standard deviation parameter vector is also equal to 5.

[0029] See Figure 6 , Figure 6 This graph illustrates the evolution of the policy network loss and value network loss of the improved SAC reinforcement learning algorithm during training, showing the trends of these losses with the number of training steps. The policy network loss is represented by a solid black line, starting at 10.0 and exhibiting an exponential decreasing trend with increasing training steps. The value network loss is represented by a dashed gray line, starting at 20.0 and showing greater fluctuations than the policy network loss during training. The horizontal axis represents the number of training steps, and the vertical axis represents the loss value. Both the policy network loss and the value network loss reach a stable fluctuation state after 500 training steps.

[0030] In practical implementation, the control system performs reparameterization sampling based on the set of action probability distribution parameters. Let the mean parameter vector be... The logarithmic standard deviation parameter vector is The control system first generates a standard normally distributed noise vector with the same dimension as the action space. ,in Each component follows a normal distribution with a mean of 0 and a variance of 1. The control system is configured according to the reparameterization formula. Generate the original welding parameter adjustment action sequence ,in This represents element-wise multiplication. This means that the standard deviation vector is obtained by performing an exponential operation on each component of the log-standard deviation parameter vector. It is a 5-dimensional vector, with five components corresponding to the original, uncorrected values ​​of the target welding current adjustment, target welding speed adjustment, target wire feed speed adjustment, target welding torch height adjustment, and target shielding gas flow rate.

[0031] In some embodiments, the control system invokes the first Critic evaluation network in the improved SAC reinforcement learning algorithm to perform joint value evaluation processing on the fused state feature tensor and the original welding parameter adjustment action sequence. The first Critic evaluation network has a three-layer fully connected feedforward neural network structure. The input layer concatenates the flattened vector of the fused state feature tensor with the 5-dimensional vector of the original welding parameter adjustment action sequence to obtain the concatenated input vector. The hidden layer contains 256 neurons, and the output layer outputs a scalar value as the first action value score. Simultaneously, the control system invokes the second Critic evaluation network in the improved SAC reinforcement learning algorithm to perform joint value evaluation processing on the fused state feature tensor and the original welding parameter adjustment action sequence. The second Critic evaluation network has the same network structure as the first Critic evaluation network, but the initialization values ​​and optimization update processes of the network parameters are independent of each other. The second Critic evaluation network outputs a scalar value as the second action value score.

[0032] In some embodiments, the control system performs a minimum value selection operation on the first action value score and the second action value score. The control system compares the numerical values ​​of the first action value score and the second action value score, and selects the smaller value as the conservative action value score. The formula for calculating the conservative action value score is: in: This indicates the value score of the first action. This indicates the value score of the second action. This indicates the value score of conservative actions.

[0033] Optionally, the control system performs action correction processing on the original welding parameter adjustment action sequence based on the conservative action value score. The control system uses the conservative action value score as a component of the loss function and adjusts the original welding parameter adjustment action sequence through gradient ascent. Specifically, the control system treats the original welding parameter adjustment action sequence as an optimizable variable, calculates the gradient of the conservative action value score relative to the original welding parameter adjustment action sequence, and then updates each component of the original welding parameter adjustment action sequence according to the gradient direction, with an update step size set to 0.01. After multiple iterations of optimization, the control system obtains a corrected welding parameter adjustment action sequence, which includes corrected target welding current adjustment, corrected target welding speed adjustment, corrected target wire feed speed adjustment, corrected target welding torch height adjustment, and corrected target shielding gas flow rate adjustment. The corrected welding parameter adjustment action sequence has a higher conservative action value score than the original welding parameter adjustment action sequence.

[0034] Optionally, the control system limits each component in the modified welding parameter adjustment sequence to a predefined action boundary range. Specifically, the boundary range for the target welding current adjustment is -50 amperes to +50 amperes; the boundary range for the target welding speed adjustment is -5 mm / s to +5 mm / s; the boundary range for the target wire feed speed adjustment is -2 m / min to +2 m / min; the boundary range for the target welding torch height adjustment is -3 mm to +3 mm; and the boundary range for the target shielding gas flow rate adjustment is -2 liters / min to +2 liters / min.

[0035] It is understandable that using the minimum value selection operation to process the first and second action value scores can suppress the excessively high estimation errors that may arise from a single Critic evaluation network. It is also understandable that applying the conservative action value score to the original welding parameter adjustment action sequence allows the final output welding parameter adjustment action sequence to obtain a higher evaluation score under the conservative value assessment, thereby improving the reliability of the welding parameter adjustment action sequence.

[0036] In one embodiment of the present invention, the construction process of the preset welding quality reward function in the improved SAC reinforcement learning algorithm is as follows: The arc sound signal, electrical signal, and arc light signal are acquired in real time during the welding process. The electrical signal includes an arc voltage signal and a welding current signal. Short-time Fourier transform processing is performed on the arc sound signal to generate a sound spectrum feature matrix. Spectral decomposition processing is performed on the arc light signal to generate a spectral intensity distribution vector. Time-domain feature extraction processing is performed on the electrical signal to generate a time-domain statistical feature set, which includes mean value features, variance features, and peak value features. The sound spectrum feature matrix, the spectral intensity distribution vector, and the time-domain statistical feature set are input into a pre-trained welding stability scoring model to generate a welding stability score. The welding quality reward function is constructed based on the welding stability score and welding process constraints. The welding quality reward function includes a welding stability reward component, a welding speed reward component, and a welding torch height penalty component. The value of the welding stability reward component is directly proportional to the welding stability score. The value of the welding speed reward component is directly proportional to the current value of the welding speed time-series signal. The value of the welding torch height penalty component is inversely proportional to the absolute value of the deviation of the current value of the welding torch height time-series signal from the preset reference height. The welding torch height penalty component takes a negative value when the current value of the welding torch height time-series signal deviates from the preset reference height by more than a preset threshold. The welding stability reward component, the welding speed reward component, and the welding torch height penalty component are linearly combined according to preset weighting coefficients to generate the final output value of the welding quality reward function.

[0037] In practical implementation, taking the welding of thick plates for large structural components in the shipbuilding industry as an example, a pre-defined welding quality reward function is constructed in the improved SAC reinforcement learning algorithm. The welding control system collects arc sound signals, electrical signals, and arc light signals in real time during the welding process. The electrical signals include arc voltage signals and welding current signals. The control system installs an electret microphone at the welding station, with a sampling frequency of 44.1 kHz, to collect arc sound signals. A voltage Hall sensor is connected in parallel and a current Hall sensor is connected in series at the welding power supply output to collect arc voltage signals and welding current signals, respectively, with sampling frequencies both set to 10 kHz. A fiber optic spectrometer is installed at the end of the welding torch, with a spectral acquisition range of 200 nm to 1000 nm, to collect arc light signals.

[0038] In specific implementation, the control system performs short-time Fourier transform processing on the arc sound signal. The control system divides the acquired arc sound signal into frames of 2048 sampling points per frame, with a frame shift of 512 sampling points. After applying a Hanning window to each frame of data, a fast Fourier transform is performed to obtain the spectral amplitude value corresponding to each time frame. The spectral amplitude values ​​of all time frames are arranged in chronological order to generate a sound spectral feature matrix. The dimension of the sound spectral feature matrix is ​​the number of frequency points multiplied by the number of time frames. The control system performs spectral decomposition processing on the arc optical signal. After wavelength calibration and baseline correction of the raw spectral data acquired by the fiber optic spectrometer, the control system extracts the light intensity value corresponding to each wavelength channel to generate a spectral intensity distribution vector. The length of the spectral intensity distribution vector is equal to the number of resolution channels of the spectrometer. The control system performs time-domain feature extraction processing on the electrical signal. The control system calculates the average value, variance, and peak value characteristics of the arc voltage signal and the welding current signal, respectively. The average value characteristic is equal to the arithmetic mean of all sampling points within a time window. The variance characteristic is equal to the average of the sum of squares of the differences between the sampling point values ​​and the average value. The peak value characteristic is equal to the difference between the maximum and minimum values ​​of the sampling points within the time window. The six characteristic values ​​of the arc voltage and welding current are combined into a set of time-domain statistical characteristics of the electrical signal.

[0039] In some embodiments, the control system inputs the sound spectrum feature matrix, the spectral intensity distribution vector, and the electrical signal time-domain statistical feature set into a pre-trained welding stability scoring model. The pre-trained welding stability scoring model outputs a welding stability score value between 0 and 100, with a higher score indicating a more stable welding process. The input processing method for the welding stability scoring model is as follows: the sound spectrum feature matrix is ​​scaled to a 224×224 grayscale image size; the spectral intensity distribution vector is normalized and adjusted to a one-dimensional vector of length 256; and the six feature values ​​of the electrical signal time-domain statistical feature set are Z-score standardized.

[0040] In some embodiments, the control system constructs the welding quality reward function based on the welding stability score and welding process constraints. The welding process constraints include a welding speed not exceeding the maximum moving speed of the welding robot's end effector (15 mm / s) and the welding torch height being maintained within ±5 mm of a reference height to prevent collisions or arc interruptions. The welding quality reward function includes a welding stability reward component, a welding speed reward component, and a welding torch height penalty component. The value of the welding stability reward component is directly proportional to the welding stability score, specifically set to equal the welding stability score divided by 10. The value of the welding speed reward component is directly proportional to the current value of the welding speed timing signal, specifically set to equal the current value of the welding speed timing signal divided by 15 mm / s. The value of the welding torch height penalty component is inversely proportional to the absolute value of the deviation of the current value of the welding torch height timing signal from a preset reference height. The welding torch height penalty component takes a negative value when the current value of the welding torch height timing signal deviates from the preset reference height by more than 5 mm, specifically set as follows: Let the reference height be... The current welding torch height is absolute value of deviation Then the welding torch height penalty component is equal to ;when When the value is in millimeters, the welding torch height penalty component is set to 0. Optionally, the welding stability reward component, welding speed reward component, and welding torch height penalty component are linearly combined according to a preset weighting coefficient to generate the final output value of the welding quality reward function. The expression of the welding quality reward function is: in: This represents the final output value of the welding quality reward function. This indicates the bonus for welding stability. This indicates a bonus component for welding speed. Indicates the welding torch height penalty component. The weighting coefficient representing the welding stability bonus component. The weighting coefficient representing the welding speed bonus component. This represents the weighting coefficient for the welding torch height penalty component.

[0041] Optionally, the weighting coefficient can be configured according to welding process requirements. For applications where welding quality is paramount, increasing the weighting coefficient is preferable. The value of ; for operating conditions where production efficiency is prioritized, increase . The value of is given in Table 1, which shows a typical weighting coefficient configuration.

[0042] Table 1: Weighting coefficients of each component of the welding quality reward function It is understandable that the introduction of a welding speed reward component encourages the welding actuator to improve welding efficiency while ensuring welding quality. It is also understandable that the design of the welding torch height penalty component provides a negative reward when the torch height deviates from a preset reference height, thereby guiding the reinforcement learning algorithm's output welding parameter adjustment sequence to maintain the torch height within a reasonable range. Furthermore, it is understandable that by linearly combining the three components and configuring different weighting coefficients, the welding quality reward function can flexibly adapt to the different priorities of quality, efficiency, and stability required for various welding tasks.

[0043] In one embodiment of the present invention, the working principle of the improved SAC reinforcement learning algorithm includes a policy iteration process based on the maximum entropy framework. Specifically, it involves collecting historical state transition trajectory data generated by the welding actuator after performing historical welding parameter adjustment actions within a historical time period. The historical state transition trajectory data includes historical state tensors, historical action tensors, historical reward values, and the state tensor at the next moment. The historical state transition trajectory data is stored in an experience replay buffer. A batch of historical state transition trajectory data is randomly sampled from the experience replay buffer to form a training batch. The gradient information of the policy network is calculated based on the training batch data. The temporal difference error of the first Critic evaluation network and the second Critic evaluation network is calculated based on the training batch data. The gradient descent update operation is performed on the network parameters of the policy network based on the gradient information of the policy network. The gradient descent update operation is also performed on the network parameters of the first Critic evaluation network and the second Critic evaluation network based on the temporal difference error of the first Critic evaluation network and the second Critic evaluation network. The improved SAC reinforcement learning algorithm also includes an automatic temperature parameter adjustment process. Specifically, it involves: initializing a target entropy parameter, setting the value of the target entropy parameter to a negative value of the action dimension; initializing the logarithm of the learnable temperature parameters; sampling batch state tensors from the experience replay buffer and inputting the batch state tensors into the policy network to generate batch action probability distribution parameters; sampling based on the batch action probability distribution parameters to generate batch sampled action tensors and calculating the actual entropy value of the batch sampled action tensors under the batch action probability distribution parameters; calculating the loss gradient of the temperature parameters based on the difference between the actual entropy value and the target entropy parameter; performing a gradient descent update operation on the logarithm of the learnable temperature parameters based on the loss gradient of the temperature parameters; and performing an exponential transformation on the updated logarithm of the learnable temperature parameters to generate the temperature parameter value at the current time.The construction and training process of the pre-trained welding stability scoring model is as follows: Multiple sets of labeled training samples are collected under various welding conditions during the welding process. Each set of labeled training samples includes arc sound spectral feature matrix samples, spectral intensity distribution vector samples, electrical signal time-domain statistical feature samples, and manually labeled welding stability scoring labels. The arc sound spectral feature matrix samples are input into the first convolutional neural network branch to generate sound embedding feature vectors. The spectral intensity distribution vector samples are input into the second convolutional neural network branch to generate spectral embedding feature vectors. The electrical signal time-domain statistical feature samples are input into the fully connected network branch to generate electrical signal embedding feature vectors. The sound embedding feature vector, the spectral embedding feature vector, and the electrical signal embedding feature vector are concatenated to generate a fused embedding feature vector. This fused embedding feature vector is then input into the output layer network to generate a predicted welding stability score. A loss function is constructed based on the error between the predicted welding stability score and the manually labeled welding stability score. The network parameters of the first convolutional neural network branch, the second convolutional neural network branch, the fully connected network branch, and the output layer network are jointly optimized using a backpropagation algorithm until the loss function converges, resulting in the pre-trained welding stability scoring model.

[0044] In practical implementation, taking the circumferential weld scenario in the manufacturing of aerospace engine thrust chambers as an example, the improved SAC reinforcement learning algorithm, based on the maximum entropy framework, is used for policy iteration. During operation, the welding control system collects historical state transition trajectory data generated by the welding actuator after performing historical welding parameter adjustment actions within a historical time period. Each historical state transition trajectory data includes a historical state tensor, a historical action tensor, a historical reward value, and a next-moment state tensor. The control system stores the generated historical state transition trajectory data in an experience replay buffer, with a capacity of 100,000 trajectories. When the number of trajectories in the experience replay buffer exceeds 10,000, the control system randomly samples a batch of historical state transition trajectory data with a size of 256 from the experience replay buffer to form the training batch data.

[0045] In practical implementation, the control system calculates the gradient information of the policy network based on the training batch data. The policy network, also known as the Actor network, aims to output action probability distribution parameters given a state tensor. The loss function of the policy network includes a maximum entropy term. The control system calculates the temporal difference error between the first and second Critic evaluation networks based on the training batch data. The temporal difference error is calculated as follows: the state tensor of the next time step from the training batch data is input into the target policy network to obtain the action probability distribution of the next time step. This distribution is then input into the first and second target Critic evaluation networks to obtain two target value scores. The minimum of the two target value scores is taken, combined with the historical reward values ​​and discount factors from the training batch data, to calculate the temporal difference target value. Finally, the value score output by the current Critic evaluation network is subtracted to obtain the temporal difference error.

[0046] In some embodiments, the control system performs gradient descent update operations on the network parameters of the policy network based on the gradient information of the policy network. The optimizer for the policy network is the Adam optimizer, with a learning rate set to 0.0003. During each update, the network parameters of the policy network are adjusted along the opposite direction of the gradient. The control system performs gradient descent update operations on the network parameters of the first and second Critic evaluation networks respectively based on the temporal difference error between the first and second Critic evaluation networks. The optimizers for both Critic evaluation networks also use the Adam optimizer, with a learning rate set to 0.0003. Every two training steps, the control system performs soft updates on the network parameters of the target policy network, the first target Critic evaluation network, and the second target Critic evaluation network, with a soft update coefficient set to 0.005.

[0047] In its implementation, the improved SAC reinforcement learning algorithm also includes an automatic adjustment process for the temperature parameter. The control system initializes the target entropy parameter, setting its value to a negative value of the action dimension. Since the action dimension is 5, the target entropy parameter is set to -5. The control system initializes a learnable logarithm of the temperature parameter, with an initial value of 0. The control system samples a batch state tensor of size 256 from the experience replay buffer and inputs it into the policy network. The policy network outputs batch action probability distribution parameters, which include a batch mean parameter vector and a batch logarithm standard deviation parameter vector. The batch size is 256, and each action probability distribution parameter corresponds to a 5-dimensional action distribution.

[0048] In some embodiments, the control system samples actions based on batch action probability distribution parameters to generate batch sampled action tensors, where the dimension of the batch sampled action tensors is 256 multiplied by 5. The control system calculates the actual entropy value of the batch sampled action tensors under the batch action probability distribution parameters. The formula for calculating the actual entropy value for each sampled action tensor is: in: Represents the state tensor Next strategy The actual entropy value of the output action distribution. Represents the action dimension and , Indicates the state Next strategy Output the first The probability density value of each action component. The control system averages the actual entropy values ​​corresponding to the 256 state tensors in the batch to obtain the batch average actual entropy value.

[0049] Optionally, the control system calculates the loss gradient of the temperature parameter based on the difference between the actual entropy value and the target entropy parameter. The loss function of the temperature parameter is defined as the negative logarithm of the learnable temperature parameter multiplied by the difference between the batch average actual entropy value and the target entropy parameter. The control system differentiates this loss function to obtain the loss gradient of the temperature parameter. Based on the loss gradient of the temperature parameter, a gradient descent update operation is performed on the logarithm of the learnable temperature parameter, with an update step size set to 0.001. An exponential transformation is performed on the updated logarithm of the learnable temperature parameter to generate the temperature parameter value at the current moment, which is equal to the natural constant raised to the power of the logarithm of the learnable temperature parameter.

[0050] In practice, the construction and training process of the pre-trained welding stability scoring model is as follows: The control system collects multiple sets of labeled training samples under various welding conditions during the welding process. Welding conditions include different plate thickness combinations, different joint types, and different welding positions. Each set of labeled training samples contains an arc sound spectrum feature matrix sample, a spectral intensity distribution vector sample, an electrical signal time-domain statistical feature sample, and a welding stability scoring label manually labeled by the welding engineer. The value of the welding stability scoring label ranges from 0 to 100. A total of 5000 sets of labeled training samples are collected and divided into training, validation, and test sets in an 8:1:1 ratio.

[0051] In some embodiments, the control system constructs a multi-branch fusion neural network for welding stability score prediction. The arc sound spectral feature matrix samples are input into a first convolutional neural network branch, which contains two convolutional layers and a global average pooling layer, outputting a 256-dimensional sound embedding feature vector. The spectral intensity distribution vector samples are input into a second convolutional neural network branch, which contains three one-dimensional convolutional layers and a flattening layer, outputting a 256-dimensional spectral embedding feature vector. The electrical signal temporal statistical feature samples are input into a fully connected network branch, which contains two fully connected layers: the first has 128 neurons, and the second has 64 neurons, outputting a 64-dimensional electrical signal embedding feature vector. Optionally, the control system concatenates the sound embedding feature vector, the spectral embedding feature vector, and the electrical signal embedding feature vector to obtain a fused embedding feature vector with dimensions 256 + 256 + 64 = 576. The fused embedded feature vector is input into the output layer network, which contains two fully connected layers. The first fully connected layer has 128 neurons and uses a linear rectified function as the activation function, while the second fully connected layer has 1 neuron and does not use an activation function. The output is a scalar value as the predicted welding stability score.

[0052] Optionally, the control system constructs a loss function based on the error between the predicted welding stability score and the manually labeled welding stability score, using the mean squared error loss function. The network parameters of the first convolutional neural network branch, the second convolutional neural network branch, the fully connected network branch, and the output layer network are jointly optimized using a backpropagation algorithm. The optimizer uses the Adam optimizer, with an initial learning rate of 0.001, a batch size of 32, and 200 training epochs. After each training epoch, the mean squared error is calculated on the validation set. Training stops when the mean squared error on the validation set no longer decreases for 10 consecutive epochs, and the parameters of each branch network are saved to obtain the pre-trained welding stability scoring model (see Table 2).

[0053] Table 2: Structural parameters of each branch network in the welding stability scoring model It is understandable that in the policy iteration process based on the maximum entropy framework, the goal of the policy network is to simultaneously maximize the expected cumulative reward and the policy entropy, thereby encouraging the policy network to explore more high-reward actions during training. It is also understandable that the automatic adjustment process of the temperature parameter allows the entropy value of the actual sampled action to gradually approach the preset target entropy parameter, avoiding the tedious work of manually adjusting the temperature parameter. Furthermore, it is understandable that the welding stability scoring model adopts a multi-branch fusion neural network structure, processing input data from different modalities separately before fusing features, effectively utilizing the complementary information contained in the arc sound, arc spectrum, and electrical signal.

[0054] In one embodiment of the present invention, the specific process of online correction of the current welding parameters according to the welding parameter adjustment action sequence to generate an optimized welding process parameter set is as follows, see below. Figure 3The process involves: acquiring the current welding parameter set, which includes the current welding current value, current welding speed value, current wire feed speed value, current welding torch height value, and current shielding gas flow rate value; adding the target welding current adjustment amount in the welding parameter adjustment sequence to the current welding current value to generate an optimized welding current value; adding the target welding speed adjustment amount in the welding parameter adjustment sequence to the current welding speed value to generate an optimized welding speed value; and adding the target wire feed speed adjustment amount in the welding parameter adjustment sequence to the current wire feed speed value. The numerical values ​​are added together to generate an optimized wire feed speed value. The target welding torch height adjustment amount in the welding parameter adjustment sequence is added to the current welding torch height value to generate an optimized welding torch height value. The target shielding gas flow rate adjustment amount in the welding parameter adjustment sequence is added to the current shielding gas flow rate value to generate an optimized shielding gas flow rate value. The optimized welding current value, the optimized welding speed value, the optimized wire feed speed value, the optimized welding torch height value, and the optimized shielding gas flow rate value are packaged into an optimized welding process parameter set. The specific process of feeding back the optimized welding process parameter set to the welding actuator to trigger the adaptive adjustment operation of the welding parameters is as follows: The optimized welding current value, optimized welding speed value, optimized wire feed speed value, optimized welding torch height value, and optimized shielding gas flow rate value are extracted from the optimized welding process parameter set; the optimized welding current value is converted into an analog control signal and output to the current adjustment interface of the welding power supply; the optimized welding speed value is converted into a pulse frequency control signal and output to the drive motor controller of the welding travel mechanism; the optimized wire feed speed value is converted into a speed control command and output to the servo driver of the wire feed mechanism; the optimized welding torch height value is converted into a displacement control command and output to the stepper motor controller of the welding torch height adjustment mechanism; the optimized shielding gas flow rate value is converted into a valve opening control signal and output to the actuator of the shielding gas flow regulating valve; after the welding actuator completes the adaptive adjustment operation, it returns to the step of acquiring the real-time multi-source sensor data set during the welding process, forming a closed-loop iterative optimization cycle.

[0055] See Figure 7 , Figure 7This diagram illustrates the optimized trajectory of the welding parameter adjustment sequence in parameter space, along with the corresponding reward function contour plot. The horizontal axis of the contour plot represents the welding current adjustment, and the vertical axis represents the welding speed adjustment. The optimized trajectory consists of a series of dotted line segments. The trajectory begins at an edge point in the parameter space and ends in the low-potential region at the center of the contour plot. The values ​​for both the welding current and welding speed adjustments are limited to between -2.0 and +2.0. The optimized trajectory fully records the coordinate path of the welding process parameter set evolving from its initial state to the target optimized state.

[0056] In practical implementation, the application scenario is an all-position automatic welding machine used in oil pipeline laying. The welding parameters are corrected online based on a sequence of adjustment actions. The welding control system reads the current welding parameter set from the welding machine controller. This set includes the current welding current, welding speed, wire feed speed, welding torch height, and shielding gas flow rate. The control system adds the target welding current adjustment amount in the welding parameter adjustment sequence to the current welding current value to generate an optimized welding current value; it adds the target welding speed adjustment amount to the current welding speed value to generate an optimized welding speed value; it adds the target wire feed speed adjustment amount to the current wire feed speed value to generate an optimized wire feed speed value; it adds the target welding torch height adjustment amount to the current welding torch height value to generate an optimized welding torch height value; and it adds the target shielding gas flow rate adjustment amount to the current shielding gas flow rate value to generate an optimized shielding gas flow rate value. The above addition operation follows the formula: in: This represents the optimized process parameter values. This represents the corresponding value in the current set of welding parameters. This indicates the corresponding adjustment amount in the sequence of welding parameter adjustment actions.

[0057] In practice, the control system packages the optimized welding current, welding speed, wire feed speed, welding torch height, and shielding gas flow rate into an optimized welding process parameter set according to a predefined data structure. This optimized welding process parameter set is stored in the control system's memory in the form of a structure or dictionary.

[0058] In some embodiments, the control system feeds back the optimized welding process parameter set to the welding actuator to trigger adaptive adjustment of the welding parameters. The control system first extracts the optimized welding current, welding speed, wire feed speed, welding torch height, and shielding gas flow rate from the optimized welding process parameter set. The control system converts the optimized welding current into an analog control signal of 4 mA to 20 mA via an analog-to-digital converter. The output value of the analog control signal is linearly related to the optimized welding current. This analog control signal is output to the current adjustment interface of the welding power supply, and the power converter inside the welding power supply adjusts the output current according to the received analog control signal. The control system converts the optimized welding speed into a pulse frequency control signal. The frequency value of the pulse frequency control signal is proportional to the optimized welding speed. This pulse frequency control signal is output to the drive motor controller of the welding travel mechanism, and the drive motor controller adjusts the speed of the drive motor according to the pulse frequency control signal.

[0059] In some embodiments, the control system converts the optimized wire feed speed value into a speed control command. The speed control command is transmitted in data frames using the RS485 communication protocol. Each data frame contains an address code, a function code, a wire feed speed value byte, and a checksum. This speed control command is output to the servo driver of the wire feed mechanism. The servo driver adjusts the speed of the servo motor according to the received speed control command to change the wire feed speed. The control system also converts the optimized welding torch height value into a displacement control command. This displacement control command includes two fields: the direction of movement and the number of steps. This displacement control command is output to the stepper motor controller of the welding torch height adjustment mechanism. The stepper motor controller generates a corresponding pulse sequence based on the displacement control command to drive the stepper motor to rotate. The stepper motor, through a lead screw and nut pair, raises or lowers the welding torch to the target height. The control system converts the optimized protective gas flow rate into a valve opening control signal of 4 mA to 20 mA. The current value of the valve opening control signal is proportional to the optimized protective gas flow rate. This valve opening control signal is output to the actuator of the protective gas flow regulating valve. The electromagnetic coil inside the actuator adjusts the valve opening according to the current to control the protective gas flow rate.

[0060] Optionally, after the welding actuator completes the adaptive adjustment operation, the control system returns to the step of acquiring the real-time multi-source sensor data set during the welding process, forming a closed-loop iterative optimization loop. Specifically, after issuing all control signals, the control system waits for a fixed sampling period, the length of which is equal to the time length of the sliding window, i.e., the time corresponding to 100 sampling points. After the wait is over, the control system re-acquires new real-time multi-source sensor data sets from each sensor, uses the new real-time multi-source sensor data sets as input for the next iteration, and re-executes the entire process of multimodal feature alignment and fusion processing, improved SAC reinforcement learning algorithm inference, and online correction of welding parameters.

[0061] Optionally, the control system includes a safety monitoring module within the closed-loop iterative optimization cycle. This module monitors in real-time whether the optimized welding current, welding speed, wire feed speed, welding torch height, and shielding gas flow rate exceed preset safety thresholds. If any optimized value exceeds the safety threshold, the safety monitoring module immediately interrupts the closed-loop iterative optimization cycle, forcibly sets the corresponding process parameter to a safety boundary value, and issues an alarm signal to notify the operator.

[0062] It is understandable that by adding the adjustment values ​​in the welding parameter adjustment sequence to the current welding parameter value through addition, online incremental correction of the welding parameters is achieved, avoiding the large fluctuations that may result from directly setting absolute values. It is also understandable that converting each value in the optimized welding process parameter set into a control signal format recognizable by the corresponding actuator ensures the correct transmission and conversion of control commands from the control system to the physical actuator. Furthermore, the step of returning to obtain the real-time multi-source sensor data set after performing the adaptive adjustment operation forms a closed-loop iterative optimization cycle, enabling the welding parameters to continuously and adaptively optimize as the welding process progresses.

[0063] The above embodiments are only used to illustrate the technical methods of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical methods of the present invention without departing from the spirit and scope of the technical methods of the present invention.

Claims

1. A welding quality self-optimization method based on multi-source sensing and deep reinforcement learning, characterized in that, The method includes: Acquire a real-time multi-source sensor data set during the welding process, the real-time multi-source sensor data set including welding current timing signal, welding speed timing signal, wire feed speed timing signal, welding torch height timing signal, and shielding gas flow rate timing signal; Perform multimodal feature alignment and fusion processing on the real-time multi-source sensor data set to generate a fused state feature tensor; The fusion state feature tensor is input into the workflow of the improved SAC reinforcement learning algorithm. The improved SAC reinforcement learning algorithm outputs a welding parameter adjustment action sequence based on the fusion state feature tensor and the preset welding quality reward function. The welding parameter adjustment action sequence includes the target welding current adjustment amount, the target welding speed adjustment amount, the target wire feed speed adjustment amount, the target welding torch height adjustment amount, and the target shielding gas flow rate adjustment amount. The current welding parameters are corrected online according to the welding parameter adjustment action sequence to generate an optimized welding process parameter set; the optimized welding process parameter set is fed back to the welding actuator to trigger the adaptive adjustment operation of the welding parameters.

2. The welding quality self-optimization method based on multi-source sensing and deep reinforcement learning according to claim 1, characterized in that, The steps of performing multimodal feature alignment and fusion processing on the real-time multi-source sensing data set to generate a fused state feature tensor specifically include: The welding current timing signal is subjected to sliding window sampling processing to obtain multiple current time window segments; the welding speed timing signal is subjected to sliding window sampling processing to obtain multiple welding speed time window segments. Perform sliding window sampling processing on the wire feeding speed timing signal to obtain multiple wire feeding speed time window segments; Perform sliding window sampling processing on the welding torch height timing signal to obtain multiple welding torch height time window segments; Perform sliding window sampling processing on the protective gas flow time sequence signal to obtain multiple protective gas flow time window segments; The multiple current time window segments, the multiple welding speed time window segments, the multiple wire feed speed time window segments, the multiple welding torch height time window segments, and the multiple shielding gas flow time window segments are aligned and spliced ​​according to timestamps to generate a multi-channel time series data matrix. Normalization scaling is performed on each channel in the multi-channel time series data matrix to map the data value range of each channel to a unified interval, thereby generating a normalized multi-channel time series data matrix. The normalized multi-channel time-series data matrix is ​​subjected to convolutional dimensionality reduction processing to extract high-dimensional feature vectors at each time step. The high-dimensional feature vectors at each time step are stacked along the time dimension to generate the fused state feature tensor.

3. The welding quality self-optimization method based on multi-source sensing and deep reinforcement learning according to claim 1, characterized in that, The steps in the workflow of inputting the fused state feature tensor into the improved SAC reinforcement learning algorithm specifically include: The Actor policy network in the improved SAC reinforcement learning algorithm is invoked to perform action probability distribution mapping on the fused state feature tensor to generate an action probability distribution parameter set, which includes a mean parameter vector and a log-standard deviation parameter vector. Based on the set of action probability distribution parameters, reparameterization sampling is performed to generate the original welding parameter adjustment action sequence; the first Critic evaluation network in the improved SAC reinforcement learning algorithm is called to perform joint value evaluation processing on the fused state feature tensor and the original welding parameter adjustment action sequence to generate the first action value score; The improved SAC reinforcement learning algorithm is used to call the second Critic evaluation network to perform joint value evaluation processing on the fused state feature tensor and the original welding parameter adjustment action sequence, and generate a second action value score. Perform a minimum value selection operation on the first action value score and the second action value score to generate a conservative action value score; The original welding parameter adjustment action sequence is modified based on the conservative action value score to generate the welding parameter adjustment action sequence.

4. The welding quality self-optimization method based on multi-source sensing and deep reinforcement learning according to claim 1, characterized in that, The construction process of the preset welding quality reward function in the improved SAC reinforcement learning algorithm specifically includes: The system acquires real-time arc sound signals, electrical signals, and arc light signals during the welding process, wherein the electrical signals include arc voltage signals and welding current signals. Perform a short-time Fourier transform on the electric arc sound signal to generate a sound spectrum feature matrix; The arc light signal is subjected to spectral decomposition processing to generate a spectral intensity distribution vector; The electrical signal is subjected to time-domain feature extraction processing to generate a time-domain statistical feature set of the electrical signal, which includes mean value features, variance features and peak value features. The sound spectrum feature matrix, the spectral intensity distribution vector, and the electrical signal time-domain statistical feature set are input into a pre-trained welding stability scoring model to generate a welding stability score. The welding quality reward function is constructed based on the welding stability score and welding process constraints.

5. The welding quality self-optimization method based on multi-source sensing and deep reinforcement learning according to claim 4, characterized in that, The welding quality reward function includes a welding stability reward component, a welding speed reward component, and a welding torch height penalty component. The value of the welding stability reward component is directly proportional to the welding stability score. The value of the welding speed reward component is directly proportional to the current value of the welding speed time-series signal. The value of the welding torch height penalty component is inversely proportional to the absolute value of the deviation of the current value of the welding torch height time-series signal from the preset reference height, and the welding torch height penalty component takes a negative value when the current value of the welding torch height time-series signal deviates from the preset reference height by more than a preset threshold. The welding stability reward component, the welding speed reward component, and the welding torch height penalty component are linearly combined according to preset weighting coefficients to generate the final output value of the welding quality reward function.

6. The welding quality self-optimization method based on multi-source sensing and deep reinforcement learning according to claim 1, characterized in that, The improved SAC reinforcement learning algorithm works by employing a policy iteration process based on a maximum entropy framework, specifically including: Collect historical state transition trajectory data generated by the welding actuator after performing historical welding parameter adjustment actions within a historical time period. The historical state transition trajectory data includes historical state tensor, historical action tensor, historical reward value and state tensor at the next moment. The historical state transition trajectory data is stored in the experience replay buffer pool; Randomly sample batches of historical state transition trajectory data from the experience replay buffer pool to form training batch data; The gradient information of the policy network is calculated based on the training batch data; the temporal difference error of the first Critic evaluation network and the second Critic evaluation network is calculated based on the training batch data. Based on the gradient information of the policy network, perform gradient descent update operation on the network parameters of the policy network; Gradient descent update operation is performed on the network parameters of the first Critic evaluation network and the second Critic evaluation network based on the temporal difference error of the first Critic evaluation network and the second Critic evaluation network.

7. The welding quality self-optimization method based on multi-source sensing and deep reinforcement learning according to claim 6, characterized in that, The improved SAC reinforcement learning algorithm also includes an automatic temperature parameter adjustment process, specifically including: Initialize the target entropy parameter, and set the value of the target entropy parameter to the negative value of the action dimension; Initialize the logarithm of the learnable temperature parameter; Sample batch state tensors from the experience replay buffer; The batch state tensor is input into the policy network to generate batch action probability distribution parameters; Based on the batch action probability distribution parameters, a batch sampling action tensor is generated; Calculate the actual entropy value of the batch sampling action tensor under the probability distribution parameters of the batch action; The loss gradient of the temperature parameter is calculated based on the difference between the actual entropy value and the target entropy parameter; A gradient descent update operation is performed on the logarithm of the learnable temperature parameters based on the loss gradient of the temperature parameters; Perform an exponential transformation on the updated learnable temperature parameters to generate the temperature parameter values ​​for the current moment.

8. The welding quality self-optimization method based on multi-source sensing and deep reinforcement learning according to claim 4, characterized in that, The construction and training process of the pre-trained welding stability scoring model specifically includes: Multiple sets of labeled training samples were collected during the welding process under various welding conditions. Each set of labeled training samples included arc sound spectrum feature matrix samples, spectral intensity distribution vector samples, electrical signal time-domain statistical feature samples, and manually labeled welding stability rating labels. The sample of the arc sound spectrum feature matrix is ​​input into the first branch of the convolutional neural network to generate a sound embedding feature vector; The spectral intensity distribution vector sample is input into the second convolutional neural network branch to generate a spectral embedding feature vector; The time-domain statistical feature samples of the electrical signal are input into a fully connected network branch to generate an electrical signal embedding feature vector; The sound embedding feature vector, the spectral embedding feature vector, and the electrical signal embedding feature vector are concatenated to generate a fused embedding feature vector. The fused embedded feature vector is input into the output layer network to generate a predicted welding stability score. A loss function is constructed based on the error between the predicted welding stability score and the manually labeled welding stability score. The network parameters of the first convolutional neural network branch, the second convolutional neural network branch, the fully connected network branch, and the output layer network are jointly optimized through the backpropagation algorithm until the loss function converges, thus obtaining the pre-trained welding stability scoring model.

9. The welding quality self-optimization method based on multi-source sensing and deep reinforcement learning according to claim 1, characterized in that, The steps of adjusting the current welding parameters online according to the welding parameter adjustment sequence to generate an optimized set of welding process parameters specifically include: Obtain the current welding parameter set, which includes the current welding current value, the current welding speed value, the current wire feed speed value, the current welding torch height value, and the current shielding gas flow rate value; The target welding current adjustment amount in the welding parameter adjustment action sequence is added to the current welding current value to generate an optimized welding current value. The target welding speed adjustment amount in the welding parameter adjustment action sequence is added to the current welding speed value to generate an optimized welding speed value. The target wire feed speed adjustment amount in the welding parameter adjustment action sequence is added to the current wire feed speed value to generate the optimized wire feed speed value. The target welding torch height adjustment amount in the welding parameter adjustment action sequence is added to the current welding torch height value to generate the optimized welding torch height value. The target shielding gas flow rate adjustment amount in the welding parameter adjustment action sequence is added to the current shielding gas flow rate value to generate the optimized shielding gas flow rate value. The optimized welding current value, the optimized welding speed value, the optimized wire feed speed value, the optimized welding torch height value, and the optimized shielding gas flow rate value are packaged into the optimized welding process parameter set.

10. The welding quality self-optimization method based on multi-source sensing and deep reinforcement learning according to claim 9, characterized in that, The step of feeding back the optimized welding process parameter set to the welding actuator to trigger the adaptive adjustment of welding parameters specifically includes: Extract the optimized welding current value, optimized welding speed value, optimized wire feed speed value, optimized welding torch height value, and optimized shielding gas flow rate value from the optimized welding process parameter set; The optimized welding current value is converted into an analog control signal and output to the current adjustment interface of the welding power supply. The optimized welding speed value is converted into a pulse frequency control signal and output to the drive motor controller of the welding walking mechanism; The optimized wire feeding speed value is converted into a speed control command and output to the servo driver of the wire feeding mechanism; The optimized welding torch height value is converted into a displacement control command and output to the stepper motor controller of the welding torch height adjustment mechanism; The optimized protective gas flow rate value is converted into a valve opening control signal and output to the actuator of the protective gas flow rate regulating valve; After the welding actuator completes the adaptive adjustment operation, it returns to the step of acquiring the real-time multi-source sensor data set during the welding process, forming a closed-loop iterative optimization loop.

Images