Double-arm welding robot collaborative welding seam tracking control method and system

Abstract

The invention discloses a double-arm welding robot collaborative welding seam tracking control method and system, and the method comprises the steps: taking a near-end strategy optimization algorithm as a basis, introducing a task modulation mechanism, and enabling a control model to achieve differential response according to different welding seam type characteristics; thirdly, designing a segmented training mechanism under precision driving, and gradually improving the tracking precision and keeping convergence stability in the training process through a dynamic precision constraint guide strategy; and finally, precision correlation adjustment is carried out in the strategy and value objective function optimization process, and the balance between the control performance and the learning efficiency is realized. According to the method, the dependence on model precision and parameter setting is remarkably reduced, the robustness of the control method is improved, stable transition from millimeter-level tracking precision to submillimeter-level tracking precision is realized, and the generalization of the strategy and the engineering practicability are enhanced while high precision is ensured.

Description

Technical Field

[0001] This invention belongs to the field of robot control technology, specifically relating to a collaborative weld seam tracking control method and system for a dual-arm welding robot. Background Technology

[0002] Welding is a critical process in shipbuilding, and its quality directly affects the strength and reliability of the hull structure. In the manufacturing of sub-assemblies in ships, double-sided welding is often required to ensure weld quality and structural consistency. Traditional manual welding methods struggle to achieve simultaneous double-sided operation, leading to uneven weld quality and low efficiency. Single-arm welding robot systems also cannot meet the requirements of simultaneous double-sided welding in terms of workspace and control. Therefore, dual-arm welding robot systems with collaborative operation capabilities are gradually becoming an important technological means in intelligent shipbuilding. However, collaborative weld seam tracking control of dual-arm welding robots faces numerous technical challenges: such as the strong coupling of kinematics and dynamics during collaborative welding, resulting in complex modeling and high control difficulty; the diverse weld seam shapes of sub-assemblies in ships, requiring the control system to have good trajectory adaptability; extremely high tracking accuracy is required for welding quality; and dual-arm welding robots must maintain motion synchronization while tracking their respective target trajectories.

[0003] Currently, research on weld seam tracking control largely focuses on single welding robot systems, employing methods including model-based control and learning-based approaches. For example, Chinese invention patent CN121061878A discloses a robotic arm trajectory tracking algorithm based on an improved sliding mode adaptive neural network. This algorithm improves the trajectory tracking accuracy and anti-interference capability of a single robotic arm by establishing a precise dynamic model of the robotic arm and combining sliding mode control with an RBF neural network for disturbance compensation. However, it heavily relies on precise dynamic modeling and parameter identification. These existing methods have significant shortcomings when applied to collaborative scenarios involving two welding robots: model-based methods rely on precise dynamic modeling and exhibit poor robustness under complex weld seam conditions; furthermore, existing reinforcement learning-based methods are mostly designed for single welding robots or simple collaborative tasks, lacking a dedicated mechanism for high-precision collaborative tracking of dual-arm welding robots.

[0004] Therefore, in order to meet the high-precision, highly adaptable, and stable collaborative control requirements of dual-arm welding robots in shipbuilding, it is urgent to develop an intelligent control method that can integrate task perception, precision-driven and collaborative optimization to improve the quality of double-sided welding, system adaptability and automation level. Summary of the Invention

[0005] Purpose of the Invention: The purpose of this invention is to provide a collaborative weld seam tracking control method and system for dual-arm welding robots, which can solve the problems of existing methods relying on precise dynamic modeling, complex parameter tuning, and insufficient adaptability to multiple types of weld seams in collaborative welding scenarios of dual-arm welding robots. By introducing a task modulation and precision driving mechanism, high-precision and robust collaborative tracking of dual-arm welding robots under different weld seam type characteristics can be achieved without precise modeling, significantly improving the system's adaptability to diverse weld seam type characteristics.

[0006] Technical solution: The present invention provides a collaborative weld seam tracking control method for a dual-arm welding robot, comprising:

[0007] Based on the near-end policy optimization algorithm, a task modulation mechanism is introduced; the task modulation mechanism embeds weld type encoding information into the feature representation process, enabling the control model to adapt to tracking control tasks of different types of welds. The task modulation mechanism includes actor network design and critic network design.

[0008] After introducing the task modulation mechanism, a precision-driven segmented training mechanism is introduced into the near-end policy optimization algorithm. The precision-driven segmented training mechanism achieves the transition from low-precision exploration to high-precision optimization by adjusting the tracking error threshold and learning rate set according to the current stage precision in stages, so that the training has a clear adaptive adjustment capability.

[0009] After introducing a precision-driven segmented training mechanism, a precision-driven policy and value objective function optimization mechanism is introduced into the near-end policy optimization algorithm. The precision-driven policy and value objective function optimization mechanism enhances the optimization dynamics and convergence stability of the control model in the high-precision stage by introducing a precision-related scaling factor into the policy and value objective functions, and finally obtains the reinforcement learning control model for collaborative weld seam tracking of the dual-arm welding robot.

[0010] A composite reward function is constructed with the optimization objectives of tracking accuracy, collaborative stability, motion smoothness, and task adaptability in the collaborative weld seam tracking control of a dual-arm welding robot.

[0011] With the goal of maximizing the composite reward function, a reinforcement learning control model for collaborative weld seam tracking of a dual-arm welding robot is trained to obtain a high-precision collaborative weld seam tracking control model for a dual-arm welding robot. The high-precision collaborative weld seam tracking control model for a dual-arm welding robot is used to realize the collaborative tracking control of the dual-arm welding robot under different types of weld seam trajectories.

[0012] Furthermore, the specific process of the actor network design is as follows:

[0013] First, feature extraction of the main observations is performed: intermediate features of the main observation information are extracted using an MLP network. :

[0014] ;

[0015] In the formula, H is the output dimension of the hidden layer; θ is the actor network parameter; The main observation information;

[0016] Then through two independent small MLP networks and Encode the weld trajectory type Two modulation parameters are generated to modulate the intermediate features, enabling the actor network to adaptively respond to different weld types:

[0017] ;

[0018] ;

[0019] In the formula, The modulated feature vector; Scaling factor ; This is the offset. ; This is an element-wise multiplication operation;

[0020] The modulated feature vector The values ​​are fed into two linear layers to output the mean of the action. and logarithmic standard deviation :

[0021] ;

[0022] ;

[0023] In the formula, and These are the weight matrix and bias term of the mean output layer, respectively. and As a linear mapping parameter for the mean of motion; tanh(•) is the hyperbolic tangent function; and These are the weight matrix and bias term of the log-standard deviation output layer, respectively;

[0024] Ultimately, based on the average of the movements and logarithmic standard deviation Calculations are obtained in state Take action below probability :

[0025] ;

[0026] In the formula, It follows a Gaussian distribution.

[0027] Furthermore, the specific process of designing the critic network is as follows:

[0028] This will include main observation information. Weld track type coding The input is sequentially processed through MLP feature extraction and FiLM module modulation, and the final output is a state value scalar:

[0029] ;

[0030] In the formula, A scalar of state value; and These are the weight matrix and bias terms of the value output layer, respectively; This is the modulated feature vector.

[0031] Furthermore, the accuracy-driven segmented training mechanism, through phased adjustment of the tracking error threshold and learning rate set according to the accuracy of the current stage, achieves a transition from low-precision exploration to high-precision optimization, enabling training to have a clear adaptive adjustment capability, including:

[0032] At the beginning of training, a tracking error threshold is set. As the tracking control strategy stabilizes at the current accuracy, the phased training mechanism will reduce the tracking error threshold according to the following formula, thereby improving the training accuracy:

[0033] ;

[0034] In the formula, and The first and The tracking error threshold during the precision training phase; The minimum tracking error threshold; This is a proportional scaling factor;

[0035] The stability of the tracking control strategy is evaluated by the success rate within the sliding window, considering only the most recent success rate shown in the following formula. The success rate of the tracking task is high, thereby effectively mitigating the impact of periodic fluctuations on the evaluation results. Success rate of the tracking task The tracking error threshold will be reduced if the following conditions are met:

[0036]

[0037] In the formula, This is the success rate threshold; The length of the window;

[0038] Meanwhile, during training, an annealing-based accuracy smoothing mechanism is introduced to improve the adaptability of the tracking control strategy to changes in accuracy.

[0039]

[0040] In the formula, The tracking error threshold used during the current training phase; For the first Precision training phase The tracking error threshold.

[0041] Furthermore, the step-by-step adjustment of the tracking error threshold and learning rate set according to the current stage accuracy enables a transition from low-precision exploration to high-precision optimization, giving the training a clear adaptive adjustment capability, including:

[0042] When entering a new precision training phase The initial policy network learning rate during this precision training phase. and value network learning rate It will be set in the following format:

[0043] ;

[0044] ;

[0045] In the formula, and The baseline learning rates for the actor network and the critic network, respectively; and The modulation coefficients for the increase of the actor network and the critic network as precision training progress, respectively, are determined as follows:

[0046] ;

[0047] ;

[0048] In the formula, , The actors' network and the critics' network were respectively in the first place. The learning rate adjustment coefficient corresponding to the precision training phase; and The basic adjustment coefficient for actor networks and critic networks; To train the algebraic adjustment factor;

[0049] During each precision training phase, the control model will adjust the actor network learning rate according to the following formula. and the network learning rate of critics Further exponential decay will occur:

[0050] ;

[0051] ;

[0052] In the formula, This represents the number of iterations for the complete training process. This is the iteration count within the phase; The exponential decay coefficient; This is the minimum learning rate.

[0053] Furthermore, the accuracy-driven policy and value objective function optimization mechanism enhances the optimization dynamics and convergence stability of the control model in the high-precision stage by introducing an accuracy-related scaling factor into the policy and value objective functions. This ultimately yields a reinforcement learning control model for collaborative weld seam tracking of a dual-arm welding robot, including:

[0054] Constructing the precision-aware advantage function The expression is as follows:

[0055] ;

[0056] In the formula, The advantage function before improvement; The modulation coefficient; The tracking error threshold set for the current accuracy;

[0057] Based on the redesigned precision-aware advantage function Construct the following precision-driven pruning strategy optimization objective function. :

[0058] ;

[0059] In the formula, To the number of iterations Experience value at that time; For strategy ratio; This is for the cropping operation.

[0060] Furthermore, the accuracy-driven policy and value objective function optimization mechanism enhances the optimization dynamics and convergence stability of the control model in the high-precision stage by introducing an accuracy-related scaling factor into the policy and value objective functions. This ultimately yields a reinforcement learning control model for collaborative weld seam tracking of a dual-arm welding robot, including:

[0061] Introducing a precision-related scaling factor, the following precision-driven value objective function is constructed:

[0062] ;

[0063] In the formula, To the number of iterations Experience value at that time; Estimate the value function; For the purpose of reward; is the modulation coefficient.

[0064] Furthermore, a composite reward function is constructed with the optimization objectives of tracking accuracy, collaborative stability, motion smoothness, and task adaptability in the collaborative weld seam tracking control of the dual-arm welding robot, including:

[0065] Define distance penalty term :

[0066] ;

[0067] In the formula, It is the sum of the distances between the end-effectors of the two welding robots and their respective weld seams to be tracked; It is a distance penalty factor;

[0068] Define precision excitation term :

[0069] ; ; ;

[0070] In the formula, and These represent the distances between the end-effectors of the first and second welding robots and their respective target weld seams to be tracked; and These represent the current number of completed target points and the total number of target points, respectively. The target schedule factor; This is the error distance adjustment factor;

[0071] Define hit reward items :

[0072] ;

[0073] In the formula, and These represent the time step when the task is completed and the maximum step size of a single agent set at the beginning of training, respectively.

[0074] Define collaborative reward items :

[0075] ;

[0076] In the formula, The Euclidean distance between the end-effectors of the two welding robots;

[0077] Define penalty items for incomplete tasks :

[0078] ;

[0079] Define motion smoothness penalty :

[0080] ;

[0081] In the formula, As a penalty factor for smoothness of motion; The number of joints in a welding robot; and The two welding robots are respectively The joint in the first Joint changes between the previous step and the previous step;

[0082] Final composite reward function The expression is as follows:

[0083] .

[0084] Furthermore, the value of the composite reward function is optimized using a reward normalization method based on a sliding window:

[0085] ;

[0086] In the formula, For normalized rewards; , These are the mean and standard deviation of the rewards obtained in the most recent training step, respectively.

[0087] Based on the same inventive concept, the present invention provides a collaborative weld seam tracking control system for a dual-arm welding robot, comprising:

[0088] The task modulation module is used to introduce a task modulation mechanism based on the near-end policy optimization algorithm. The task modulation mechanism embeds weld type encoding information into the feature representation process, enabling the control model to adapt to tracking control tasks of different types of welds. The task modulation mechanism includes actor network design and critic network design.

[0089] The precision-driven segmented training module is used to introduce a precision-driven segmented training mechanism into the near-end policy optimization algorithm after the introduction of the task modulation mechanism. The precision-driven segmented training mechanism achieves the transition from low-precision exploration to high-precision optimization by adjusting the tracking error threshold and learning rate set according to the current stage precision in stages, so that the training has a clear adaptive adjustment capability.

[0090] The precision-driven optimization module is used to introduce a precision-driven segmented training mechanism and then introduce a precision-driven policy and value objective function optimization mechanism into the near-end policy optimization algorithm. The precision-driven policy and value objective function optimization mechanism enhances the optimization dynamics and convergence stability of the control model in the high-precision stage by introducing a precision-related scaling factor into the policy and value objective functions, and finally obtains the reinforcement learning control model for collaborative weld seam tracking of the dual-arm welding robot.

[0091] The composite reward function building module is used to construct a composite reward function with the optimization objectives of tracking accuracy, collaborative stability, motion smoothness and task adaptability in the collaborative weld seam tracking control of dual-arm welding robots.

[0092] The model training module is used to train the reinforcement learning control model for collaborative weld seam tracking of the dual-arm welding robot with the goal of maximizing the composite reward function, so as to obtain a high-precision collaborative weld seam tracking control model for the dual-arm welding robot; the high-precision collaborative weld seam tracking control model for the dual-arm welding robot is used to realize the collaborative tracking control of the dual-arm welding robot under different types of weld seam trajectories.

[0093] Beneficial effects: Compared with the prior art, the significant technical effects of the present invention are as follows:

[0094] (1) This invention achieves high-precision collaborative weld seam tracking control of a dual-arm welding robot: a multi-stage precision-progressive reinforcement learning method is designed. It combines a task modulation mechanism, a precision-driven segmented training mechanism, and a precision-driven strategy and value objective function optimization mechanism, demonstrating strong adaptability and efficient precision improvement capability, and ensuring the stability and collaborative capability of the control model under complex weld seam characteristic conditions.

[0095] (2) The present invention enhances the system's adaptability and generalization ability: by introducing a task modulation mechanism, the actor network can flexibly adapt to changes in different weld seam types, thereby enhancing the generalization ability of the control model; by combining task modulation and precision adjustment, the robustness and reliability of the welding robot in the face of complex and ever-changing weld seam tracking tasks are further improved.

[0096] (3) This invention improves training stability and convergence efficiency: By introducing a precision-driven segmented training mechanism, the control model proposed in this invention can maintain the stability and continuous optimization of policy updates as the task accuracy gradually improves. This mechanism effectively avoids the instability caused by excessively rapid policy updates, especially in high-precision welding tasks, and avoids the problem of the policy failing to obtain effective learning signals.

[0097] (4) This invention optimizes the overall control performance: by designing a composite reward function, multiple key performance indicators such as tracking accuracy, dual-arm coordination, system stability, and motion smoothness are integrated into a unified optimization objective. The control system trained based on this reward function can stably and efficiently complete tasks for various weld seams, demonstrating excellent tracking accuracy and strong adaptability.

[0098] (5) This invention promotes the application of model-free intelligent control in the field of welding: it provides an intelligent control solution for collaborative welding of dual-arm welding robots that does not rely on a precise dynamic model, is data-driven, and is task-adaptive, and provides a feasible technical path for the application of reinforcement learning in complex industrial collaborative tasks. Attached Figure Description

[0099] Figure 1 This is a flowchart illustrating a collaborative weld seam tracking control method for a dual-arm welding robot disclosed in an embodiment of the present invention.

[0100] Figure 2 This is a schematic diagram of a reinforcement learning control model for collaborative weld seam tracking of a dual-arm welding robot disclosed in an embodiment of the present invention;

[0101] Figure 3 This is a schematic diagram of the task modulation mechanism disclosed in an embodiment of the present invention;

[0102] Figure 4 This is a schematic diagram of a typical shipboard sub-assembly welding model disclosed in an embodiment of the present invention;

[0103] Figure 5 This is a schematic diagram of simulation results under different weld trajectories disclosed in the embodiments of the present invention;

[0104] Figure 6 These are comparison diagrams of end-point errors under different weld trajectories disclosed in the embodiments of the present invention;

[0105] Figure 7 This is a schematic diagram illustrating the changing trend of the simulation experiment reward function disclosed in the embodiments of the present invention;

[0106] Figure 8 This is a schematic diagram illustrating the trend of the success rate of the simulation experiment disclosed in the embodiments of the present invention;

[0107] Figure 9 This is a schematic diagram of the learning rate change curve in the simulation experiment disclosed in the embodiments of the present invention;

[0108] Figure 10 This is a comparison chart of the ablation experiment accuracy variation curves disclosed in the embodiments of the present invention;

[0109] Figure 11This is a comparison chart of the success rate curves of ablation experiments disclosed in the embodiments of the present invention;

[0110] Figure 12 This is a comparison chart of success rate curves for different algorithms disclosed in the embodiments of the present invention.

[0111] Figure 13 This is a comparison chart of quantitative analysis of different algorithms disclosed in the embodiments of the present invention.

[0112] Figure 14 This is a schematic diagram of the structure of a collaborative weld seam tracking control system for a dual-arm welding robot disclosed in an embodiment of the present invention. Detailed Implementation

[0113] The technical solution of the present invention will now be described in detail with reference to specific embodiments and accompanying drawings.

[0114] Example 1

[0115] like Figure 2 As shown, this invention addresses the collaborative weld seam tracking task of a dual-arm welding robot under complex intelligent shipbuilding conditions. Focusing on the core objective of progressively improving tracking accuracy, it proposes a collaborative weld seam tracking control method for dual-arm welding robots that incorporates task modulation and accuracy-driven mechanisms. Through the introduction of a task modulation mechanism and segmented training under accuracy-driven conditions, the optimization of the strategy and value objective function—that is, the accuracy-driven mechanism—achieves high-precision and highly stable collaborative weld seam tracking by the dual-arm welding robot.

[0116] like Figure 1 As shown, a collaborative weld seam tracking control method for a dual-arm welding robot according to the present invention includes the following steps:

[0117] S1, such as Figure 3 As shown, based on the Proximal Policy Optimization (PPO) algorithm, a task modulation mechanism is introduced. The task modulation mechanism embeds weld type encoding information into the feature representation process, enabling the control model to adapt to tracking control tasks of different types of welds. The task modulation mechanism includes actor network design and critic network design.

[0118] The specific implementation process of step S1 is as follows:

[0119] S1.1 Actor Network Design, the specific process is as follows:

[0120] In this invention, the actor network uses a multidimensional Gaussian distribution to model the incremental movements of the welding torch poses at the end of two welding robots, representing the continuous motion space as a conditional probability distribution, thereby achieving a stochastic strategy output for the collaborative weld seam tracking control actions. The input observation vector... It consists of two parts: the main observation information at the front end. And the One-Hot code representing the weld type at the back end. Since there are six types of welds in this embodiment, namely... , and The weld trajectories are straight and curved in three directions; therefore, in the preferred embodiment of the present invention, the weld trajectories are... It is 6.

[0121] Actor Network Design: First, subject observation feature extraction is performed: intermediate features of subject observation information are extracted using an MLP network. :

[0122] ;

[0123] In the formula, The output dimension of the hidden layer is 256 in a preferred embodiment of the present invention; For actor network parameters; The main observation information;

[0124] Then, through two independent small MLP networks and Encode the weld trajectory type Generate two modulation parameters (i.e., scaling factors) respectively. and offset By modulating intermediate features, the actor network can adaptively respond to different weld types:

[0125] ;

[0126] ;

[0127] In the formula, The modulated feature vector; Scaling factor ; This is the offset. ; This is an element-wise multiplication operation;

[0128] Next, the modulated feature vector The values ​​are fed into two linear layers to output the mean of the action. and logarithmic standard deviation :

[0129] ;

[0130] ;

[0131] In the formula, and These are the weight matrix and bias term of the mean output layer, respectively. and As a linear mapping parameter for the mean of action; It is the hyperbolic tangent function; and These are the weight matrix and bias term of the log-standard deviation output layer, respectively;

[0132] Finally, based on the average of the movements and logarithmic standard deviation Calculations are obtained in state Take action below probability :

[0133] ;

[0134] In the formula, It follows a Gaussian distribution.

[0135] S1.2, Critics Network Design, the specific process is as follows:

[0136] The critic network design is basically the same as the actor network, which will include subject observation information. Weld track type coding The input is sequentially processed through MLP feature extraction and FiLM module modulation, and the final output is a state value scalar:

[0137] ;

[0138] In the formula, A scalar of state value; and These are the weight matrix and bias terms of the value output layer, respectively; This is the modulated feature vector.

[0139] The actor network and critic network incorporate a task modulation mechanism to embed weld type encoding information into the feature representation process, enabling the control model to adapt to tracking control tasks for different types of welds.

[0140] S2. After introducing the task modulation mechanism, a precision-driven segmented training mechanism is introduced into the near-end policy optimization algorithm. The precision-driven segmented training mechanism achieves the transition from low-precision exploration to high-precision optimization by adjusting the tracking error threshold and learning rate set according to the current stage precision in stages, so that the training has a clear adaptive adjustment capability.

[0141] Weld seam tracking tasks place extremely stringent requirements on the positional error of the welding robot's end effector. If high precision, i.e., a small tracking error threshold, is directly used as the training objective, reinforcement learning will struggle to obtain effective rewards in the initial exploration phase, leading to a failure of the control strategy to converge. Therefore, this paper designs a precision-driven segmented training mechanism, which guides the tracking control strategy to learn progressively under different precision requirements by adjusting the tracking error threshold in stages.

[0142] The specific implementation process of step S2 is as follows:

[0143] S2.1, The tracking error threshold is adjusted in stages, as follows:

[0144] At the beginning of training, a relatively large tracking error threshold is set. As the tracking control strategy stabilizes at the current accuracy, the phased training mechanism will reduce the tracking error threshold according to the following formula, thereby increasing the training accuracy and enhancing the robot's weld seam tracking capability.

[0145] ;

[0146] In the formula, and The first and The tracking error threshold during the precision training phase; The minimum tracking error threshold ranges from 0.0001 to 0.01, and is set to 0.0003 in a preferred embodiment of the present invention. The scaling factor is a proportional scaling factor, with a value ranging from 0.8 to 0.99, and is set to 0.95 in the preferred embodiment of the present invention.

[0147] The stability of the tracking control strategy is evaluated by the success rate within the sliding window, considering only the most recent success rate shown in the following formula. The success rate of the tracking task is high, thereby effectively mitigating the impact of periodic fluctuations on the evaluation results. Success rate of the tracking task The tracking error threshold will be reduced if the following conditions are met:

[0148] ;

[0149] In the formula, The success rate threshold is set to a value between 0.5 and 1, and is 0.8 in a preferred embodiment of the present invention. The value is the window length, ranging from 20 to 200. In the preferred embodiment of the present invention, it is set to 30. After reaching the final tracking error threshold, it is adjusted to 100 to improve the resolution of the success rate.

[0150] Meanwhile, to prevent strategy collapse due to sudden increases in accuracy during training, this invention introduces an annealing-based accuracy smoothing mechanism to improve the adaptability of the tracking control strategy to changes in accuracy.

[0151]

[0152] In the formula, The tracking error threshold used during the current training phase; For the first Precision training phase The tracking error threshold; when the annealing precision smoothing mechanism is triggered for the first time (i.e., k = 1), due to the lack of... Therefore, only and Probability sampling is performed between these points, which is considered a degenerate situation.

[0153] S2.2, Adjust the learning rate in stages, as follows:

[0154] To coordinate with the phased adjustment of the tracking error threshold, this invention also designs a matching adaptive learning rate adjustment mechanism based on the accuracy phase.

[0155] When entering a new precision training phase k, the initial policy network learning rate for this precision training phase... and value network learning rate It will be set in the following format:

[0156] ;

[0157] ;

[0158] In the formula, and The basic learning rates for the actor network and the critic network are respectively, with values ​​ranging from [value range missing]. In the preferred embodiment of the present invention, respectively take and ; and The modulation coefficients for the increase of the actor network and the critic network as precision training progress, respectively, are determined as follows:

[0159] ;

[0160] ;

[0161] In the formula, , The actors' network and the critics' network were respectively in the first place. The learning rate adjustment coefficient corresponding to the precision training phase; and The basic adjustment coefficients for the actor network and the critic network range from 0.1 to 0.3, and in the preferred embodiment of the present invention, they are all set to 0.15; The training algebraic adjustment factor has a value range of 5 to 100, and in the preferred embodiment of the present invention, it is 50.

[0162] During each precision training phase, the control model will adjust the actor network learning rate according to the following formula. and the network learning rate of critics Further exponential decay will occur:

[0163] ;

[0164] ;

[0165] In the formula, This represents the number of iterations for the complete training process. This is the iteration count within the phase; The exponential decay coefficient has a value range of 0.01 to 0.05, and is 0.02 in the preferred embodiment of the present invention. The minimum learning rate has a range of values. In the preferred embodiment of the present invention, .

[0166] S3. After introducing the precision-driven segmented training mechanism, a precision-driven policy and value objective function optimization mechanism is introduced into the near-end policy optimization algorithm. The precision-driven policy and value objective function optimization mechanism enhances the optimization dynamics and convergence stability of the control model in the high-precision stage by introducing a precision-related scaling factor into the policy and value objective functions, and finally obtains the reinforcement learning control model for collaborative weld seam tracking of the dual-arm welding robot.

[0167] In the PPO algorithm, to suppress excessive policy update magnitude, a pruning mechanism is introduced to construct a pruning-type policy optimization objective function. This objective function constrains the policy change magnitude through the ratio of the probabilities of the old and new policies, and uses the advantage function A... t As a weighting factor, the improvement degree of different "state-action" pairs is measured, thereby guiding the policy to be updated stably in a favorable direction. However, under different accuracy requirements, the advantage function is not adapted to the scale of the tracking error threshold, making it difficult to form an effective synergy with the accuracy-driven segmented training mechanism. To address this, this invention proposes an accuracy-driven policy and value objective function optimization mechanism.

[0168] The specific implementation process of step S3 is as follows:

[0169] S3.1 Constructing a precision-aware advantage function The tracking error threshold set at the current accuracy level is incorporated into the loss calculation to dynamically increase its magnitude based on task accuracy while maintaining its update direction. This provides a stronger optimization driving force for the actor network under high accuracy requirements, helping the tracking control strategy to converge quickly at different accuracy stages and meet the weld seam tracking accuracy requirements. Specifically, this includes:

[0170] Constructing the precision-aware advantage function The expression is as follows:

[0171] ;

[0172] In the formula, The advantage function before improvement; The modulation coefficient has a value ranging from 0.1 to 0.9, and in the preferred embodiment of the present invention, it is 0.5. The tracking error threshold set for the current accuracy;

[0173] Based on the redesigned precision-aware advantage function Construct the following precision-driven pruning strategy optimization objective function. :

[0174] ;

[0175] In the formula, To the number of iterations Experience value at that time; For strategy ratio; This is for the cropping operation.

[0176] S3.2 To ensure consistency between the strategy and the value objective function under the accuracy-driven training framework, an accuracy-related scaling factor is introduced to construct the following accuracy-driven value objective function. :

[0177] ;

[0178] In the formula, To the number of iterations Experience value at that time; Estimate the value function; For the purpose of reward; The value is the modulation coefficient, which ranges from 0.1 to 0.9, and is 0.3 in the preferred embodiment of the present invention.

[0179] S4. A composite reward function is constructed, with tracking accuracy, collaborative stability, motion smoothness, and task adaptability as optimization objectives in the collaborative weld seam tracking control of a dual-arm welding robot. Details are as follows:

[0180] The specific implementation process of step S4 is as follows:

[0181] S4.1 Define the distance penalty term A penalty is applied to the Euclidean distance between the end-effectors of the two welding robots and the corresponding weld seam points to be tracked, in order to encourage them to approach the target as quickly as possible. The expression is as follows:

[0182] ;

[0183] In the formula, It is the sum of the distances between the end-effectors of the two welding robots and their respective weld seams to be tracked; The distance penalty factor has a value range of 5 to 10, and is set to 6 in the preferred embodiment of the present invention.

[0184] Define precision excitation term This term guides two welding robots to gradually learn a high-precision and stable weld seam tracking control strategy during training, as the tracking error threshold gradually tightens. It consists of three parts: a tolerance region reward term, which provides continuous and stable guidance signals during periods of large errors to avoid reward sparsity; a precision incentive term, which directly enhances the weld seam tracking accuracy of the two welding robots; and a target completion progress factor, which characterizes the completion progress of target points in a single weld seam tracking task, guiding the strategy to maintain continuous and stable tracking behavior under multi-target sequences. The expression is as follows:

[0185] ; ; ;

[0186] In the formula, and These represent the distances between the end-effectors of the first and second welding robots and their respective target weld seams to be tracked; and These represent the current number of completed target points and the total number of target points, respectively. The target schedule factor; This is the error distance adjustment factor.

[0187] Define hit reward items When both welding robots reach the set tracking error threshold, a hit reward will be given, and the reward intensity will be dynamically adjusted according to the remaining length of the task. If the end point of the weld is reached, an additional large completion reward will be given to guide the strategy to converge quickly, and the reward intensity will be dynamically adjusted according to the number of completion time steps. The expression is as follows:

[0188] ;

[0189] In the formula, and These represent the time step when the task is completed and the maximum step size of a single agent set at the beginning of training, respectively.

[0190] Define collaborative reward items Encourage collaborative operation between two welding robots by constructing an exponentially decaying reward based on the distance between their end-effector welding torches. The expression is as follows:

[0191] ;

[0192] In the formula, The Euclidean distance between the end-effectors of the two welding robots;

[0193] Define penalty items for incomplete tasks When a single agent reaches its maximum training steps and the weld seam tracking task is not completed, a negative penalty will be imposed based on the progress ratio to prevent the tracking control strategy from repeatedly trying inefficient paths. The expression is as follows:

[0194] ;

[0195] Define motion smoothness penalty To encourage the strategy to generate smoother, lower-energy-consumption motion sequences, penalties are imposed on the range of joint motion changes of the two welding robots, thereby improving the stability and executability of actual tracking control. The expression is as follows:

[0196] ;

[0197] In the formula, The value of the penalty factor for smoothness of motion ranges from 0.1 to 0.5, and is 0.2 in a preferred embodiment of the present invention. The number of joints in the welding robot is 6 in a preferred embodiment of the present invention; and The two welding robots are respectively The joint in the first Joint changes between the previous step and the previous step;

[0198] S4.2 The expression for the composite reward function R is as follows:

[0199] .

[0200] S4.3 To suppress excessive differences in the magnitudes of various sub-items during training, which could cause the policy to deviate from its direction, this invention optimizes the value of the composite reward function using a sliding window-based reward normalization method:

[0201] ;

[0202] In the formula, For normalized rewards; , These are the mean and standard deviation of the reward obtained in the most recent training step, respectively, with a value range of 500 to 1000, and 800 in the preferred embodiment of the present invention.

[0203] A reward function that meets comprehensive requirements such as accuracy, speed, stability, and coordination is constructed through multiple sub-items. Each sub-item is not an isolated module, but is designed around the core objective of "multi-stage accuracy-driven control," thereby forming a training scheme with fine control capabilities.

[0204] S5. With the goal of maximizing the composite reward function, a reinforcement learning control model for collaborative weld seam tracking of a dual-arm welding robot is trained to obtain a high-precision collaborative weld seam tracking control model for a dual-arm welding robot. The high-precision collaborative weld seam tracking control model for a dual-arm welding robot is used to realize the collaborative tracking control of the dual-arm welding robot under different types of weld seam trajectories.

[0205] In step S5, the collaborative weld seam tracking control model of the high-precision dual-arm welding robot is tested and its performance is evaluated.

[0206] The specific implementation process of step S5 is as follows:

[0207] S5.1 The training and testing of the model in this invention are conducted in the same environment to minimize the interference of external factors on the experimental results. Specifically, the hardware platform is an Intel i5-12600kf CPU and an NVIDIA GeForce RTX3080 GPU, and simulation and experimental data processing are performed in Windows 11. Deep learning uses the PyTorch framework in the 2023 version of PyCharm Community Edition, with the compilation environment set to Python 3.9.21 + PyTorch 2.3.1 + CUDA 12.6.

[0208] S5.2 To further verify the effectiveness and superiority of the reinforcement learning control model for collaborative weld seam tracking of the dual-arm welding robot, fixed parameter settings were used during both training and testing. The specific parameter settings in the experiment are shown in Table 1.

[0209] Table 1:

[0210]

[0211] S5.3. By randomly generating weld seam trajectories in the training space, the reinforcement learning control model for collaborative weld seam tracking of the dual-arm welding robot is fully trained to obtain the collaborative weld seam tracking control model of the dual-arm welding robot, and then the model is tested.

[0212] Weld seam tracking tests were conducted using a collaborative weld seam tracking control model with a dual-arm welding robot, as detailed below:

[0213] S5.3.1, such as Figure 4 The image shows a typical welding model of a small assembly in a ship, as disclosed in an embodiment of the present invention. To evaluate the tracking performance of the method of the present invention under different weld trajectories, welds numbered 1-4 were selected as examples for simulation testing of the tracking control of a dual-arm welding robot. Weld number 1 is a curve, while the other three are straight lines. Weld tracking experiments were conducted on this welding model. Figure 5 As shown, this demonstrates the performance of the dual-arm welding robot in tracking a typical weld seam trajectory. Figure 5 Figure (a) in the figure is a schematic diagram of the simulation results of trajectory 1; Figure 5 Figure (b) in the figure is a schematic diagram of the simulation results of trajectory 2; Figure 5 Figure (c) in the figure is a schematic diagram of the simulation results of trajectory 3; Figure 5 Figure (d) in the figure shows a schematic diagram of the simulation results for trajectory 4. It can be clearly seen from the figure that the dual-arm welding robot can accurately complete cooperative weld seam tracking for weld seam trajectories with different orientations and spatial positions. No obvious deviations are observed at points where the curvature of the trajectory changes, and the tracking curve is smooth and continuous. This indicates that, guided by the composite reward function, the trained control model can effectively achieve cooperative weld seam tracking control for both arms.

[0214] S5.3.2, such as Figure 6 As shown, the error distribution of the welding torch at the end of the dual-arm welding robot during weld seam trajectory tracking is illustrated, with the error curves corresponding to each trajectory segment. Figure 4 The weld trajectories shown in Table 2 are presented. The error statistics in Table 2 indicate that the average tracking error of each trajectory is less than 0.21 mm, with the maximum error remaining stable within the range of 0.29–0.30 mm. Furthermore, the error fluctuations between different trajectories are small. The overall error distribution is concentrated, and the root mean square error remains stable, demonstrating that the proposed method can maintain high tracking accuracy and error consistency under different weld directions and spatial positions. The system exhibits good stability and anti-interference capabilities.

[0215] Table 2:

[0216]

[0217] S5.3.3, such as Figure 7As shown, the reward function value gradually increases with the number of iterations and eventually converges to a stable value, indicating that the training process tends to stabilize under the guidance of the accuracy-driven segmented training mechanism. Figure 8 The results demonstrate that the success rate fluctuates in stages with adjustments to the accuracy threshold and then recovers quickly, validating the policy's adaptability to accuracy changes and its reconvergence efficiency. The curve changes reflect the synergistic effect between the reward function design, accuracy stage switching, and policy updates, demonstrating the effectiveness of the training mechanism.

[0218] S5.3.4, such as Figure 9 As shown, the learning rates of the actor network and the critic network exhibit a phased decreasing trend followed by a smooth decay within each phase during the training process. This mechanism balances the need for sufficient exploration in the early stages of training with the requirement for stable convergence in the later stages, while setting a minimum threshold during the learning rate decrease process to prevent training from stagnation. The adaptive adjustment of the learning rate, in conjunction with the accuracy improvement phase, ensures the smoothness of policy updates and the robustness of system convergence.

[0219] To verify the effectiveness of the proposed method, ablation experiments and comparative experiments were conducted. All experiments were performed at a 1 mm accuracy threshold to ensure a fair comparison with existing methods.

[0220] In the ablation experiments, the baseline model without any innovative modules was used as the performance benchmark, and the full model integrating all modules was used as the performance ceiling. By successively removing the accuracy-driven training mechanism (Module A), the task modulation mechanism (Module B), and the accuracy-driven policy and value objective function optimization mechanism (Module C), corresponding variant models were constructed (1. Full Model; 2. Ablation-A: Module A removed; 3. Ablation-B: Module B removed; 4. Ablation-C: Module C removed; 5. PPO: Baseline Model), and compared with the baseline model. All models were trained and tested under the same environment and parameter settings to evaluate the independent and synergistic effects of each module.

[0221] like Figure 10 and Figure 11 As shown, the changes in accuracy threshold and success rate during the training process are illustrated in the ablation experiments. The results indicate that the complete model outperforms all variant models in both accuracy improvement speed and final success rate. Removing either module B or module C leads to slower accuracy convergence and increased success rate fluctuations, validating the necessity and synergistic effect of each module in improving training efficiency and stability.

[0222] Combination Figure 11As shown in the quantitative analysis in Table 4, the complete model outperforms all ablation variant models in terms of the number of ablation cycles required to achieve the required accuracy, the maximum success rate, and the average success rate in the later stages. Among these, removing module A has the most significant impact on performance; removing module B is the second most significant, as its absence weakens the model's adaptability to multiple weld types; while removing module C has a relatively smaller impact on the final success rate, it reduces the reconvergence speed of the strategy during the accuracy variation phase. Experimental results indicate that the combined effect of all modules significantly improves training stability, adaptability, and overall performance, and all modules are essential components with no redundancy.

[0223] Table 4:

[0224]

[0225] To further verify the superiority of the high-precision dual-arm welding robot collaborative weld seam tracking control model, this invention compares it with the PPO, A2C, DDPG and SAC algorithms, which also perform well in welding robot trajectory tracking.

[0226] like Figure 12 As shown in the comparative experiments, the success rate and convergence stability of the method proposed in this invention are superior to mainstream algorithms such as PPO, A2C, DDPG, and SAC. This method achieves a maximum success rate of 100% during training, with less fluctuation in later stages, demonstrating better tracking accuracy and adaptability.

[0227] like Figure 13 As shown in the results, after multiple independent tests, the proposed method achieved an average success rate of 96%, with a maximum of 99%, significantly higher than the compared algorithms such as PPO, A2C, DDPG, and SAC. This result further demonstrates that the proposed method exhibits superior stability and adaptability in tasks involving variable weld trajectories.

[0228] The present invention proposes The algorithm model not only improves the accuracy and stability of collaborative weld seam tracking in dual-arm welding robots, but also provides new ideas and effective technical solutions for the application of reinforcement learning in complex collaborative control tasks.

[0229] Example 2

[0230] like Figure 14 As shown, a collaborative weld seam tracking control system for a dual-arm welding robot according to the present invention includes:

[0231] The task modulation module is used to introduce a task modulation mechanism based on the near-end policy optimization algorithm. The task modulation mechanism embeds weld type encoding information into the feature representation process, enabling the control model to adapt to tracking control tasks of different types of welds. The task modulation mechanism includes actor network design and critic network design.

[0232] The precision-driven segmented training module is used to introduce a precision-driven segmented training mechanism into the near-end policy optimization algorithm after the introduction of the task modulation mechanism. The precision-driven segmented training mechanism achieves the transition from low-precision exploration to high-precision optimization by adjusting the tracking error threshold and learning rate set according to the current stage precision in stages, so that the training has a clear adaptive adjustment capability.

[0233] The precision-driven optimization module is used to introduce a precision-driven segmented training mechanism and then introduce a precision-driven policy and value objective function optimization mechanism into the near-end policy optimization algorithm. The precision-driven policy and value objective function optimization mechanism enhances the optimization dynamics and convergence stability of the control model in the high-precision stage by introducing a precision-related scaling factor into the policy and value objective functions, and finally obtains the reinforcement learning control model for collaborative weld seam tracking of the dual-arm welding robot.

[0234] The composite reward function building module is used to construct a composite reward function with the optimization objectives of tracking accuracy, collaborative stability, motion smoothness and task adaptability in the collaborative weld seam tracking control of dual-arm welding robots.

[0235] The model training module is used to train the reinforcement learning control model for collaborative weld seam tracking of the dual-arm welding robot with a composite reward function as the objective, so as to obtain a high-precision collaborative weld seam tracking control model for the dual-arm welding robot; the high-precision collaborative weld seam tracking control model for the dual-arm welding robot is used to realize the collaborative tracking control of the dual-arm welding robot under different types of weld seam trajectories.

[0236] In an optional embodiment, the collaborative weld seam tracking control method for a dual-arm welding robot includes: a) introducing a task modulation mechanism based on a proximal policy optimization algorithm; b) introducing a precision-driven segmented training mechanism, followed by introducing a precision-driven policy and value objective function optimization mechanism in the proximal policy optimization algorithm; c) introducing a precision-driven policy and value objective function optimization mechanism in the proximal policy optimization algorithm after introducing the precision-driven segmented training mechanism; d) constructing a composite reward function with tracking accuracy, collaborative stability, motion smoothness, and task adaptability as optimization objectives in the collaborative weld seam tracking control of the dual-arm welding robot; e) training the reinforcement learning control model for collaborative weld seam tracking of the dual-arm welding robot with the composite reward function as the objective to obtain a high-precision collaborative weld seam tracking control model for the dual-arm welding robot; and using the high-precision collaborative weld seam tracking control model for the dual-arm welding robot to realize collaborative tracking control under different types of weld seam trajectories.

Claims

1. A collaborative weld seam tracking control method for a dual-arm welding robot, characterized in that, include: Based on the near-end policy optimization algorithm, a task modulation mechanism is introduced; the task modulation mechanism embeds weld type encoding information into the feature representation process, enabling the control model to adapt to tracking control tasks of different types of welds. The task modulation mechanism includes actor network design and critic network design. After introducing the task modulation mechanism, a precision-driven segmented training mechanism is introduced into the near-end policy optimization algorithm. The precision-driven segmented training mechanism achieves the transition from low-precision exploration to high-precision optimization by adjusting the tracking error threshold and learning rate set according to the current stage precision in stages, so that the training has a clear adaptive adjustment capability. After introducing a precision-driven segmented training mechanism, a precision-driven policy and value objective function optimization mechanism is introduced into the near-end policy optimization algorithm. The precision-driven policy and value objective function optimization mechanism enhances the optimization dynamics and convergence stability of the control model in the high-precision stage by introducing a precision-related scaling factor into the policy and value objective functions, and finally obtains the reinforcement learning control model for collaborative weld seam tracking of the dual-arm welding robot. A composite reward function is constructed with the optimization objectives of tracking accuracy, collaborative stability, motion smoothness, and task adaptability in the collaborative weld seam tracking control of a dual-arm welding robot. With the goal of maximizing the composite reward function, a reinforcement learning control model for collaborative weld seam tracking of a dual-arm welding robot is trained to obtain a high-precision collaborative weld seam tracking control model for a dual-arm welding robot. The high-precision collaborative weld seam tracking control model for a dual-arm welding robot is used to realize the collaborative tracking control of the dual-arm welding robot under different types of weld seam trajectories.

2. The collaborative weld seam tracking control method for a dual-arm welding robot according to claim 1, characterized in that, The specific process of the actor network design is as follows: First, feature extraction of the main observations is performed: intermediate features of the main observation information are extracted using an MLP network. : ； In the formula, Output dimension for the hidden layer; For actor network parameters; The main observation information; Then through two independent small MLP networks and Encode the weld trajectory type Two modulation parameters are generated to modulate the intermediate features, enabling the actor network to adaptively respond to different weld types: ； ； In the formula, The modulated feature vector; Scaling factor ; This is the offset. ; This is an element-wise multiplication operation; The modulated feature vector The values ​​are fed into two linear layers to output the mean of the action. and logarithmic standard deviation : ； ； In the formula, and These are the weight matrix and bias term of the mean output layer, respectively. and All are linear mapping parameters of the action mean; It is the hyperbolic tangent function; and These are the weight matrix and bias term of the log-standard deviation output layer, respectively; Ultimately, based on the average of the movements and logarithmic standard deviation Calculations are obtained in state Take action below probability : ； In the formula, It follows a Gaussian distribution.

3. The collaborative weld seam tracking control method for a dual-arm welding robot according to claim 1, characterized in that, The specific process of designing the critic network is as follows: This will include main observation information. Weld track type coding The input is sequentially processed through MLP feature extraction and FiLM module modulation, and the final output is a state value scalar: ； In the formula, A scalar of state value; and These are the weight matrix and bias terms of the value output layer, respectively; This is the modulated feature vector.

4. The collaborative weld seam tracking control method for a dual-arm welding robot according to claim 1, characterized in that, The precision-driven segmented training mechanism, through phased adjustment of the tracking error threshold and learning rate set according to the precision of the current stage, achieves a transition from low-precision exploration to high-precision optimization, enabling training to have a clear adaptive adjustment capability, including: At the beginning of training, a tracking error threshold is set. As the tracking control strategy stabilizes at the current accuracy, the phased training mechanism will reduce the tracking error threshold according to the following formula, thereby improving the training accuracy: ； In the formula, and The first and The tracking error threshold during the precision training phase; The minimum tracking error threshold; This is a proportional scaling factor; The stability of the tracking control strategy is evaluated by the success rate within the sliding window, considering only the most recent success rate shown in the following formula. The success rate of the tracking task is high, thereby effectively mitigating the impact of periodic fluctuations on the evaluation results. Success rate of the tracking task The tracking error threshold will be reduced if the following conditions are met: ； In the formula, This is the success rate threshold; The length of the window; Meanwhile, during training, an annealing-based accuracy smoothing mechanism is introduced to improve the adaptability of the tracking control strategy to changes in accuracy. ； In the formula, The tracking error threshold used during the current training phase; For the first Precision training phase The tracking error threshold.

5. The collaborative weld seam tracking control method for a dual-arm welding robot according to claim 1, characterized in that, The process involves phased adjustment of the tracking error threshold and learning rate based on the current stage's accuracy, enabling a transition from low-precision exploration to high-precision optimization. This allows the training to possess a clear adaptive adjustment capability, including: When entering a new precision training phase The initial policy network learning rate during this precision training phase. and value network learning rate It will be set in the following format: ； ； In the formula, and The baseline learning rates for the actor network and the critic network, respectively; and The modulation coefficients for the increase of the actor network and the critic network as precision training progress, respectively, are determined as follows: ； ； In the formula, , The actors' network and the critics' network were respectively in the first place. The learning rate adjustment coefficient corresponding to the precision training phase; and The basic adjustment coefficient for actor networks and critic networks; To train the algebraic adjustment factor; During each precision training phase, the control model will adjust the actor network learning rate according to the following formula. and the network learning rate of critics Further exponential decay will occur: ； ； In the formula, This represents the number of iterations for the complete training process. This is the iteration count within the phase; The exponential decay coefficient; This is the minimum learning rate.

6. The collaborative weld seam tracking control method for a dual-arm welding robot according to claim 1, characterized in that, The accuracy-driven policy and value objective function optimization mechanism enhances the optimization dynamics and convergence stability of the control model in the high-precision stage by introducing an accuracy-related scaling factor into the policy and value objective functions. This ultimately yields a reinforcement learning control model for collaborative weld seam tracking of a dual-arm welding robot, including: Constructing the precision-aware advantage function The expression is as follows: ； In the formula, The advantage function before improvement; The modulation coefficient; The tracking error threshold set for the current accuracy; Based on the redesigned precision-aware advantage function Construct the following precision-driven pruning strategy optimization objective function. : ； In the formula, To the number of iterations Experience value at that time; For strategy ratio; This is for the cropping operation.

7. The collaborative weld seam tracking control method for a dual-arm welding robot according to claim 1, characterized in that, The accuracy-driven policy and value objective function optimization mechanism enhances the optimization dynamics and convergence stability of the control model in the high-precision stage by introducing an accuracy-related scaling factor into the policy and value objective functions. This ultimately yields a reinforcement learning control model for collaborative weld seam tracking of a dual-arm welding robot, including: Introducing a precision-related scaling factor, the following precision-driven value objective function is constructed. : ； In the formula, To the number of iterations Experience value at that time; Estimate the value function; For the purpose of reward; is the modulation coefficient.

8. The collaborative weld seam tracking control method for a dual-arm welding robot according to claim 1, characterized in that, The aforementioned composite reward function is constructed with tracking accuracy, collaborative stability, motion smoothness, and task adaptability as optimization objectives in the collaborative weld seam tracking control of a dual-arm welding robot. This function includes: Define distance penalty term : ； In the formula, It is the sum of the distances between the end-effectors of the two welding robots and their respective weld seams to be tracked; It is a distance penalty factor; Define precision excitation term : ； ； ； In the formula, and These represent the distances between the end-effectors of the first and second welding robots and their respective target weld seams to be tracked; and These represent the current number of completed target points and the total number of target points, respectively. The target schedule factor; This is the error distance adjustment factor; Define hit reward items : ； In the formula, and These represent the time step when the task is completed and the maximum step size of a single agent set at the beginning of training, respectively. Define collaborative reward items : ； In the formula, The Euclidean distance between the end-effectors of the two welding robots; Define penalty items for incomplete tasks : ； Define motion smoothness penalty : ； In the formula, As a penalty factor for smoothness of motion; The number of joints in a welding robot; and The two welding robots are respectively The joint in the first Joint changes between the previous step and the previous step; Final composite reward function The expression is as follows: 。 9. The collaborative weld seam tracking control method for a dual-arm welding robot according to claim 8, characterized in that, The value of the composite reward function is optimized using a sliding window-based reward normalization method: ； In the formula, For normalized rewards; , These are the mean and standard deviation of the rewards obtained in the most recent training step, respectively.

10. A collaborative weld seam tracking control system for a dual-arm welding robot, characterized in that, include: The task modulation module is used to introduce a task modulation mechanism based on the near-end policy optimization algorithm. The task modulation mechanism embeds weld type encoding information into the feature representation process, enabling the control model to adapt to tracking control tasks of different types of welds. The task modulation mechanism includes actor network design and critic network design. The precision-driven segmented training module is used to introduce a precision-driven segmented training mechanism into the near-end policy optimization algorithm after the introduction of the task modulation mechanism. The precision-driven segmented training mechanism achieves the transition from low-precision exploration to high-precision optimization by adjusting the tracking error threshold and learning rate set according to the current stage precision in stages, so that the training has a clear adaptive adjustment capability. The precision-driven optimization module is used to introduce a precision-driven segmented training mechanism and then introduce a precision-driven policy and value objective function optimization mechanism into the near-end policy optimization algorithm. The precision-driven policy and value objective function optimization mechanism enhances the optimization dynamics and convergence stability of the control model in the high-precision stage by introducing a precision-related scaling factor into the policy and value objective functions, and finally obtains the reinforcement learning control model for collaborative weld seam tracking of the dual-arm welding robot. The composite reward function building module is used to construct a composite reward function with the optimization objectives of tracking accuracy, collaborative stability, motion smoothness and task adaptability in the collaborative weld seam tracking control of dual-arm welding robots. The model training module is used to train the reinforcement learning control model for collaborative weld seam tracking of the dual-arm welding robot with the goal of maximizing the composite reward function, so as to obtain a high-precision collaborative weld seam tracking control model for the dual-arm welding robot; the high-precision collaborative weld seam tracking control model for the dual-arm welding robot is used to realize the collaborative tracking control of the dual-arm welding robot under different types of weld seam trajectories.

Images