Visual prediction fused robot circumference welding control method

Abstract

The invention discloses a visual prediction fused robot circumference welding control method. The method comprises the following steps: firstly, stably extracting weld track characteristics in a strong interference environment through circular motion constrained visual tracking; then, constructing a multi-modal fusion framework, and combining nonlinear state estimation with a space-time attention prediction network to realize prospective compensation of thermal deformation; and finally, a smooth and accurate robot track is generated by adopting hierarchical model predictive control. According to the method, through deep fusion of physical constraint and data driving, the problem of track deviation in the welding process is solved, and the welding seam forming quality and the system adaptability are remarkably improved.

Description

Technical Field

[0001] This invention belongs to the field of robot control, specifically relating to a visual prediction fusion-based robot circular welding control method. Background Technology

[0002] Robotic circular welding is widely used in industrial applications such as pipe circumferential seams and flange connections. This process requires the robot's end effector (welding torch) to perform continuous and uniform welding along the workpiece's circumference to form a closed and mechanically reliable annular weld.

[0003] However, in actual circular welding operations, robot trajectory control faces a series of unique challenges: First, the circular closed trajectory requires extremely high continuity and smoothness of motion. Any slight deviation in posture may cause the arc start point and arc end point of the weld to not match, or produce defects such as uneven layering and coverage. Second, due to the closed welding path, the welding heat input continuously accumulates in the local area of ​​the workpiece and is difficult to dissipate. The resulting nonlinear thermal deformation will significantly change the instantaneous geometry of the workpiece, causing a continuously changing deviation between the preset robot trajectory and the actual weld position. Third, the intense arc light, spatter, and smoke during the welding process seriously interfere with the vision-based weld tracking system, leading to unstable feature extraction.

[0004] Existing technologies typically employ a combination of offline programming and online teaching, but these methods struggle to compensate for dynamic thermal deformation. Online correction methods based on single vision or force sensors are prone to failure under strong interference environments, and their delayed response to thermal deformation leads to untimely compensation. Furthermore, traditional predictive control methods often rely on linear models, which cannot accurately describe the complex nonlinearity of the welding thermal process. Summary of the Invention

[0005] This invention discloses a visual prediction fusion-based robot circular welding control method, comprising the following steps: Step 1, Visual feature tracking and trajectory recognition based on physical constraints: During the welding process, by setting adaptive ROI regions on both sides of the weld and extracting feature points, an optimization objective function containing image brightness matching terms and circular motion model regularization terms is constructed, the displacement vector of the feature points is solved, and the geometric parameters of the weld are obtained by fitting. Step 2, Multimodal Data Fusion and Intelligent Prediction: Simultaneously collect visual, force, and process parameter data, obtain the current system state and thermal deformation offset through a nonlinear state estimator, and use a deep learning model to predict future thermal deformation trends. The online fine-tuning training of the deep learning model adopts a weighted composite loss function, and the loss weights are allocated based on the uncertainty quantification value of the state estimation. The moment with high uncertainty corresponds to a lower loss weight. The state estimation results and prediction results are then adaptively weighted and fused based on confidence to generate a comprehensive compensation amount. Step 3, Hierarchical Model Predictive Control: Based on the comprehensive compensation amount, a reference trajectory is constructed, and a hierarchical predictive controller is used to generate robot control commands. The upper-level controller performs trajectory planning, and the lower-level controller performs trajectory tracking and compliant control.

[0006] Specifically, the objective function to be optimized is: , Where (x,y) represents the pixel coordinates within the neighborhood window of the feature point, I x I y It is the image spatial gradient, I t Let (u,v) be the time gradient, and (u,v) be the displacement vector of the feature point to be determined. m i ,v m i SSIM(·) is the predicted displacement of feature point i based on the circular motion model, λ0 is the basic regularization weight, σ is the scaling parameter, and SSIM(·) calculates the structural similarity of image patches.

[0007] Specifically, the state vector in step 2 includes the robot TCP position and pose and the offset caused by welding thermal deformation.

[0008] Specifically, the deep learning model in step 2 is a spatiotemporal attention Transformer architecture, which learns the long-term dependency between process parameters and thermal deformation through a multi-head self-attention mechanism.

[0009] Specifically, the loss function is as follows:

[0010] Among them, ||·||2 2 Let α, β, and γ represent the square of the L2 norm, N be the number of samples in the training batch, α, β, and γ be the weighting coefficients, and η be the weighting coefficient. i It is a weight based on the quantified value of uncertainty.

[0011] Specifically, in the hierarchical predictive control in step 3, the objective function of the upper-level controller simultaneously considers trajectory tracking error and control cost, and includes kinematic, dynamic and process constraints.

[0012] Specifically, the objective function is: Where u(k|t) is the control input vector to be optimized at discrete time step k, N is the prediction time domain, Q is the weight matrix of the state error, R is the weight matrix of the control input, and ||(·)|| Q 2 for(·) T Q(·), ||(·)|| R 2 for(·)T R(·), X ref =X nominal +ΔX thermal +ω ff ·ΔX ff +ω tr ·Δt·Δ 2 X ff ω ff =sig(δ·CI-μ·||ΔX thermal ||), ω tr =tanh(τ·L), where X =tanh(τ·L), nominal For the nominal trajectory, sig is the sigmoid function, δ is the confidence scaling factor, CI is the confidence level of the deep learning prediction, μ is the bias scaling factor, tanh is the hyperbolic tangent function, τ is the welding scaling factor, and L is the welded length.

[0013] Specifically, in step 3, the lower-level controller receives the optimized trajectory generated by the upper-level controller, combines it with the real-time state estimation deviation, and uses impedance control to achieve accurate trajectory tracking and contact force adjustment.

[0014] Beneficial technical effects: By constraining circular motion, visual tracking robustness is maintained under strong interference such as arcing and splashing; by fusing state estimation and deep learning prediction, feedforward-feedback composite compensation for thermal deformation is achieved, significantly reducing system hysteresis; adaptive weight adjustment based on confidence level intelligently balances the reliability of multi-source information; by using uncertainty quantification weighted training, the prediction accuracy and generalization ability of deep learning models are improved; and a hierarchical predictive control structure is adopted to simultaneously ensure trajectory accuracy and motion smoothness. Attached Figure Description

[0015] Figure 1 A flowchart of the robot circular welding control method based on visual prediction fusion according to the present invention; Figure 2 This is a schematic diagram of trajectory fitting according to the present invention. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] like Figure 1 As shown, this invention discloses a visual prediction fusion-based robot circular welding control method, which includes: Step 1: Physically Constrained Visual Feature Tracking and Trajectory Recognition: In environments with strong interference such as arc light, spatter, and smoke, the geometric features of the weld trajectory are extracted stably and accurately to provide reliable position and pose information for subsequent control. The output of this step, such as the fitted center and radius, is three-dimensional information in the robot's base coordinate system. Through steps such as hand-eye calibration, it can be used as the perceptual input for robot trajectory control.

[0018] Step 1.1: Adaptive feature point detection within the ROI region Based on the baseline template and real-time acquired images, rapid initial localization is performed, and feature points within the Region of Interest (ROI) are detected. Rapid initial localization can be achieved using a multi-scale FFT phase method. In circular welding, the ROI is defined as a strip, with inner and outer annular ROIs defined on both sides of the weld. The specific dimensions can be adaptively adjusted according to the expected deviation range and weld width. For example, the inner ROI is offset inward by 2-4 mm from the ideal trajectory, with a width of 4-8 mm; the outer ROI is offset outward by 2-4 mm from the ideal trajectory, with a width of 4-8 mm. Feature point detection can be performed using the Shi-Tomasi corner detection method. Specifically, the spatial gradient of each ROI is calculated, the autocorrelation matrix of each pixel is constructed, the corner response function is calculated, a response threshold is set, and points with a response function greater than the threshold are retained. Non-maximum suppression is performed within a 5×5 window, ultimately retaining multiple (e.g., 15-25) of the best-quality corner points in both the inner and outer ROIs.

[0019] Step 1.2: Feature point tracking and trajectory fitting under circular motion constraints Step 1.2.1 Construct the constrained optimization objective function for the feature points: For feature points on the circular weld seam trajectory, they satisfy the circular motion constraint. Furthermore, under the same lighting conditions, their brightness theoretically remains constant at different times. Therefore, for the i-th feature point, within its neighborhood window W (e.g., 15×15 pixels), an optimization objective function E containing image brightness matching terms and motion model regularization terms can be constructed: , .

[0020] Where (x,y) represents the pixel coordinates within the neighborhood window of the feature point, I x I y It is the image spatial gradient, I t The time gradient is (u,v), and (u,v) is the displacement vector of the feature point to be determined, i.e., the amount of motion of the feature point in the image plane from the previous frame to the current frame. m i ,v m iSSIM(·) is a prediction of the displacement of feature point i based on a circular motion model. It can predict the ideal circular motion displacement of feature point i based on welding angular velocity and image frame time interval. λ0 is the basic regularization weight, and σ is a scaling parameter set according to the image noise level to balance the trust between image data and motion model. SSIM(·) calculates the structural similarity of image blocks. The regularization weight λ can be dynamically adjusted according to image quality (such as contrast and noise level). When the image quality is high and the similarity is large, the proportion of circular motion constraint is reduced and the image data is trusted. When the image quality is poor and the similarity is small, the proportion of circular motion constraint is increased and the motion model is trusted, thereby achieving intelligent balance.

[0021] Step 1.2.2 Construct a system of linear equations and update the feature point positions: Solve for (u,v) and update the feature point positions p. t i =p t-1 i +(u,v), where p t i p represents the position coordinates of the i-th feature point in the current frame. t-1 i This represents the position coordinates of the i-th feature point in the previous frame.

[0022] Step 1.2.3 Trajectory Fitting and Baseline Template Update: Randomly select 3 non-collinear points (circle fitting) or 5 points (ellipse fitting) from the feature points. Solve for the circle or ellipse parameters, including the center and radius, through geometric constraints to obtain the fitted circle or ellipse. Calculate the distance from all points to the fitted circle or ellipse, and set a threshold to filter points that meet the model parameters. Iterate and repeat multiple times, selecting the model with the most points that meet the model parameters. Use all points that meet the model parameters to fit the optimal circle or ellipse using the least squares method, and output the center and radius of the current weld trajectory. Continuously monitor the changing trend of the circle or ellipse parameters. When the center position and / or radius exceed the preset threshold, dynamically update the template.

[0023] Existing visual tracking methods are prone to losing feature points and experiencing tracking discontinuities under strong welding interference (arc light, spatter, smoke), such as... Figure 2 As shown, this step enables visual tracking to make judgments under extreme interference. By introducing a circular motion model as a constraint, even if splashing temporarily obscures some feature points, the algorithm can infer their reasonable positions based on the motion trend, which greatly improves the robustness and continuity of feature tracking and provides a more stable and reliable data source for subsequent control.

[0024] Step 2: Multimodal data fusion and intelligent prediction: Achieve accurate estimation of the current state and intelligent prediction of future trends, forming full-band compensation capability.

[0025] Step 2.1 Multi-source data synchronous acquisition and preprocessing: Synchronously acquire data from multiple sources, perform data alignment and preprocessing to ensure that data from each source is collected at the same time. Angle information of each joint can be read from the robot controller, and the robot's TCP position and orientation can be obtained through kinematic calculations. Simultaneously, real-time parameters of the welding process are acquired, including welding current, arc voltage, wire feed speed, etc., and the cumulative heat input is calculated. External sensor data includes visual data and tactile data. Visual data includes the TCP position and pose information identified by the aforementioned vision system, and tactile data includes contact force information measured by force sensors. Preprocessing may include distortion correction and coordinate system transformation of the visual images, converting the image coordinates to three-dimensional position and pose in the robot's base coordinate system, and filtering the tactile data to eliminate high-frequency noise and transient interference.

[0026] Step 2.2 State Estimation Based on Nonlinear Filtering: Based on the current system state estimate and covariance matrix, generate a set of state sampling points representing the current state distribution of the system; calculate the predicted state and covariance for the next moment using the state transition function (based on robot kinematics and thermal deformation model); utilize the observations from vision and force sensors, map visual observations, force observations, and process parameter observations through the observation function, calculate the Kalman gain, update the state estimate and covariance based on the difference between the actual and predicted observations, and output the optimal current state estimate result and covariance matrix. The covariance matrix is ​​the quantified value of the uncertainty of the state estimate. The system state includes the robot's real-time motion state and the weld thermal deformation offset, thus the state vector X corresponding to the system state includes: the position and pose of the robot TCP (tool center point) in the base coordinate system and the weld thermal deformation offset, i.e., X=[p x ,p y ,p z ,θ x ,θ y ,θ z ,Δp x-thermal ,Δp y-thermal ,Δp z-thermal ,Δθ x-thermal ,Δθ y-thermal ,Δθ z-thermal ], where the weld thermal deformation offset ΔX thermal =[Δp x-thermal ,Δp y-thermal ,Δp z-thermal ,Δθ x-thermal ,Δθ y-thermal ,Δθ z-thermal ], where p x ,p y ,p z For TCP position coordinates, θ x ,θ y ,θz For TCP attitude angle, Δp x-thermal ,Δp y-thermal ,Δp z-thermal Δθ represents the positional offset of the TCP in each direction caused by thermal deformation. x-thermal ,Δθ y-thermal ,Δθ z-thermal The TCP (Tail Collision Point) attitude angular offsets caused by thermal deformation. Predicting the state at the next moment includes predicting position and pose based on current velocity and acceleration, predicting velocity based on current acceleration, and updating thermal deformation based on welding current and heat accumulation coefficient. The weld thermal deformation offset represents the compensation amount for the robot trajectory caused by thermal deformation, including translation and rotation. That is, the offset of the robot TCP point relative to the desired position caused by localized thermal deformation of the workpiece due to welding heat input. For example, when the workpiece expands due to heat, the TCP needs to compensate for this deformation to maintain the correct trajectory. The output includes the optimal current state estimation results of position and pose estimation and thermal deformation offset ΔX. thermal This will be used to calculate the deviation between the current actual position pose and the desired position pose. This deviation will serve as the feedback input for the subsequent Model Predictive Controller (MPC) to adjust the robot's trajectory in real time.

[0027] This step focuses on using high-frequency visual and force data to perform millisecond-level optimal current state estimation and filter out sensor noise. Its output (such as the precise current thermal deformation) can serve as a high-quality training label and validation benchmark for subsequent deep learning. By fusing multi-sensor data, filtering out noise from individual sensors, and leveraging the complementary characteristics of different sensors (absolute position accuracy of vision, contact information from force sensing), high-frequency, high-precision estimation of the robot's state and thermal deformation is achieved, solving the problems of unreliability and noise interference from single sensors.

[0028] Step 2.3 Deep Learning Prediction Based on Spatiotemporal Attention Mechanism: This step focuses on learning the slow-varying nonlinear dynamics of the welding process to predict future state evolution. The input to deep learning includes time-series data of welding process parameters, weld geometry (plate thickness, bevel angle, weld length, material type encoding), historical information on heat accumulation (proportion of welded length, cumulative heat input, number of local thermal cycles), environmental factors (ambient temperature, initial workpiece temperature), and specially added quantification information on state estimation uncertainty. These features are processed by a pre-trained spatiotemporal attention Transformer model. The Transformer model is a multi-layer Transformer model, with each layer containing a multi-head self-attention mechanism and a feedforward neural network. The self-attention mechanism is used to capture the long-term dependence of process parameters on thermal deformation at each time step in the input sequence and calculate their relative importance weights. The feedforward network performs nonlinear transformation. Before model training, a large amount of welding process data containing various working conditions (different currents, voltages, plate thicknesses, welding speeds) needs to be collected, along with the state estimation results ΔX at the same time step. thermal As supervised labels, the training process includes offline pre-training and online fine-tuning. Offline pre-training uses historical datasets, with mean squared error as the primary loss function, and employs the Adam optimizer with a cosine annealing learning rate. Online fine-tuning involves fine-tuning the pre-trained model using a small amount of data collected for the current task when welding begins on a specific workpiece, allowing it to quickly adapt to the specific working conditions. The loss function includes absolute error, trend error, and smoothness constraints.

[0029] Among them, ||·||2 2 The loss function represents the square of the L2 norm, where N is the number of samples in the batch. During training, model weights are not updated using a single data sample, but rather multiple samples are combined into a "batch." The total or average loss for this batch is then calculated, and a weight update is performed based on this. N in the loss function represents the batch size, i is the index of a sample in the batch (from 1 to N), and α, β, and γ are weight coefficients used to balance the importance of different parts of the loss function. These are typically set empirically or through grid search. η i It is based on the weights of the uncertainty quantization value (inverse of the covariance matrix) in step 2.2. For moments with high uncertainty quantization values, η i When the uncertainty quantification value is small, its weight is reduced; when the uncertainty quantification value is low, its weight is increased, so that the model focuses more on the data with high confidence during training, thereby improving the robustness and prediction accuracy of the model.

[0030] Deep learning uses fully connected layers to map the features output by the Transformer to predicted thermal deformation values, resulting in a predicted thermal deformation value ΔX over a future period. ff =[Δp x-ff ,Δp y-ff ,Δp z-ff ,Δθ x-ff ,Δθ y-ff ,Δθ z-ff ] and the trend of change Δ 2 X ff Δ 2 (·)=dΔ(·) / dt, where Δp x-ff ,Δp y-ff ,Δp z-ff Δθ represents the predicted future thermal deformation deviation of the TCP position coordinates. x-ff ,Δθ y-ff ,Δθ z-ff To predict the future thermal deformation deviation of TCP at various attitude angles, the confidence level (CI) of this prediction is evaluated based on the quality of the input features and the degree of dispersion of the internal attention mechanism.

[0031] Output thermal deformation prediction value ΔX ff That is, the predicted future deviation and the thermal deformation offset ΔX in step 2.2. thermal The estimated current deviation is dynamically weighted and summed to obtain the total compensation. Through this composite compensation, the system can compensate for thermal deformation more timely and accurately, improving welding quality. When the confidence level of the deep learning prediction is high, the weight of the output thermal deformation prediction value is increased; when the uncertainty quantization value of the state estimation is high, the weight of the thermal deformation offset in step 2.2 is increased, thereby obtaining the final compensation amount, which is then converted into angle commands for each joint of the robot through inverse kinematics calculation.

[0032] This compensation amount possesses real-time accuracy and interpretability of state estimation, as well as the powerful predictive ability of deep learning for complex nonlinear systems, thus achieving full-band, forward-looking, and accurate compensation for welding trajectory deviations.

[0033] Furthermore, the difference between the deep learning prediction and the state updated by observation in step 2.2 can be compared. If the deep learning prediction is more accurate than the state estimation in step 2.2, the deep learning prediction can be used to correct the state estimation process in step 2.2, including but not limited to using the state change predicted by deep learning as a correction term in the state estimation process in step 2.2, that is, adding the correction term predicted by deep learning to the state update function of the state estimation in step 2.2.

[0034] Traditional feedback control suffers from response lag in the face of slowly varying and nonlinear thermal deformation, and struggles to accurately predict complex welding thermal processes using only physical models. This invention addresses this issue by proactively predicting thermal deformation and advancing compensation actions, significantly reducing system tracking lag and achieving feedforward compensation for nonlinear thermal deformation, complementing feedback compensation from state estimation. This deeply integrated approach embeds uncertainty information from state estimation into deep learning predictions and utilizes deep learning to correct model parameters for state estimation, enabling organic collaboration between the two intelligent modules. The system can intelligently adjust control strategies based on perceived quality and predictive reliability, significantly improving the accuracy and adaptability of welding trajectory tracking while ensuring real-time response.

[0035] Step 3: Hierarchical Model Predictive Control: Based on fused information, smooth, accurate, and forward-looking control commands are generated to achieve full-band disturbance suppression. It can be divided into an upper-layer trajectory planner (MPC) and a lower-layer high-precision motion actuator. The upper layer solves large-scale trajectory optimization with prediction, while the lower layer handles high-frequency fine-tuning, resulting in reasonable computational load distribution.

[0036] The optimization objective of the upper-level trajectory planner MPC is to minimize the difference between the actual trajectory and the reference trajectory X in the future time domain. ref To minimize errors while ensuring smooth motion and continuous acceleration, the prediction model includes a robot kinematics model and a welding process thermal deformation dynamics model (such as the heat input-deformation relationship). The output of MPC is the optimized and compensated smoothed desired robot trajectory over a future period. MPC constraints include kinematic constraints (joint limits, velocity and acceleration limits), dynamic constraints (torque limits, inertial coupling effects), and process constraints (uncertainty boundaries for thermal deformation prediction). The optimal current state estimation results for position, pose, velocity, and acceleration are used as initial conditions for the MPC prediction model to ensure prediction accuracy.

[0037] The objective function of MPC is:

[0038] Where u(k|t) is the control input vector to be optimized at discrete time step k (calculated from the current time t), specifically the instruction sent to the robot actuator, such as the torque of the joint, or the velocity / acceleration instruction of the end effector; N is the prediction time domain, i.e., how many future time steps the controller predicts during optimization; Q is the weight matrix of the state error; R is the weight matrix of the control input; ||(·)|| Q 2 The weighted L2 norm square is specifically calculated as (·). T Q(·), ||(·)|| R 2 for(·) TR(·).

[0039] Reference trajectory X ref Specifically: X ref =X nominal +ΔX thermal +ω ff ·ΔX ff +ω tr ·Δt·Δ 2 X ff ω ff =sig(δ·CI-μ·||ΔX thermal ||), ω tr =tanh(τ·L), where X =tanh(τ·L), nominal The nominal trajectory is the pre-programmed ideal robot trajectory (including position and pose). `sig` is the sigmoid function, and `δ` is the confidence scaling factor, used to adjust the influence of confidence on the weights. The larger `δ` is, the greater the impact of confidence on `ω`. ff The stronger the influence, the more likely it is to be set based on the system sensitivity. CI is the confidence level of the deep learning prediction, a scalar value (between 0 and 1) representing the reliability of the prediction. It can be calculated based on the model's internal attention mechanism or prediction variance. μ is the bias scaling factor, used to adjust the impact of the current bias on the weights. The larger μ is, the stronger the impact of the current bias on ω. ff The stronger the suppression, the lower the prediction weight when the current deviation is large. tanh is the hyperbolic tangent function, and τ is the welding scaling factor used to adjust the influence of welding length on the weight. The larger τ is, the greater the influence of welding length on ω. tr The stronger the influence, the greater the trend compensation weight ω, where L is the welded length, a scalar value representing the welding progress. It reflects the degree of heat accumulation; the longer the weld length, the more significant the thermal deformation may be. tr It should be increased.

[0040] These parameters work together to achieve intelligent compensation: when the prediction confidence is high, increase ω. ff When the current deviation is large, reduce ω. ff As welding progresses, ω is gradually increased. tr To cope with cumulative thermal deformation.

[0041] The lower-level high-precision motion actuator receives the desired trajectory output from the upper-level MPC and, combined with the real-time minute deviations from the state estimation in step 2.2, performs PID and / or impedance control to achieve final accurate trajectory tracking. Force sensor information at this layer is primarily used to achieve compliant control, preventing excessive contact between the welding torch and the workpiece.

[0042] The upper-level MPC optimizes the fused reference trajectory to generate a smooth, feasible robot trajectory that has been pre-compensated for predicted thermal deformation. This solves the problems of jitter and trajectory non-compliance with dynamic constraints that may be caused by direct control. The lower-level controller receives the upper-level trajectory and performs high-frequency tracking. Combined with force sensing, it achieves compliant control, realizes the final accurate trajectory tracking, and prevents hard collisions between the welding torch and the workpiece, thus improving the safety and quality of the welding process.

[0043] In the several embodiments provided by this invention, it should be understood that the disclosed methods can be implemented in other ways. For example, the embodiments described above are merely illustrative. It will be apparent to those skilled in the art that this invention is not limited to the details of the above exemplary embodiments, and that it can be implemented in other specific forms without departing from the basic characteristics of the invention. Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this invention and not to limit it. Although this invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of this invention without departing from the spirit and scope of the technical solutions of this invention.

Claims

1. A robot circular welding control method based on visual prediction fusion, characterized in that, Includes the following steps: Step 1, Visual feature tracking and trajectory recognition based on physical constraints: During the welding process, by setting adaptive ROI regions on both sides of the weld and extracting feature points, an optimization objective function containing image brightness matching terms and circular motion model regularization terms is constructed, the displacement vector of the feature points is solved, and the geometric parameters of the weld are obtained by fitting. Step 2, Multimodal Data Fusion and Intelligent Prediction: Simultaneously collect visual, force, and process parameter data, obtain the current system state and thermal deformation offset through a nonlinear state estimator, and use a deep learning model to predict future thermal deformation trends. The online fine-tuning training of the deep learning model adopts a weighted composite loss function, and the loss weights are allocated based on the uncertainty quantification value of the state estimation. The moment with high uncertainty corresponds to a lower loss weight. The state estimation results and prediction results are then adaptively weighted and fused based on confidence to generate a comprehensive compensation amount. Step 3, Hierarchical Model Predictive Control: Based on the comprehensive compensation amount, a reference trajectory is constructed, and a hierarchical predictive controller is used to generate robot control commands. The upper-level controller performs trajectory planning, and the lower-level controller performs trajectory tracking and compliant control.

2. The method according to claim 1, characterized in that, The objective function to be optimized is: , Where (x,y) represents the pixel coordinates within the feature point's neighborhood window W, and I x I y It is the image spatial gradient, I t Let (u,v) be the time gradient, and (u,v) be the displacement vector of the feature point to be determined. m i ,v m i SSIM(·) is the predicted displacement of feature point i based on the circular motion model, λ0 is the basic regularization weight, σ is the scaling parameter, and SSIM(·) calculates the structural similarity of image patches.

3. The method according to claim 1, characterized in that, The state vector in step 2 includes the robot's TCP position and pose, as well as the offset caused by welding thermal deformation.

4. The method according to claim 1, characterized in that, The deep learning model in step 2 is a spatiotemporal attention Transformer architecture, which learns the long-term dependency between process parameters and thermal deformation through a multi-head self-attention mechanism.

5. The method according to claim 1, characterized in that, The loss function is specifically as follows: , of which ||·||2 2 Let α, β, and γ represent the square of the L2 norm, N be the number of samples in the training batch, α, β, and γ be the weighting coefficients, and η be the weighting coefficient. i It is a weight based on the quantized value of uncertainty, ΔX thermal ΔX represents the thermal deformation offset. ff This is the predicted value for thermal deformation.

6. The method according to claim 1, characterized in that, In the hierarchical predictive control in step 3, the objective function of the upper-level controller simultaneously considers trajectory tracking error and control cost, and includes kinematic, dynamic and process constraints.

7. The method according to claim 6, characterized in that, The objective function is: Where u(k|t) is the control input vector to be optimized at discrete time step k, N is the prediction time domain, Q is the weight matrix of the state error, and ||(·)|| Q 2 for(·) T Q(·), R is the weight matrix of the control input, ||(·)|| R 2 for(·) T R(·), X ref Specifically: X ref =X nominal +ΔX thermal +ω ff ·ΔX ff +ω tr ·Δt·Δ 2 X ff ω ff =sig(δ·CI-μ·||ΔX thermal ||), ω tr =tanh(τ·L), where X is the actual trajectory, X nominal For the nominal trajectory, sig is the sigmoid function, δ is the confidence scaling factor, CI is the confidence level of the deep learning prediction, μ is the bias scaling factor, tanh is the hyperbolic tangent function, τ is the weld scaling factor, L is the welded length, and ΔX is the welded length. thermal ΔX represents the thermal deformation offset. ff This is the predicted value for thermal deformation.

8. The method according to claim 1, characterized in that, In step 3, the lower-level controller receives the optimized trajectory generated by the upper-level controller, combines it with the real-time state estimation deviation, and uses impedance control to achieve accurate trajectory tracking and contact force adjustment.

Images