Welding robot track self-adaptive correction method based on multi-modal data fusion

Abstract

The invention discloses a welding robot track self-adaptive correction method based on multi-modal data fusion, and particularly relates to the technical field of industrial robots, which comprises the following steps: synchronously acquiring visual three-dimensional point cloud and arc current and voltage transient waveform data in a welding process in real time; an enhanced working trajectory feature vector fusing the visual trajectory features and the arc deviation compensation amount is constructed; respectively calculating real-time confidence coefficients of the visual sense and the arc sensing mode; inputting the confidence coefficient into a weight distribution strategy model, dynamically resolving vision and arc weight coefficients, and when the confidence coefficient of one sensor is low, lifting the weight of the other sensor to dominate correction; performing weighted fusion on the two pose adjustment amounts by using a weight coefficient to generate a six-degree-of-freedom comprehensive trajectory correction amount; and finally, the robot is driven to execute online adaptive trajectory correction. Through a confidence-driven dynamic fusion strategy, the limitation of a single sensor is overcome, and the welding robot trajectory tracking precision and the adaptive capacity are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of industrial robots, more particularly, the present application relates to a welding robot trajectory adaptive correction method based on multi-modal data fusion. BACKGROUND

[0002] In the field of automated robot operation, industrial robots usually operate according to preset programmed trajectories. However, due to factors such as workpiece machining and assembly errors, welding thermal deformation, etc., the actual weld position may deviate from the preset trajectory, and in order to ensure welding quality, the robot must have the ability to perceive and correct the trajectory online. In the prior art, a single type of sensor is generally used for trajectory tracking and correction. Among them, the scheme based on visual sensor acquires the macro-geometric information of the weld through a two-dimensional camera or a three-dimensional scanner, and then guides the robot to track the trajectory; the scheme based on arc sensor utilizes the mapping relationship between welding electrical parameters and arc length to realize closed-loop control of the welding torch pose, which has advantages in fine-tuning of welding torch height and transverse deviation.

[0003] However, in actual use, there are still some shortcomings, such as the vision-based scheme is susceptible to environmental interference in the welding site, resulting in a decline in image quality and failure of feature extraction, at which time if still relying on visual data for control, it will lead to incorrect robot trajectory correction or even loss of control; the arc-based scheme can only perceive the local and microscopic changes in the relative position between the welding torch and the workpiece, and cannot obtain the overall three-dimensional spatial trend of the weld, thus cannot independently complete the macro trajectory tracking task; some existing simple multi-sensor fusion methods mostly use fixed weights or logical switching, and cannot make dynamic and flexible fusion decisions according to the real-time reliability of sensor data during welding; when a sensor suddenly loses data reliability due to sudden interference, the fixed weight fusion method cannot timely suppress its adverse effects, resulting in inaccurate trajectory correction after fusion, thereby affecting the welding forming quality and process stability SUMMARY

[0004] In order to overcome the above-mentioned defects of the prior art, the present application provides a welding robot trajectory adaptive correction method based on multi-modal data fusion, which solves the problems raised in the background art by the following scheme.

[0005] To achieve the above-mentioned purpose, the present application provides the following technical scheme: a welding robot trajectory adaptive correction method based on multi-modal data fusion, comprising:

[0006] S1: real-time synchronous acquisition of multi-modal sensor data in the operation process, including three-dimensional point cloud data of the working area acquired by the visual sensor, and welding current and voltage transient waveform data acquired by the arc sensor;

[0007] S2: Constructing a multi-modal feature vector of the working trajectory

[0008] S201: Processing the three-dimensional point cloud data to extract a three-dimensional space coordinate sequence of the weld centerline in the robot base coordinate system as a visual trajectory feature;

[0009] S202: Processing the transient waveform data to extract a feature quantity representing the dynamic change of the arc length, and based on a pre-set arc length and welding gun pose mapping model, calculating the feature quantity into a lateral deviation compensation quantity and a height deviation compensation quantity of the welding gun end;

[0010] S203: Spatially registering the visual trajectory feature, the lateral deviation compensation quantity and the height deviation compensation quantity to form an enhanced working trajectory feature vector;

[0011] S3: Based on the enhanced working trajectory feature vector, respectively calculating the real-time confidence of the visual and arc sensing modalities; wherein the visual confidence is quantified based on the point cloud density and noise rate of the three-dimensional point cloud data, and the arc confidence is quantified based on the signal-to-noise ratio of the transient waveform data;

[0012] S4: Inputting the real-time confidence into a pre-set weight distribution strategy model to dynamically calculate the visual weight coefficient and the arc weight coefficient; the weight distribution strategy model is configured to perform the following rules: when the visual confidence is lower than a first threshold, the arc weight coefficient is increased to dominate the height direction correction; when the arc confidence is lower than a second threshold, the visual weight coefficient is increased to dominate the three-dimensional space trajectory tracking;

[0013] S5: Using the calculated weight coefficients to weight and fuse the pose adjustment quantity determined by the visual trajectory feature and the pose adjustment quantity determined by the arc deviation compensation quantity to generate a six-degree-of-freedom comprehensive trajectory correction quantity of the robot end effector;

[0014] S6: Inputting the comprehensive trajectory correction quantity into the robot controller to drive the welding robot to perform online adaptive trajectory correction.

[0015] Technical effects and advantages of the present application:

[0016] 1. Through multi-modal fusion of visual and arc sensors, the limitations of single sensor technology are overcome. When welding spatter and smoke cause a decrease in visual data quality, the system can enhance the reliance on arc data, and vice versa;

[0017] 2. Real-time confidence quantification and evaluation of visual and arc sensing modalities are introduced, and weights are dynamically distributed accordingly, which can sense the working state of each sensor in real time, improving the accuracy and safety of trajectory correction;

[0018] 3、The method effectively combines the advantages of visual sensors in macro three-dimensional space trajectory tracking and the high precision characteristics of arc sensors in micro height and lateral deviation compensation. Through weighted fusion, the generated comprehensive trajectory correction quantity can ensure that the robot moves correctly along the macro weld direction and maintains the optimal welding posture and arc length in real time, achieving collaborative control effect and improving the welding forming quality;

[0019] 4、The whole method forms a real-time closed-loop system from perception, evaluation, decision-making to execution. The robot is no longer just an equipment that executes preset programs, but can adjust the trajectory online and adaptively, reducing the dependence on precise tooling and offline programming, and improving the ability of the robot to cope with product diversity and small batch production. BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1 is a schematic diagram of the overall structure of the present application.

[0021] Figure 2 is a schematic diagram of the data acquisition and feature extraction structure of the present application.

[0022] Figure 3 is a schematic diagram of the confidence calculation and weight distribution structure of the present application.

[0023] Figure 4 is a schematic diagram of the trajectory correction and execution structure of the present application.

[0024] Figure 5 is a relationship function diagram of the six-degree-of-freedom comprehensive trajectory correction quantity of the present application. DETAILED DESCRIPTION

[0025] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0026] Reference Figures 1-5 The welding robot trajectory adaptive correction method based on multi-modal data fusion shown includes:

[0027] S1: Real-time synchronous acquisition of multi-modal sensing data in the working process, including three-dimensional point cloud data of the working area obtained by the visual sensor, and welding current and voltage transient waveform data obtained by the arc sensor;

[0028] S2: Constructing multi-modal feature vectors of the working trajectory:

[0029] S201: processing the three-dimensional point cloud data, extracting the three-dimensional space coordinate sequence of the weld centerline in the robot base coordinate system as the visual trajectory feature;

[0030] S202: processing the transient waveform data, extracting a feature quantity representing the dynamic change of the arc length, and based on a preset arc length and welding gun pose mapping model, calculating the feature quantity into a lateral deviation compensation quantity and a height deviation compensation quantity of the welding gun end;

[0031] S203: spatially registering the visual trajectory feature, the lateral deviation compensation quantity and the height deviation compensation quantity, and fusing to form an enhanced working trajectory feature vector;

[0032] S3: based on the enhanced working trajectory feature vector, respectively calculating the real-time confidence of the visual and arc sensing modalities; wherein the visual confidence is quantified based on the point cloud density and noise rate of the three-dimensional point cloud data, and the arc confidence is quantified based on the signal-to-noise ratio of the transient waveform data;

[0033] S4: inputting the real-time confidence into a preset weight distribution strategy model to dynamically calculate the visual weight coefficient and the arc weight coefficient; the weight distribution strategy model is configured to perform the following rules: when the visual confidence is lower than a first threshold, the arc weight coefficient is increased to dominate the height direction correction; when the arc confidence is lower than a second threshold, the visual weight coefficient is increased to dominate the three-dimensional space trajectory tracking;

[0034] S5: using the calculated weight coefficients to weight and fuse the pose adjustment quantity determined by the visual trajectory feature and the pose adjustment quantity determined by the arc deviation compensation quantity, to generate a six-degree-of-freedom comprehensive trajectory correction quantity of the robot end effector;

[0035] S6: inputting the comprehensive trajectory correction quantity into the robot controller to drive the welding robot to perform online adaptive trajectory correction.

[0036] The S1: real-time synchronous acquisition of multi-modal sensing data during the operation process: is the basis for realizing multi-modal fusion, and the core lies in the synchronization of data and the effectiveness of data. The specific implementation process is as follows:

[0037] Visual data acquisition: a structured light vision sensor is used to scan the weld and its adjacent area at a frequency of 50Hz while the robot is welding, the sensor captures the weld image modulated by structured light, and through stereo vision algorithm, generates three-dimensional point cloud data of the working area. The point cloud data contains the three-dimensional space coordinate information of the weld bevel and the workpiece surface in the sensor coordinate system.

[0038] Arc data acquisition: Hall effect current sensor and voltage isolation sensor are used to synchronously collect the transient waveform data of welding current and voltage in the welding circuit at a high sampling rate of 100 kHz. The waveform data directly reflects the static and dynamic characteristics of the arc, which contains information such as arc length and droplet transfer.

[0039] Synchronous triggering mechanism: To achieve strict synchronization of two kinds of heterogeneous data, the invention adopts a hardware trigger synchronization scheme. The upper industrial computer sends out a synchronization pulse signal, which simultaneously triggers the structured light vision sensor to perform an image acquisition and applies a precise timestamp to the data stream collected by the arc sensor. All data are marked with uniform timing information and transmitted to the upper computer for subsequent processing, ensuring the consistency of visual features and arc features on the time axis and laying the foundation for subsequent space-time registration.

[0040] S2: Constructing a multi-modal feature vector of the working trajectory: The heterogeneous sensor data collected in S1 is converted into a unified and information-enhanced trajectory feature description. The specific implementation process is as follows:

[0041] S201: Process the three-dimensional point cloud data and extract the three-dimensional spatial coordinate sequence of the weld centerline in the robot base coordinate system as the visual trajectory feature;

[0042] Point cloud preprocessing: The original three-dimensional point cloud from the structured light vision sensor is preprocessed. First, the voxel grid downsampling method is used to reduce the number of point clouds without losing shape features. Then, the statistical outlier removal filter is used to remove discrete noise points.

[0043] Work area segmentation: Based on the geometric features of the point cloud, the random sample consensus algorithm is used to fit the plane model of the workpiece surface and segment the point cloud cluster in the weld bevel area.

[0044] Centerline extraction: The segmented weld bevel point cloud cluster is sliced along the welding direction. For each slice section, the principal component analysis method is used to fit two bevel edges, and the center point of the angle point is calculated as the weld center point of the section. Connecting all the center points of the sections, the three-dimensional spatial coordinate sequence of the weld centerline in the vision sensor coordinate system is obtained.

[0045] Coordinate system conversion: The transformation matrix obtained through hand-eye calibration converts the weld centerline coordinate sequence from the vision sensor coordinate system to the robot base coordinate system.

[0046] The conversion formula is: .

[0047] Thus, the three-dimensional coordinate sequence of the weld centerline in the robot's base coordinate system, which serves as a visual trajectory feature, is obtained. .

[0048] S202: Process the transient waveform data, extract the feature quantity that characterizes the dynamic change of the arc length, and based on the preset arc length and welding torch posture mapping model, calculate the feature quantity as the lateral deviation compensation amount and height deviation compensation amount of the welding torch end.

[0049] Waveform preprocessing and feature extraction: Digital filtering is performed on the acquired high-speed welding current and voltage transient waveform data to remove high-frequency noise. Within one process cycle, the effective value of the voltage is calculated as a stable characteristic quantity characterizing the dynamic changes in the arc length. The effective voltage value typically increases as the arc length increases, and decreases as the arc length decreases.

[0050] Mapping Model Call and Solving: A mapping model between arc length and welding torch posture was established in advance through welding process experiments. This model is essentially a parameterized function or lookup table describing the relationship between welding torch height, lateral position, and arc voltage.

[0051] Height deviation mapping: Under the standard welding torch height, the height of the welding torch is finely adjusted, and the corresponding arc voltage value is recorded to establish a "voltage-height" mapping function. or its inverse function During real-time processing, the extracted voltage characteristics are substituted into this function to calculate the height deviation compensation amount relative to the standard height. .

[0052] Lateral Deviation Mapping: Under standard alignment, fine-tuning the lateral position of the welding torch causes a characteristic change in the arc voltage due to the altered arc shape. Based on this, a "voltage-lateral position" mapping function is established. or its inverse function During real-time processing, the lateral deviation compensation amount relative to the standard alignment position can be calculated. .

[0053] S203: Spatial registration is performed on the visual trajectory features, lateral deviation compensation amount and height deviation compensation amount, and they are fused to form an enhanced working trajectory feature vector;

[0054] Spatial registration: due to visual trajectory characteristics It is a macroscopic three-dimensional spatial path, while the electric arc is calculated... and This refers to the microscopic deviation of the welding torch tip relative to the desired pose. This needs to be addressed using a robot kinematics model. and Map it to the robot's base coordinate system and spatially align it with the visual trajectory feature points at the current welding torch position.

[0055] Vector fusion: for a given processing cycle, construct the registered data into an enhanced working trajectory feature vector :

[0056]

[0057] where, is the coordinate of the weld center point perceived by vision at current time, and are the lateral and height deviation compensation quantities perceived by arc. The vector contains both the macro-geometric information of the weld and the micro-alignment and stability information of the welding process.

[0058] S3: based on the enhanced working trajectory feature vector, calculate the real-time confidence of the vision and arc sensing modalities respectively; where the vision confidence is quantified based on the point cloud density and noise rate of the three-dimensional point cloud data, and the arc confidence is quantified based on the signal-to-noise ratio of the transient waveform data; the specific implementation process is as follows:

[0059] S301: calculation of vision sensing modality confidence

[0060] The vision confidence is determined by two key indicators: point cloud density and noise rate .

[0061] Calculation of point cloud density : draw an area of interest near the weld centerline extracted in S201. Calculate the number of point clouds in a unit volume of 1 cm³ in this area as the point cloud density . Its calculation formula is: ;

[0062] where, is the number of point clouds in the area of interest, is the volume of the area. The higher the point cloud density, the richer the details of the vision perception, and the more reliable the feature extraction.

[0063] Calculation of noise rate : the noise rate is estimated by counting the proportion of the number of outliers filtered out after preprocessing to the total number of point clouds. Its calculation formula is: ;

[0064] where, is the number of outliers filtered out, is the total number of point clouds before preprocessing. The higher the noise rate, the more serious the interference of welding spatter and smoke on the sensor, and the lower the data reliability.

[0065] vision confidence​ The fusion calculation uses a weighted geometric mean to merge the two indicators into a single confidence value, which ranges from [0,1].

[0066] in:

[0067] It is the maximum point cloud density calibrated under ideal conditions;

[0068] It is the acceptable maximum noise rate threshold;

[0069] and It is a weighting coefficient, and ; , It focuses more on the contribution of density.

[0070] It is important to further clarify that point cloud density is a fundamental and decisive factor for the effectiveness of visual perception. In welding scenarios, sufficiently high point cloud density is an absolute prerequisite for accurately extracting the geometric features of the weld bevel. If the point cloud is too sparse, even with a low noise rate, the 3D morphology of the weld cannot be reliably reconstructed, leading to feature extraction failure or significant errors. Therefore, density is of greater importance in evaluating the overall reliability of a vision system. ;

[0071] Compared to insufficient density, noise can be effectively filtered through post-processing algorithms to a certain extent. Although a high noise rate indicates severe interference, as long as the underlying point cloud density is sufficient, the algorithm still has a good chance of preserving effective structural information through intelligent filtering. Therefore, its importance is secondary to density. .

[0072] S302: Arc sensing modal confidence level Calculation

[0073] Arc confidence is primarily determined by the signal-to-noise ratio of the transient waveform data it acquires. Decide.

[0074] Signal-to-noise ratio Calculation: The transient waveform data of welding current within one cycle collected in S202. The signal-to-noise ratio is calculated using the following formula: ;

[0075] in, It is the signal power, which is obtained by summing the energy of the main harmonic components after performing a Fourier transform on the waveform data. It is the noise power, which is obtained by calculating the variance of the residual signal after removing the main harmonic components.

[0076] Arc confidence Mapping calculation: the calculated SNR (in dB) is mapped to the confidence interval [0, 1] through a sigmoid function.

[0077] Wherein:

[0078] is the set SNR reference value, for example, 15 dB can be taken.

[0079] is the steepness factor of the curve, used to control the sensitivity of the mapping, for example, 0.5 can be taken.

[0080] Through this step, the system can evaluate the reliability of the visual and arc sensors in the current working condition in real time and quantitatively. When welding spatter and smoke cause the quality of the visual point cloud to decrease, will automatically decrease; when the arc is unstable and the interference is serious, it will also decrease accordingly.

[0081] S4: Convert the quantitative confidence calculated in S3 into the actual available sensor weight coefficient in the control system. The specific implementation preferably adopts an adaptive weight allocator based on multi-modal confidence, and the internal logic is realized through the following sub-steps:

[0082] S401: Weight base value calculation: first, calculate the initial base value of the weight according to the confidence. The calculation formula of the visual weight base value and the arc weight base value is as follows:

[0083] ;

[0084] ;

[0085] Wherein, and are the real-time confidence output by S3. This calculation ensures that when the confidence of both is normal, the weight is proportional to its reliability.

[0086] S402: Strategic adjustment based on threshold (execute the rules): this is the key link to embody the inventiveness of the present application. The system presets the first threshold (0.4) and the second threshold (0.3), and performs the following condition judgment and weight adjustment:

[0087] Condition one: when When this condition is met, it indicates that the visual sensor is severely disturbed (e.g. splashes, smoke); at this time, the system raises the arc weight coefficient, making it dominant in the height direction of the correction.

[0088] The implementation is: on the basis of the weight value, a reinforcement function is applied. Specifically, the final arc weight coefficient . Wherein, is 0.7, ensuring that the arc sensor dominates in decision-making. At the same time, the visual weight is inhibited accordingly, .

[0089] Condition two: when , this state indicates that the arc sensor signal is unreliable (e.g. arc length fluctuates sharply, arc striking fails). At this time, the system raises the visual weight coefficient, making it dominant in three-dimensional space trajectory tracking.

[0090] The implementation is: the final visual weight coefficient . Wherein, is 0.8, ensuring that the visual sensor dominates in macro trajectory planning. At the same time, .

[0091] Condition three: when the above two conditions are not met, the system maintains the weight calculated by the basic value, that is , , achieving balanced fusion.

[0092] It needs to be further explained that the first threshold is 0.4. Setting the threshold to 0.4 is a forward-looking and preventive strategy. When the visual confidence drops to 0.4, it indicates that its reliability has decreased significantly but not completely failed;

[0093] The second threshold is 0.3. Setting the threshold to 0.3 is a robustness strategy against transient interference. It ensures that the system will not easily and completely discard the arc information due to short-term and slight fluctuations in the arc signal; only when the arc confidence continues to decrease to 0.3 or below, does the system determine that the arc sensor is indeed unreliable, thus decisively switching to the visual-dominated mode.

[0094] It needs to be further explained that is 0.7, setting the arc weight to 0.7 on the one hand ensures that the arc sensor has absolute dominance in its most proficient height control, enabling rapid and stable welding process; on the other hand, it retains 30% of the visual weight, which can prevent the robot trajectory from deviating significantly due to the complete loss of visual macro guidance, thus playing the role of "safety redundancy";

[0095] For 0.8, the visual weight is set to 0.8, giving the visual sensor absolute control over trajectory tracking, ensuring that the robot's movement basis is correct and safe; the remaining 20% weight is left to the arc, which is used to smoothly reintroduce its correction amount in the early stages of arc signal recovery, avoiding sudden changes in control commands.

[0096] S403: Weight output: After the above adjustment, the final normalized visual weight coefficient is output and the arc weight coefficient , and ensure .

[0097] S5: The execution link of converting the decision into specific robot action, the core of which is to implement the intelligent weight distribution scheme output by S4 into the trajectory correction of each control cycle through robot kinematics. The implementation is described in detail as follows:

[0098] S501: Determine the reference trajectory and the desired pose: the robot controller pre-stores the reference welding trajectory based on offline programming or initial teaching. The trajectory is composed of a series of end effector (welding torch) pose points in the robot base coordinate system, where is the position, is the attitude based on Euler angles or angles.

[0099] S502: Calculate the independent pose adjustment amount of each sensing modality:

[0100] Visual trajectory feature determined pose adjustment amount : Get the visual trajectory feature point at the current time from S201 . The desired pose adjustment amount of the visual modality mainly acts on the position, which is calculated as follows:

[0101] ;

[0102] For the attitude, the visual modality usually does not provide a direct adjustment amount, so . Therefore .

[0103] Arc deviation compensation amount determined pose adjustment amount : Get the transverse deviation compensation amount and the height deviation compensation amount of the welding torch end from S202. The adjustment amount of the arc modality mainly acts on the specific direction of the welding torch:

[0104] Height direction (Z direction): ;

[0105] Transverse (perpendicular to the welding direction, such as Y direction): ;

[0106] Welding direction (X direction): , the arc usually does not provide correction of the forward direction;

[0107] Therefore, the position adjustment amount , this coordinate is expressed in the welding gun tool coordinate system.

[0108] In addition, in order to maintain the optimal welding posture, the arc information is sometimes also used to fine-tune the posture of the welding gun, such as a slight rotation around the welding gun axis (Z axis of TCP) to improve the weld formation, so .

[0109] Finally, the and need to be converted to the robot base coordinate system through the transformation from the robot tool coordinate system to the base coordinate system, to obtain .

[0110] S503: Weighted fusion based on weights: the visual weight coefficient and the arc weight coefficient are used to perform weighted fusion on the above two position adjustment amounts to generate a six-degree-of-freedom comprehensive trajectory correction amount :

[0111] This formula separately performs weighted calculation on each component of the position and posture. For example, for the correction amount in the height direction (Z direction):

[0112] Through this operation, the collaborative control strategy of "visual dominant macro trajectory and arc dominant height and posture fine-tuning" is realized. When the visual confidence is low and is increased, and the proportion in the comprehensive correction amount increases, realizing the correction of the key process parameters dominated by the arc.

[0113] It needs to be further explained that in this experiment, part of the basic scene parameters are as follows:

[0114] The position adjustment amount determined by the visual trajectory feature and the position adjustment amount determined by the arc deviation compensation amount are not modified in the Y and Z axes and the modification angle is 0. The six-degree-of-freedom comprehensive trajectory correction amount is obtained, and the table is as follows:

[0115]

[0116] The data in the table shows that when the visual weight coefficient increases, the fusion trajectory correction amount is closer to the pose adjustment amount determined by the visual trajectory features, and the dominant role of the visual sensor in trajectory correction decision-making is enhanced; conversely, when the arc weight coefficient increases, the fusion result is more inclined to the arc deviation compensation amount, and the dominant role of the arc sensor in fine-tuning process parameters is highlighted.

[0117] S504: Generate the final desired pose: Apply the comprehensive trajectory correction to the reference trajectory to obtain the final desired pose that the robot's end effector needs to reach in the current control cycle. :

[0118]

[0119] S6: The six-degree-of-freedom integrated trajectory correction amount generated in S5, which includes intelligent fusion decision-making. This is translated into the actual movement of the robot body, ultimately achieving precise and adaptive welding trajectory tracking. The specific implementation process is as follows:

[0120] Input and integration of trajectory correction values: calculated by the host computer (industrial control computer). The data is transmitted in real time to the robot controller via an Ethernet communication interface. The controller contains a real-time trajectory interpolator. This interpolator does not simply replace the original path with a new target point, but rather... As a dynamic perturbation vector, it is related to the currently executing reference welding trajectory. The points are superimposed to instantly generate a corrected desired trajectory point. ,Right now: ;

[0121] This process is repeated every control cycle (typically a few milliseconds) to achieve continuous, smooth, and dynamic correction of the baseline trajectory.

[0122] Inverse kinematics and joint space control: The robot controller receives the pose space (Cartesian space) Then, its inverse kinematics algorithm is invoked. This algorithm, based on the robot's model parameters (such as DH parameters), quickly calculates the target angles that each joint of the robot needs to rotate to achieve the desired pose. .

[0123] Servo drive and execution: The controller will calculate the target angles of each joint. Compared with the actual joint angle fed back by the joint encoder The joint angle error is obtained by comparison. This error signal is amplified and processed by the servo drive system inside the controller to generate corresponding control current, which drives the servo motors of each joint to move.

[0124] Implementation of online adaptive trajectory correction: servo motor generates accurate torque and displacement according to control instruction, through transmission mechanism such as reducer, drives each connecting rod of robot to move, finally makes end effector (welding gun) track the above-mentioned expected trajectory which is dynamically corrected in real time ;

[0125] Since the whole process from S1 to S5 is executed online and in real time during the operation, the robot can continuously adjust its motion trajectory according to the actual weld condition and welding state. For example, when the vision sensor finds that the weld is offset, the trajectory will be immediately corrected; when the arc sensor detects that the welding gun is too high, the robot will immediately lower the height, so as to always ensure that the welding gun is in the optimal position and posture relative to the weld.

[0126] Secondly: the drawings of the disclosed embodiments only involve the structures involved in the disclosed embodiments, other structures can refer to the usual design, and in the case of no conflict, the same embodiment and different embodiments of the present application can be combined with each other;

[0127] Finally: the above only describes the preferred embodiments of the present application and is not used to limit the present application, any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A welding robot trajectory adaptive correction method based on multi-modal data fusion, characterized in that, Comprise: S1: real-time synchronous acquisition of multi-modal sensing data in the working process, including three-dimensional point cloud data of the working area acquired by a visual sensor, and welding current and voltage transient waveform data acquired by an arc sensor; S2: constructing a multi-modal feature vector of the working trajectory: S201: processing the three-dimensional point cloud data, extracting a three-dimensional space coordinate sequence of the weld centerline under the robot base coordinate system as the visual trajectory feature; S202: processing the transient waveform data, extracting a feature quantity representing the dynamic change of the arc length, and based on a preset arc length and welding torch pose mapping model, calculating the feature quantity into a lateral deviation compensation quantity and a height deviation compensation quantity of the welding torch end; S203: spatially registering the visual trajectory feature, the lateral deviation compensation quantity and the height deviation compensation quantity, and fusing to form an enhanced working trajectory feature vector; S3: based on the enhanced working trajectory feature vector, calculating the real-time confidence of the visual and arc sensing modalities respectively; wherein the visual confidence is quantified based on the point cloud density and the noise point rate of the three-dimensional point cloud data, and the arc confidence is quantified based on the signal-to-noise ratio of the transient waveform data; S4: inputting the real-time confidence into a preset weight distribution strategy model to dynamically calculate the visual weight coefficient and the arc weight coefficient; the weight distribution strategy model is configured to perform the following rules: when the visual confidence is lower than a first threshold, the arc weight coefficient is increased to dominate the height direction correction; when the arc confidence is lower than a second threshold, the visual weight coefficient is increased to dominate the three-dimensional space trajectory tracking; S5: using the calculated weight coefficients to weight and fuse the pose adjustment quantity determined by the visual trajectory feature and the pose adjustment quantity determined by the arc deviation compensation quantity, to generate a six-degree-of-freedom comprehensive trajectory correction quantity of the robot end effector; S6: inputting the comprehensive trajectory correction quantity into the robot controller to drive the welding robot to perform online adaptive trajectory correction.

2. The welding robot trajectory adaptive correction method based on multi-modal data fusion according to claim 1, characterized in that, The multi-modal sensing data comprises: through a hardware trigger synchronization scheme, a synchronization pulse signal is sent by an upper industrial computer, which simultaneously triggers the structure light visual sensor to perform an image acquisition, and timestamps the data stream collected by the arc sensor.

3. The welding robot trajectory adaptive correction method based on multi-modal data fusion according to claim 1, characterized in that, The spatial registration comprises: through a robot kinematics model, the lateral deviation compensation quantity and the height deviation compensation quantity calculated by the arc are mapped to the robot base coordinate system, and are spatially aligned with the visual trajectory feature points at the current welding torch position, to construct an enhanced working trajectory feature vector containing both weld macro-geometric information and weld process micro-alignment and stability information.

4. The welding robot trajectory adaptive correction method based on multi-modal data fusion according to claim 1, characterized in that, The real-time confidence comprises: visual confidence and arc confidence; Wherein, the visual confidence is quantified based on the point cloud density and the noise point rate of the three-dimensional point cloud data, and the specific calculation process comprises: calculating the point cloud density in the region of interest near the weld centerline; calculating the proportion of the number of outliers removed in the preprocessing process to the total number of point clouds as the noise point rate; The point cloud density and the noise rate are fused into a visual confidence value in a range of [0, 1] by using a weighted geometric mean, wherein the weight of the point cloud density index is greater than the weight of the noise rate index; The arc confidence is quantified based on a signal-to-noise ratio of the transient waveform data, and the specific calculation process includes: Fourier transform is performed on the welding current transient waveform data in one cycle to calculate the signal power and the noise power, and then the signal-to-noise ratio is obtained; The calculated signal-to-noise ratio value is mapped to an arc confidence value in a range of [0, 1] by using an S-shaped function.

5. The welding robot trajectory adaptive correction method based on multi-modal data fusion according to claim 1, characterized in that, The specific rules performed by the redistribution strategy model include: When the visual confidence is lower than a first threshold value, the arc weight coefficient is raised to a dominant value, and the visual weight coefficient is correspondingly suppressed; When the arc confidence is lower than a second threshold value, the visual weight coefficient is raised to a dominant value, and the arc weight coefficient is correspondingly suppressed; When neither of the above two conditions is met, the visual weight coefficient and the arc weight coefficient are distributed in proportion to their confidences.

6. The welding robot trajectory adaptive correction method based on multi-modal data fusion according to claim 1, characterized in that, The six-degree-of-freedom comprehensive trajectory correction amount includes each component of the position and the attitude of the pose adjustment amount determined by the visual trajectory feature and the pose adjustment amount determined by the arc deviation compensation amount, which are respectively weighted and calculated by the calculated visual weight coefficient and the arc weight coefficient.

7. The welding robot trajectory adaptive correction method based on multi-modal data fusion according to claim 1, characterized in that, The visual trajectory feature includes: after preprocessing and work area segmentation of the point cloud, slicing processing is performed along the welding direction, the principal component analysis method is used to fit the bevel edge line and calculate the center point for each slice section to obtain the weld center line coordinate sequence.

8. The welding robot trajectory adaptive correction method based on multi-modal data fusion according to claim 1, characterized in that, The adaptive trajectory correction includes: the comprehensive trajectory correction amount is sent to the robot controller through the Ethernet communication interface, the trajectory real-time interpolator of the controller superimposes it with the reference welding trajectory, and the corrected trajectory is executed through inverse kinematics calculation and servo drive system.

9. The welding robot trajectory adaptive correction method based on multi-modal data fusion according to claim 1, characterized in that, The weight coefficient includes: the visual weight coefficient and the arc weight coefficient; wherein the visual weight basic value is calculated according to the proportion of the visual confidence in the sum of the visual confidence and the arc confidence, and the arc weight basic value is calculated according to the proportion of the arc confidence in the sum of the visual confidence and the arc confidence.

Images