Welding robot motion path optimization planning method based on multi-sensor fusion

Abstract

The invention belongs to the technical field of robot welding, and discloses a welding robot motion path optimization planning method based on multi-sensor fusion. Comprising the steps that sensor data are obtained, and the sensor data comprise a welding seam three-dimensional point cloud, a molten pool image, acceleration and angular velocity of the tail end of a welding gun in three axial directions, welding current and welding voltage; converting the sensor data into a unified coordinate system to obtain converted sensor data; performing filtering processing on the converted three-dimensional point cloud of the welding seam to obtain a filtering point cloud, and performing feature extraction on the filtering point cloud based on an RANSAC (Random Sample Consensus) algorithm to obtain a center line feature of the welding seam; in combination with the converted three-dimensional point cloud of the welding seam and the characteristics of the center line of the welding seam, an initial welding path is obtained through planning based on an improved A * algorithm; intelligent and precise adjustment of welding parameters is achieved, the stability of welding quality is guaranteed, and the performance of motion path planning and welding operation of the welding robot is improved in an all-around mode.

Description

Technical Field

[0001] The present invention relates to the technical field of robotic welding, and more specifically, to a method for optimizing the motion path planning of a welding robot based on multi-sensor fusion. Background Art

[0002] In modern manufacturing, welding, as a key joining process, is widely used in many fields. With the continuous improvement of the requirements for product quality and production efficiency in the manufacturing industry, welding robots have gradually become an important tool for welding operations.

[0003] Chinese Patent Application with Publication No. CN117300301A discloses a welding robot seam tracking system and method based on a monocular line laser: Step 1, the welding robot carries a vision sensor to collect seam images; Step 2, the image processing module processes the seam images, and obtains the world coordinates of the seam feature points through image denoising, extraction of the center line of the laser stripe, identification of seam feature points, and three-dimensional reconstruction of the coordinates of the seam feature points, and at the same time determines the type of seam feature points; Step 3, the image processing module stores the coordinates of the seam feature points into the trajectory planning module; Step 4, the trajectory planning module establishes a local coordinate system of the seam pose based on the coordinates of the seam feature points in the front and rear two frames of images, obtains the welding pose of the robot welding torch, and generates a welding path; Step 5, the welding process matching module matches welding process parameters based on the required welding process; Step 6, the seam tracking system stores the welding path information, the type of seam feature points, and the welding process parameters into the upper computer buffer area. The upper computer communicates with the robot controller through TCP / IP, and sends the welding pose, the type of seam feature points, and the welding process parameters to the robot controller; Step 7, the robot controller parses the data of the upper computer. The robot adjusts the torch posture according to the welding pose data. The robot determines the swing arc shape and welding speed according to the welding process parameters. The welding robot performs arc starting, welding, or arc extinguishing operations according to the data type of the seam feature points. The robot controller sends the welding process parameters to the digital intelligent welding machine through DevicetNet to control the welding current and voltage. This invention realizes the real-time tracking of the seam and the real-time adjustment of the welding posture.

[0004] Although the above method can meet most scenarios, through research and practical application of the above method and the existing technologies, it is found that the above method and the existing technologies have at least the following partial defects:

[0005] The monocular line laser sensor has data limitations under the dynamic changes of seam features, which is not sufficient to support the accurate judgment and adjustment of the welding process; only establishing a local coordinate system of the seam pose based on the coordinates of the seam feature points in the front and rear two frames of images to generate the welding path and adjust the torch posture, it is difficult to achieve accurate path planning and real-time correction when facing seams with complex shapes or large deviations during the welding process.

[0006] In view of this, the present invention proposes a method for optimizing and planning the motion path of a welding robot based on multi-sensor fusion to solve the above problems. Summary of the Invention

[0007] In order to overcome the above defects of the prior art and achieve the above object, the present invention provides the following technical solution: A method for optimizing and planning the motion path of a welding robot based on multi-sensor fusion, comprising the following steps:

[0008] Collect sensor data, where the sensor data includes three-dimensional weld point cloud, molten pool image, acceleration and angular velocity of the welding torch end in three axes, welding current, and welding voltage;

[0009] Convert the sensor data to a unified coordinate system to obtain the converted sensor data;

[0010] Perform filtering processing on the converted three-dimensional weld point cloud to obtain the filtered point cloud, and perform feature extraction on the filtered point cloud based on the RANSAC algorithm to obtain the weld centerline feature;

[0011] Combine the converted three-dimensional weld point cloud and the weld centerline feature, and plan an initial welding path based on the improved A* algorithm, and perform smoothing processing on the initial welding path to obtain a smooth welding path;

[0012] Fuse the weld centerline feature, the converted acceleration and angular velocity of the welding torch end in three axes through an extended Kalman filter to obtain the welding torch pose feature;

[0013] During the welding process of the welding robot according to the smooth welding path, calculate through the weld deviation prediction model in combination with the welding torch pose feature to obtain the predicted weld deviation, correct the welding path in the same layer of welding space based on the predicted weld deviation, and the welding robot welds based on the corrected welding path in the same layer of welding space;

[0014] Use the converted molten pool image as the input of the convolutional neural network to obtain the segmented molten pool contour, and analyze the segmented molten pool contour to obtain the molten pool feature;

[0015] Fuse the molten pool feature, the converted welding current, and the converted welding voltage through attention convolution to obtain the fusion feature; compare the fusion feature with the data stored in the preset process knowledge base, and adaptively adjust the welding parameters of the welding passes in different layers; calculate the optimal lap position of the next layer through the spiral scanning algorithm for the corrected welding path in the same layer of welding space, and correct the welding paths in the welding spaces of different layers; use the corrected welding paths in the welding spaces of different layers as the optimized motion path of the welding robot.

[0016] Further, the method for obtaining the initial welding path includes:

[0017] Step 1: Obtain the welding space of the robot based on the three-dimensional point cloud of the weld seam, divide the welding space according to the preset layer height to obtain N layers of welding space, and traverse based on the boundaries of the N layers of welding space to divide the N layers of welding space into discrete grid nodes, where each node represents a welding torch position;

[0018] Step 2: Select the starting and ending positions of the weld seam according to the direction of the weld seam centerline feature, and select the starting point and ending point of each layer of welding space in the grid nodes. Among them, the ending point of the upper layer of welding space is the same as the starting point position of the lower layer of welding space; for each layer of weld seam space, repeat Steps 3 - 9;

[0019] Step 3: Preset and initialize the open list and the closed list: The open list is used to store the nodes to be evaluated, and the initialized open list contains the starting point; the closed list is used to store the evaluated nodes, and the initialized closed list is empty;

[0020] Step 4: Define the heuristic function from the current node to the end point;

[0021] Step 5: Select the node with the minimum evaluation function value from the open list through the evaluation function regarding the heuristic function ;

[0022] Step 6: Move the node from the open list to the closed list;

[0023] Step 7: Check whether the node is the end point: If the node is the end point, then a path from the starting point to the end point is found and the algorithm ends; otherwise, continue with Step 8;

[0024] Step 8: Generate all adjacent nodes of the node , and the adjacent nodes represent the reachable nodes in the up, down, left, right, and diagonal directions in the grid of the node ;

[0025] Step 9: When the end point is found, then starting from the end point, generate the welding path from the starting point to the end point of the corresponding layer of weld seam space by backtracking the parent node;

[0026] Step 10: Connect the welding paths from the starting point to the end point of each layer of weld seam space in sequence to obtain the initial welding path.

[0027] Further, in the said Step 8, for each adjacent node , perform A - D:

[0028] A. If the node is in the closed list, skip the node ;

[0029] B. Calculate the actual cost from the starting point through the node to the node based on the evaluation function regarding the heuristic function ;

[0030] C. If the node is not in the open list, add the node to the open list, and obtain the corresponding actual cost, heuristic function value, and evaluation function value of the node . Meanwhile, record the parent node of the node as ;

[0031] D. If the node is already in the open list, compare the currently calculated value with the existing actual cost in the open list; if the currently calculated actual cost is smaller, then update the actual cost, evaluation function value, and parent node of the node in the open list to .

[0032] Furthermore, the method for obtaining a smooth welding path includes:

[0033] Smoothing the path node coordinates through the B-spline curve formula; connecting all the smoothed path node coordinates in sequence to obtain a smooth welding path.

[0034] Furthermore, the method for obtaining the weld prediction deviation during the welding process includes:

[0035] Establish a weld deviation prediction model based on the current layer height, reference layer height, actual groove width, desired groove width, and welding torch attitude deviation value, and calculate the weld prediction deviation based on the weld deviation prediction model; wherein, the welding torch attitude deviation value is the difference between the welding torch pose feature and the preset welding torch pose feature.

[0036] Furthermore, the method for obtaining the optimal lap position of the next layer includes:

[0037] Expand outward through the helix equation regarding the helix radius, helix angle, and pitch to search for the optimal lap position of the next layer; search for the lap position that meets the welding quality requirements by adjusting the values of the helix radius, helix angle, and pitch, and use the lap position that meets the preset welding quality requirements as the optimal lap position of the next layer.

[0038] Furthermore, the molten pool characteristics include the molten pool length, molten pool width, and molten pool trailing angle; the method for obtaining the molten pool length includes:

[0039] Taking the molten pool contour as the input of the image segmentation model to obtain the probability that each pixel point belongs to the molten pool contour; for the th pixel point the probability of belonging to the molten pool contour perform binarization processing to obtain the binarization processing result of the th pixel point ; extract the pixel points with the value of 1 in the binarization processing result to obtain the molten pool contour after binarization processing; traverse the molten pool contour after binarization processing to obtain the boundary points of the molten pool contour, calculate the Euclidean distance between any two points on the molten pool contour as the contour distance, and select the maximum value of the contour distance as the molten pool length; is the number of pixel points in the molten pool contour;

[0040] The method for obtaining the molten pool width includes:

[0041] Obtain the two contour points corresponding to the contour length and , calculate the molten pool contour length direction vector formed by the two contour points and the modulus corresponding to the molten pool contour length direction vector; for any contour point , calculate the projection length of the vector obtained from the contour point and the contour point , calculate the vector perpendicular to the molten pool length equation based on the projection length; calculate the distance from the contour point to the straight line in the molten pool contour length direction based on the vector perpendicular to the molten pool length equation; select the maximum value of the distance from the contour point to the straight line in the molten pool contour length direction as the molten pool width;

[0042] The method for obtaining the trailing angle of the molten pool includes:

[0043] Obtain the preset welding direction, select W contour points at one end far from the welding starting point, perform fitting through an R-degree polynomial, calculate the coefficients of the polynomial by minimizing the sum of squared errors combined with the coordinates of the selected contour points, substitute the calculated coefficients of the polynomial into the R-degree polynomial to obtain the fitting curve equation, calculate the derivative equation of the fitting curve equation, and substitute the coordinates of the th contour point into the derivative equation to obtain the corresponding tangent slope , calculate the angle between the tangent direction of the th contour point and the positive direction of the horizontal axis, and calculate the trailing angle of the molten pool based on the angle;

[0044] Calculate the average value of the angles between the tangent directions of the selected W contour points and the welding direction as the trailing angle of the molten pool.

[0045] Furthermore, the method for obtaining the fusion feature includes:

[0046] Concatenate the molten pool length, molten pool width, and trailing angle of the molten pool as the visual feature vector , and concatenate the current and voltage as the arc feature vector , and map the visual feature vector and the arc feature vector to the same dimension to obtain the mapped visual feature vector and the mapped arc feature vector ; Calculate the attention weight based on the similarity score between the mapped visual feature vector and the mapped arc feature vector ; Calculate the fused feature based on the attention weight.

[0047] Furthermore, the method for obtaining the weld centerline feature includes:

[0048] Step a: Arbitrarily select three points in the filtered point cloud to form a plane to be measured, and construct the plane equation of the plane to be measured;

[0049] Step b: Calculate the distance from the th other point in the filtered point cloud to the plane to be measured;

[0050] Step c: Judge the relationship between the distances from all other points to the plane to be measured and the preset distance threshold. If the number of other points with distances less than the preset distance threshold exceeds the preset quantity threshold, it is judged that the points with distances less than the preset distance threshold belong to the same relevant plane;

[0051] Step d: Repeat steps a - c K times, and select the relevant plane containing the largest number of points in the filtered point cloud as the weld plane;

[0052] Step e: Transform the filtered point cloud contained in the weld plane from the base coordinate system to the parameter space through a transformation formula;

[0053] Step f: Substitute the coordinates of the points on the weld centerline in the filtered point cloud into the parameter equation, and solve the parameter equation by the least square method to obtain the solution result. Transform the solution result back to the unified coordinate system to obtain the position equation of the weld centerline. Substitute the preset value into the position equation of the weld centerline to obtain the set of weld centerline feature points. Calculate the feature vector formed by adjacent weld centerlines by the vector method, and concatenate all the feature vectors as the weld centerline feature.

[0054] Furthermore, the method for transforming the sensor data to the unified coordinate system includes:

[0055] Build a base coordinate system with the robot base as the origin to obtain the original coordinate system corresponding to the sensor data;

[0056] The joint motion of a robot with n joints is described using a homogeneous transformation matrix, and the transformation matrix from the base coordinate system to the original coordinate system is expressed as the product of the transformation matrices of each joint;

[0057] The coordinates of the sensor data in the original coordinate system are transformed to the base coordinate system through a conversion formula to obtain the transformed sensor data.

[0058] Furthermore, the method for obtaining the filtered point cloud includes:

[0059] For each point in the three-dimensional weld point cloud , where , and are respectively the abscissa value, ordinate value, and vertical coordinate value of the -th point in the three-dimensional weld point cloud in the base coordinate system; the filtered point is calculated through an adaptive Gaussian function; the set of all filtered points in the three-dimensional weld point cloud is used as the filtered point cloud. The filtered point ;

[0060] Furthermore, the method for obtaining the pose characteristics of the welding torch includes:

[0061] Define the system state equation;

[0062] The weld centerline feature, the accelerations and angular velocities of the welding torch end in three axes are spliced as the observation value corresponding to the current moment ;

[0063] Predict the state and covariance at the next moment according to the system state equation;

[0064] In the update step, update the state and covariance at the next moment according to the observation value to obtain the updated state estimate and the updated covariance, and use the updated state estimate as the pose characteristics of the welding torch.

[0065] The technical effects and advantages of the method for optimizing the motion path of a welding robot based on multi-sensor fusion in the present invention:

[0066] The present invention realizes the all-round and multi-dimensional precise perception of the welding process by collecting multi-source sensor data such as three-dimensional weld point cloud, molten pool image, acceleration and angular velocity of the welding torch end in three axial directions, welding current and voltage, etc., and converting them into a unified coordinate system, providing a basis for subsequent processing; filtering the converted three-dimensional weld point cloud and extracting the weld centerline features based on the RANSAC algorithm, planning and smoothing the initial welding path by combining the three-dimensional weld point cloud with the features using the improved A* algorithm, making the welding path planning more accurate and more in line with the actual welding requirements; using the extended Kalman filter to fuse the weld centerline features with the acceleration and angular velocity of the welding torch end to obtain the welding torch pose features, and combining these features during welding to correct the welding space paths of the same layer and different layers respectively through the weld deviation prediction model and the spiral scanning algorithm, effectively improving the real-time adaptability and accuracy of the welding path during the actual welding process; processing the converted molten pool image through a convolutional neural network to obtain molten pool features, fusing them with the welding current and voltage through attention convolution, and comparing with the preset process knowledge base to adaptively adjust the welding parameters, realizing the intelligent and precise adjustment of the welding parameters, ensuring the stability of the welding quality, and comprehensively improving the performance of the motion path planning and welding operation of the welding robot. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 It is a schematic flow chart of the method for optimizing and planning the motion path of a welding robot based on multi-sensor fusion according to the present invention;

[0068] Figure 2 It is a schematic flow chart of the method for obtaining the initial welding path according to the present invention;

[0069] Figure 3 It is a schematic flow chart of the method for self-updating parameters according to the present invention;

[0070] Figure 4 It is a schematic flow chart of the method for coping with sudden interference according to the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0071] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0072] Embodiment 1

[0073] Please refer to Figure 1 As shown, the method for optimizing and planning the motion path of a welding robot based on multi-sensor fusion in this embodiment includes the following steps:

[0074] Sensor data is collected, including three-dimensional weld point cloud, molten pool image, accelerations and angular velocities of the torch end in three axes, welding current, and welding voltage. The three-dimensional weld point cloud can be collected by a depth camera. Collecting the three-dimensional weld point cloud can accurately obtain the spatial position and shape information of the weld, providing a basis for planning the welding path. The molten pool image is collected by a high-definition camera. Collecting the molten pool image can intuitively reflect the state of the welding molten pool, helping to adjust the welding parameters and path according to the changes in the molten pool. The accelerations of the torch end in three axes are measured by an accelerometer, and the angular velocities of the torch end in three axes are measured by a gyroscope. Collecting the accelerations and angular velocities of the torch end in three axes can grasp the motion posture and dynamic changes of the torch in real time, ensuring the accuracy and stability of the welding path. The welding current and voltage can be collected through an ammeter and a voltmeter. Collecting the welding current and voltage can understand the energy input situation of welding, facilitating the optimization of welding parameters according to the actual welding state, and indirectly optimizing the motion path of the welding robot to ensure the welding quality.

[0075] Based on the motion model of the welding robot, the sensor data is converted into a unified coordinate system to obtain the converted sensor data.

[0076] The method for converting the sensor data into a unified coordinate system includes:

[0077] Taking the robot base as the origin, a base coordinate system is established to obtain the original coordinate system corresponding to the sensor data.

[0078] Using the homogeneous transformation matrix to describe the joint motion of a robot with n joints, where is the rotation matrix, used to represent the rotation relationship of the coordinate system; is the translation vector, used to describe the translation relationship of the coordinate system; The transformation matrix from the base coordinate system to the original coordinate system is expressed as the product of each joint transformation matrix, such as ; is the transformation matrix from the base coordinate system to the original coordinate system; , and are the first, second, and th joint transformation matrices.

[0079] Through the conversion formula, the coordinates of the sensor data in the original coordinate system are converted into the base coordinate system, such as , where is the converted sensor data; is the sensor data before conversion, and the converted sensor data is obtained.

[0080] The three-dimensional point cloud of the weld seam is processed by adaptive Gaussian filtering to obtain the filtered point cloud, and then the RANSAC algorithm is used to extract the features of the filtered point cloud to obtain the weld seam centerline features;

[0081] The method for obtaining the filtered point cloud includes:

[0082] For each point in the three-dimensional point cloud of the weld seam , where , and are respectively the abscissa value, ordinate value and vertical coordinate value of the th point in the three-dimensional point cloud of the weld seam in the base coordinate system; The filtered point of point is calculated by the adaptive Gaussian function: :

[0083] ;

[0084] where is the neighborhood point set of point ; is the weight between point and the neighborhood point , which is adaptively adjusted according to the distribution and distance of the neighborhood points; is the rd neighborhood point of point ; The set of all filtered points in the three-dimensional point cloud of the weld seam is used as the filtered point cloud.

[0085] The method for obtaining the weld seam centerline features includes:

[0086] Step a: Arbitrarily select three points in the filtered point cloud to form a plane to be measured, and establish the plane equation of the plane to be measured ; where , , and are all coefficients of the plane equation of the plane to be measured;

[0087] Step b: Calculate the distance from the th other point in the filtered point cloud to the plane to be measured :

[0088] ;

[0089] where , and are the abscissa value, ordinate value and vertical coordinate value of the th other point in the base coordinate system;

[0090] Step c: Determine the relationship between the distances from all other points to the plane to be measured and the preset distance threshold. If the number of other points with distances less than the preset distance threshold exceeds the preset quantity threshold, it is determined that the points with distances less than the preset distance threshold belong to the same relevant plane;

[0091] Step d: Repeat steps a - c for K times, and select the relevant plane containing the largest number of points in the filtered point cloud as the weld plane;

[0092] Step e: Transform the filtered point cloud contained in the weld plane from the base coordinate system to the parameter space through the transformation formula where, is a fixed point on the line; is the direction vector of the line; is the parameter;

[0093] Step f: Substitute the coordinates of the points located on the weld centerline in the point cloud into the parametric equation, solve the parametric equation by least - squares fitting to obtain the solution result, transform the solution result back to the unified coordinate system to obtain the position equation of the weld centerline, substitute the preset value into the position equation of the weld centerline to obtain the set of feature points of the weld centerline, calculate the feature vectors formed by adjacent weld centerlines by the vector method, and splice all the feature vectors as the weld centerline feature.

[0094] Combined with the transformed three - dimensional weld point cloud and the weld centerline feature, an initial welding path is planned based on the improved A* algorithm, and the initial welding path is smoothed to obtain a smooth welding path;

[0095] Referring to Figure 2 , the method for obtaining the initial welding path includes:

[0096] Step 1: Obtain the welding space of the robot according to the three - dimensional weld point cloud, divide the welding space according to the preset layer height to obtain N layers of welding space, and traverse based on the boundaries of the N layers of welding space to divide the N layers of welding space into discrete grid nodes, and each node represents a welding torch position;

[0097] Step 2: Select the starting and ending positions of the weld according to the direction of the weld centerline feature, select the starting point and ending point of each layer of welding space in the grid nodes, where the ending point of the upper - layer welding space is the same as the starting point position of the lower - layer welding space; for each layer of weld space, repeat steps 3 - 9;

[0098] Step 3: Preset and initialize the open list and the closed list: The open list is used to store the nodes to be evaluated, and the initialized open list contains the starting point; the closed list is used to store the evaluated nodes, and the initialized closed list is empty.

[0099] Step 4: Set the current node as , and the end point as , and define the heuristic function ; where is the straight-line distance from node to the end point , is the difference metric between the current welding direction and the direction pointing to the end point, and are weight coefficients, which can be adjusted according to actual welding requirements.

[0100] Step 5: Select the node with the minimum evaluation function value from the open list through the evaluation function with respect to the heuristic function, where is the actual cost from the start point to node ; is the heuristic function value corresponding to node .

[0101] Step 6: Move the node from the open list to the closed list.

[0102] Step 7: Check whether the node is the end point: If the node is the end point, a path from the start point to the end point is found and the algorithm ends; otherwise, continue to Step 8;

[0103] Step 8: Generate all adjacent nodes of node , and the adjacent node represents the reachable nodes in the up, down, left, right, and diagonal directions in the grid of node ; for each adjacent node , execute A - D:

[0104] A. If the node is in the closed list, skip the node ;

[0105] B. Calculate the actual cost from the start point through node to node based on the evaluation function with respect to the heuristic function, where is the movement cost from node to node , considering factors such as the movement distance and whether the welding posture needs to be changed. For example, the cost of horizontal or vertical movement is 1, the cost of diagonal movement is , and if a large change in the welding posture is required, the cost increases accordingly.

[0106] C. If the node is not in the open list, add the node to the open list, and obtain the corresponding , and values. Among them, is the heuristic function value corresponding to the node ; is the evaluation function value corresponding to the node ; At the same time, record that the parent node of the node is .

[0107] D. If the node is already in the open list, compare the currently calculated value with the existing value in the open list; if the currently calculated value is smaller, then update the and , values of the node in the open list and the parent node is

[0108] This step reflects the dynamic optimization of the path by the improved A* algorithm when considering the actual welding situation, ensuring that the found path is not only short in distance but also meets the welding process requirements.

[0109] Step 9. When the end point is found or the open list is empty, if the end point is found, start from the end point and generate the welding path from the starting point to the end point in the weld space of the corresponding layer by backtracking the parent node;

[0110] The method for obtaining the smooth welding path includes:

[0111] Smoothing the path node coordinates through the B-spline curve formula ; Among them, is the point on the B-spline curve, that is, the th path node coordinate after smoothing; is the th B-spline basis function for the th path node coordinate; is the th path node coordinate of the initial welding path; V is the number of path nodes; Connect all the smoothed path node coordinates in sequence to obtain the smooth welding path.

[0112] Fusing the weld centerline features, the accelerations and angular velocities of the end of the welding torch in three axial directions through an extended Kalman filter to obtain the pose features of the welding torch;

[0113] The method for obtaining the pose features of the welding torch includes:

[0114] Define the system state equation ; where, is the system state corresponding to the current moment ; is the state transition function; is the system state corresponding to the previous moment ; is the control input corresponding to the previous moment ; is the process noise;

[0115] Stitch together the weld centerline features, the accelerations and angular velocities of the end of the welding torch in three axial directions as the observed value corresponding to the current moment ; where, is the observation function; is the observation noise;

[0116] Predict the state at the next moment according to the system state equation ; where, is the optimal estimated system state at the moment based on all the observation data at the moment and before; Predict the covariance at the next moment ; where, is the state transition matrix; is the covariance matrix of the system state estimate at the moment based on all the observation data at the moment and before; is the transpose of the vector; is the process noise;

[0117] In the update step, update the state estimate according to the observed value:

[0118] ; where, is the Kalman gain, is the estimated value obtained by predicting the state at the moment ; Update the covariance according to the observed value ; where, is the identity matrix; is the observation matrix at the moment, is the prior covariance matrix, representing the degree of uncertainty; the updated state estimate obtained by updating is used as the torch pose feature.

[0119] During the welding process of the welding robot according to the smooth welding path, calculations are performed through the weld deviation prediction model in combination with the torch pose feature to obtain the predicted weld deviation. Based on the predicted weld deviation, the welding path in the same layer of the welding space is corrected. The welding robot adjusts the pose of the welding robot based on the corrected welding path in the same layer of the welding space in a PID control manner for welding;

[0120] The method for obtaining the predicted weld deviation during the welding process includes:

[0121] According to the current layer height 、the reference layer height 、the actual groove width 、the desired groove width and the torch attitude deviation value to establish a weld deviation prediction model , and calculate the predicted weld deviation based on the weld deviation prediction model; where is the predicted weld deviation; the torch attitude deviation value is the difference between the torch pose feature and the preset torch pose feature; 、 and are adaptive weight coefficients, which can be trained and adjusted according to the actual welding data by the least squares method or the gradient descent method.

[0122] Taking the molten pool image as the input of the convolutional neural network, obtaining the segmented molten pool contour, analyzing the segmented molten pool contour, and obtaining the molten pool features, where the molten pool features include the molten pool length, the molten pool width, and the molten pool trailing angle;

[0123] The training method of the convolutional neural network includes:

[0124] Pre-collect D groups of training data, where the training data includes molten pool images and the corresponding molten pool contours.

[0125] Taking the molten pool image as the input of the convolutional neural network and the molten pool contour as the output of the convolutional neural network, with the goal of minimizing the error between the output molten pool contour and the actual molten pool contour, optimizing the network parameters of the convolutional neural network through a nature-inspired optimization algorithm, obtaining the network parameters corresponding to minimizing the error between the molten pool contour output by the convolutional neural network and the actual molten pool contour, and constructing the convolutional neural network with the corresponding network parameters as the trained convolutional neural network.

[0126] The method for obtaining the molten pool length includes:

[0127] Take the molten pool contour as the input of the image segmentation model to obtain the probability that each pixel point belongs to the molten pool contour. The training method of the image segmentation model is similar to that of the convolutional neural network; for the th pixel point probability of belonging to the molten pool contour perform binarization , to obtain the binarization result of the th pixel point ; where, is the preset contour threshold; is the number of pixel points in the molten pool contour; extract the pixel points with the value of 1 in the binarization result to obtain the binarized molten pool contour; traverse the binarized molten pool contour to obtain the boundary points of the molten pool contour, calculate the Euclidean distance between any two points on the molten pool contour as the contour distance, and select the maximum value of the contour distance as the molten pool length. The method for obtaining the molten pool width includes:

[0128] Obtain the two contour points corresponding to the contour length

[0129] and and , calculate to obtain the molten pool contour length direction vector , and the modulus corresponding to the molten pool contour length direction vector; for any contour point , calculate the projection length of the vector obtained from the contour point and the contour point , is the unit vector corresponding to the molten pool contour length direction vector ; calculate the vector perpendicular to the molten pool length equation; calculate the distance from the contour point to the straight line in the molten pool contour length direction based on the vector perpendicular to the molten pool length equation; select the maximum value of the distance from the contour point to the straight line in the molten pool contour length direction as the molten pool width.

[0130] The method for obtaining the trailing angle of the molten pool includes:

[0131] Obtain the preset welding direction, select W contour points at one end far from the welding starting point, fit them with an R-degree polynomial, calculate the coefficients of the polynomial by minimizing the sum of squared errors combined with the coordinates of the selected contour points, substitute the calculated coefficients of the polynomial into the R-degree polynomial to obtain the fitting curve equation, calculate the derivative equation of the fitting curve equation, and substitute the coordinates of the th contour point into the derivative equation to obtain the corresponding tangent slope , calculate to obtain the The angle between the tangent direction of a contour point and the positive direction of the horizontal axis , calculate the trailing angle of the molten pool ; among them, ; is the angle between the welding direction and the positive direction of the horizontal axis.

[0132] Calculate the average value of the angles between the tangent directions of the selected W contour points and the welding direction as the trailing angle of the molten pool.

[0133] Fuse the molten pool features, the converted welding current, and the converted welding voltage through attention convolution to obtain fused features; compare the fused features with the data stored in the preset process knowledge base to adaptively adjust the welding parameters; calculate the welding path in the corrected welding space of the same layer through the spiral scanning algorithm to obtain the optimal lap position of the next layer, and correct the welding paths of the welding spaces of different layers; use the corrected welding paths of the welding spaces of different layers as the optimized motion path of the welding robot; the welding parameters include welding speed, welding current, and welding inclination.

[0134] The method for obtaining the fused features includes:

[0135] Concatenate the molten pool length, molten pool width, and trailing angle of the molten pool as the visual feature vector , concatenate the current and voltage as the arc feature vector , map the visual feature vector and the arc feature vector to the same dimension to obtain the mapped visual feature vector and the mapped arc feature vector ; calculate the obtained attention weight ; among them, is the similarity function; is the mapped visual feature vector and the mapped arc feature vector of the similarity score; is the mapped arc feature vector and the mapped visual feature vector of the similarity score; based on the attention weight calculate the obtained fused feature .

[0136] The method for adaptively adjusting the welding parameters includes:

[0137] The preset process knowledge base stores the optimal welding parameters corresponding to different intervals of fusion features. The fusion features are matched with the intervals corresponding to the fusion features to obtain a matching interval. The current welding parameters are compared with the optimal welding parameters corresponding to the matching interval. If the current welding parameters are different from the optimal welding parameters corresponding to the matching interval, the current welding parameters are adaptively adjusted to be the same as the optimal welding parameters corresponding to the matching interval.

[0138] The method for obtaining the optimal lap position of the next layer includes:

[0139] Expand outward through the spiral equation to search for the optimal lap position of the next layer. The spiral equation is as follows:

[0140] ;

[0141] ;

[0142] ;

[0143] Wherein, , and are the coordinates of the optimal lap position of the next layer in the unified coordinate system, that is, the corrected path node coordinates; , and are the path node coordinates in the unified coordinate system; is the spiral radius; is the spiral angle; is the pitch; By adjusting , and values, search for the lap position that meets the welding quality requirements, and take the lap position that meets the preset welding quality requirements as the optimal lap position of the next layer.

[0144] Embodiment 2

[0145] Please refer to Figure 3 shown. This embodiment provides a parameter self-update method applied to the motion path optimization planning method of a welding robot based on multi-sensor fusion, including the following steps:

[0146] Define the state space , and the state space includes welding speed, welding current, layer height, and weld prediction deviation;

[0147] Define the action space , and the action space includes the adjustment amount of welding parameters;

[0148] Define the reward function ; where, is the forming quality; is the welding efficiency (such as welding time); is the welding energy consumption; , and are weight coefficients, which can be adjusted according to actual requirements.

[0149] Initialize a Q-network and a target Q-network. The Q-network is used to select actions, and the target Q-network is used to calculate the target Q-value to stabilize the learning process;

[0150] During the welding process, let the welding robot interact with the environment and continuously collect sample data. Each time of interaction, the welding robot selects an action according to the current state through the Q-network. After executing the selected action, the environment returns a new state and a reward function value. Store the current state, the selected action, the reward function value, and the new state returned by the environment into the experience replay pool. When the data in the experience replay pool reaches a certain quantity, randomly sample a batch of data from it for training the Q-network. Update the parameters of the Q-network by minimizing the loss function, so that the Q-network can more accurately estimate the Q-value and guide the robot to select better actions.

[0151] After welding every 5 weld seams, update the welding parameter library according to the current welding data and the reward function value.

[0152] Regularly evaluate the trained model. By running the model in an actual welding scenario or a simulation environment, observe the indicators such as welding quality, efficiency, and energy consumption. According to the evaluation results, optimize and adjust the parameters, state space, action space, or reward function of the model.

[0153] Embodiment 3

[0154] Please refer to Figure 4 shown in the figure. This embodiment provides a method for coping with sudden disturbances in a welding robot motion path optimization and planning method based on multi-sensor fusion, including the following steps:

[0155] Calculate the change rate of the welding current by the ratio of the difference in welding current at different times to the corresponding time interval. When it is detected that the change rate of the welding current exceeds the preset current threshold, immediately trigger the emergency stop of the welding robot;

[0156] The welding robot records the break point position through a position sensor. According to the break point position and the welding process requirements, use the improved A* algorithm to generate a repair path.

[0157] As described above, it is only the specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, and all of them should be covered by the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the said claims.

[0158] Finally, the above are only the preferred embodiments of the present invention and are not used to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for optimizing the motion path planning of a welding robot based on multi-sensor fusion, characterized in that, It includes the following steps: Collect sensor data, which includes three-dimensional weld point cloud, molten pool image, acceleration and angular velocity of the welding torch end in three axes, welding current, and welding voltage; Convert the sensor data to a unified coordinate system to obtain the converted sensor data; Perform filtering on the converted three-dimensional weld point cloud to obtain the filtered point cloud, and extract features from the filtered point cloud based on the RANSAC algorithm to obtain the weld centerline features; Combine the converted three-dimensional weld point cloud and the weld centerline features, and plan an initial welding path based on the improved A* algorithm, and smooth the initial welding path to obtain a smooth welding path; Fuse the weld centerline features, the acceleration and angular velocity of the converted welding torch end in three axes through an extended Kalman filter to obtain the welding torch pose features; During the welding process of the welding robot according to the smooth welding path, calculate through the weld deviation prediction model in combination with the welding torch pose features to obtain the predicted weld deviation, and correct the welding path in the same layer of welding space based on the predicted weld deviation. The welding robot welds based on the corrected welding path in the same layer of welding space; Use the converted molten pool image as the input of the convolutional neural network to obtain the segmented molten pool contour, and analyze the segmented molten pool contour to obtain the molten pool features; Fuse the molten pool features, the converted welding current, and the converted welding voltage through attention convolution to obtain the fused features; compare the fused features with the data stored in the preset process knowledge base, and adaptively adjust the welding parameters of the welding passes in different layers; calculate the optimal lap position of the next layer through the spiral scanning algorithm for the corrected welding path in the same layer of welding space, and correct the welding paths in the welding spaces of different layers; use the corrected welding paths in the welding spaces of different layers as the optimized motion path of the welding robot.

2. The method for optimizing the motion path planning of a welding robot based on multi-sensor fusion according to claim 1, wherein, The method for obtaining the initial welding path includes: Step 1: Obtain the welding space of the robot according to the three-dimensional weld point cloud, divide the welding space according to the preset layer height to obtain N layers of welding space, and traverse based on the boundaries of the N layers of welding space to divide the N layers of welding space into discrete grid nodes, and each node represents a welding torch position; Step 2: Select the starting and ending positions of the weld according to the direction of the weld centerline features, and select the starting and ending points of each layer of welding space in the grid nodes. Among them, the ending point of the upper layer of welding space is the same as the starting point position of the lower layer of welding space; for each layer of weld space, repeat Steps 3 - 9; Step 3: Preset and initialize the open list and the closed list: The open list is used to store the nodes to be evaluated, and the initialized open list contains the starting point; the closed list is used to store the evaluated nodes, and the initialized closed list is empty; Step 4, define a heuristic function from the current node to the end point; Step 5: Select the node with the minimum evaluation function value from the open list through the evaluation function regarding the heuristic function ; Step 6. Move the node from the open list to the closed list; Step 7, Check the node Whether it is the end point: If the node is the end point, then find a path from the start point to the end point, and the algorithm ends; otherwise, continue with Step 8; Step 8, generate nodes of all adjacent nodes , where adjacent nodes represent reachable nodes in the up, down, left, right, and diagonal directions in the grid of node ; Step 9: When the ending point is found, start from the ending point, and generate the welding path from the starting point to the ending point of the corresponding layer of weld space by backtracking the parent node; Step 10: Connect the welding paths from the starting point to the ending point of each layer of weld space in sequence to obtain the initial welding path.

3. The method for optimizing the motion path planning of a welding robot based on multi-sensor fusion according to claim 2, wherein In step 8, for each adjacent node , perform A - D: A. If the node is in the closed list, skip the node ; B. Calculate the actual cost from the starting point through the node to reach the node ; ; C. If the node is not in the open list, add the node to the open list, and obtain the actual cost, heuristic function value, and evaluation function value corresponding to the node . At the same time, record that the parent node of the node is ; D. If the node is already in the open list, compare the currently calculated value with the actual cost already in the open list; If the currently calculated actual cost is smaller, then update the actual cost, evaluation function value, and parent node of the node in the open list to be .

4. The method for optimizing the motion path planning of a welding robot based on multi-sensor fusion according to claim 3, wherein, The method for obtaining the smooth welding path includes: Smoothing the path node coordinates through the B-spline curve formula; connecting all the smoothed path node coordinates in sequence to obtain a smoothed welding path.

5. The welding robot motion path optimization planning method based on multi-sensor fusion according to claim 4, characterized in that The method for obtaining the welding seam prediction deviation during the welding process includes: Establishing a welding seam deviation prediction model based on the current layer height, reference layer height, actual groove width, desired groove width, and welding torch pose deviation value, and calculating the welding seam prediction deviation based on the welding seam deviation prediction model; wherein, the welding torch pose deviation value is the difference between the welding torch pose feature and the preset welding torch pose feature.

6. The method for optimizing the motion path planning of a welding robot based on multi-sensor fusion according to claim 5, characterized in that The method for obtaining the optimal lap position of the next layer includes: Expanding outward through the helix equation regarding the helix radius, helix angle, and pitch to search for the optimal lap position of the next layer; adjusting the values of the helix radius, helix angle, and pitch to search for the lap position that meets the welding quality requirements, and taking the lap position that meets the preset welding quality requirements as the optimal lap position of the next layer.

7. The method for optimizing the motion path planning of a welding robot based on multi-sensor fusion according to claim 1, characterized in that, The molten pool features include the molten pool length, molten pool width, and molten pool trailing angle; The method for obtaining the molten pool length includes: Use the molten pool contour as the input of the image segmentation model to obtain the probability that each pixel point belongs to the molten pool contour; for the probability that the pixel point belongs to the molten pool contour, perform binarization to obtain the binarization result of the pixel point; extract the pixel points with a value of 1 in the binarization result to obtain the binarized molten pool contour; traverse the binarized molten pool contour to obtain the boundary points of the molten pool contour, calculate the Euclidean distance between any two points on the molten pool contour as the contour distance, and select the maximum value of the contour distance as the molten pool length; The method for obtaining the molten pool width includes: Obtain the two contour points corresponding to the contour length and , calculate the vector in the molten pool contour length direction formed by the two contour points and the modulus corresponding to the molten pool contour length direction vector; for any contour point , calculate the projection length of the vector obtained from the contour point and the contour point . Calculate the vector perpendicular to the molten pool length equation based on the projection length; calculate the distance from the contour point to the straight line in the molten pool contour length direction based on the vector perpendicular to the molten pool length equation; select the maximum value of the distance from the contour point to the straight line in the molten pool contour length direction as the molten pool width. The method for obtaining the molten pool trailing angle includes: Obtain a preset welding direction, select W contour points at one end far from the welding starting point, fit them with an R-degree polynomial, calculate the coefficients of the polynomial by minimizing the sum of squared errors combined with the coordinates of the selected contour points, substitute the calculated coefficients of the polynomial into the R-degree polynomial to obtain the fitting curve equation, calculate the derivative equation of the fitting curve equation, substitute the coordinates of the th contour point into the derivative equation to obtain the corresponding tangent slope , calculate the angle between the tangent direction of the th contour point and the positive direction of the horizontal axis, and calculate the trailing angle of the molten pool based on the angle; Calculating the average value of the angles between the tangent directions of the selected W contour points and the welding direction as the molten pool trailing angle.

8. The method for optimizing the motion path planning of a welding robot based on multi-sensor fusion according to claim 7, characterized in that, The method for obtaining the fusion feature includes: Concatenate the molten pool length, molten pool width, and trailing angle of the molten pool as the visual feature vector , and concatenate the current and voltage as the arc feature vector , map the visual feature vector and the arc feature vector to the same dimension to obtain the mapped visual feature vector and the mapped arc feature vector ; calculate the attention weight based on the similarity score between the mapped visual feature vector and the mapped arc feature vector ; calculate the fused feature based on the attention weight.

9. The method for optimizing the motion path planning of a welding robot based on multi-sensor fusion according to claim 1, wherein The method for obtaining the welding seam centerline feature includes: Step a: Arbitrarily select three points in the filtered point cloud to form a plane to be measured, and establish the plane equation of the plane to be measured; Step b, calculate the distance from the th other point in the filtered point cloud to the plane to be measured; Step c: Judge the relationship between the distances from all other points to the plane to be measured and the preset distance threshold. If the number of other points with distances less than the preset distance threshold exceeds the preset number threshold, then judge that the points with distances less than the preset distance threshold belong to the same relevant plane; Step d: Repeat steps a - c K times, and select the relevant plane containing the largest number of points in the filtered point cloud as the welding seam plane; Step e: Convert the filtered point cloud contained in the welding seam plane from the base coordinate system to the parameter space through the conversion formula; Step f: Substitute the coordinates of the points on the welding seam centerline in the filtered point cloud into the parameter equation, solve the parameter equation by least squares fitting to obtain the solution result, convert the solution result back to the unified coordinate system to obtain the position equation of the welding seam centerline, substitute the preset value into the position equation of the welding seam centerline to obtain the set of welding seam centerline feature points, calculate the feature vectors formed by adjacent welding seam centerlines by the vector method, and splice all the feature vectors as the welding seam centerline feature.

10. The method for optimizing the motion path planning of a welding robot based on multi-sensor fusion according to claim 1, wherein, The method for converting sensor data to the unified coordinate system includes: Establishing a base coordinate system with the robot base as the origin to obtain the original coordinate system corresponding to the sensor data; Using the homogeneous transformation matrix to describe the joint motion of a robot with n joints, and representing the transformation matrix from the base coordinate system to the original coordinate system as the product of each joint transformation matrix; Converting the coordinates of the sensor data in the original coordinate system to the base coordinate system through the conversion formula to obtain the converted sensor data.

11. The method for optimizing the motion path planning of a welding robot based on multi-sensor fusion according to claim 1, wherein, The method for obtaining the filtered point cloud includes: For each point in the three-dimensional point cloud of the weld seam , where , and are the abscissa value, ordinate value, and vertical coordinate value of the point of the three-dimensional point cloud of the th weld seam in the base coordinate system respectively; calculate the filtered point of point through the adaptive Gaussian function; take the set of all points of the three-dimensional point cloud of the weld seam after filtering as the filtered point cloud.

12. The method for optimizing the motion path planning of a welding robot based on multi-sensor fusion according to claim 1, wherein, The method for obtaining the welding torch pose feature includes: Defining the system state equation; Stitch the weld centerline feature, the accelerations and angular velocities of the torch tip in three axial directions as the observation value corresponding to the current moment ; Predicting the state and covariance of the next moment according to the system state equation; In the update step, the state and covariance at the next moment are updated based on the observed values to obtain the updated state estimate and the updated covariance, and the updated state estimate is used as the torch pose feature.

Images