Welding robot motion path optimization planning method based on multi-sensor fusion

Abstract

The present invention belongs to the technical field of robot welding and discloses a method for optimizing the motion path of a welding robot based on multi-sensor fusion. The method comprises the following steps: obtaining sensor data, the sensor data including a three-dimensional point cloud of a weld, an image of a molten pool, acceleration and angular velocity of a welding gun end in three axes, welding current, and welding voltage; converting the sensor data into a unified coordinate system to obtain converted sensor data; filtering the converted three-dimensional point cloud of the weld to obtain a filtered point cloud, extracting features of the filtered point cloud based on a RANSAC algorithm to obtain weld centerline features; combining the converted three-dimensional point cloud of the weld and the weld centerline features, and obtaining an initial welding path based on an improved A* algorithm. The present invention realizes intelligent and precise adjustment of welding parameters, ensures the stability of welding quality, and comprehensively improves the performance of the welding robot's motion path planning and welding operation.

Description

Technical Field

[0001] The present invention relates to the technical field of robot welding, and more particularly to a method for optimizing motion path planning of a welding robot based on multi-sensor fusion. Background Art

[0002] In modern manufacturing, welding is a key joining process used in many fields. As the manufacturing industry continues to increase its requirements for product quality and production efficiency, welding robots have gradually become an important tool for welding operations.

[0003] The Chinese patent application with publication number CN117300301A discloses a welding robot weld tracking system and method based on a monocular line laser: Step 1, the welding robot carries a visual sensor to collect weld images; Step 2, the image processing module processes the weld image, obtains the world coordinates of the weld feature points through image denoising, laser stripe centerline extraction, weld feature point identification, and three-dimensional reconstruction of the weld feature point coordinates, and judges the type of the weld feature point at the same time; Step 3, the image processing module stores the weld feature point coordinates in the trajectory planning module; Step 4, the trajectory planning module establishes a local coordinate system of the weld posture based on the weld feature point coordinates of the previous and next two frames of images, obtains the welding posture of the robot welding gun, and generates a welding path; Step 5, welding process matching The module matches welding process parameters based on the desired welding process. In step 6, the weld tracking system stores the welding path information, weld feature point types, and welding process parameters in a host computer cache. The host computer communicates with the robot controller via TCP / IP and sends the welding posture, weld feature point types, and welding process parameters to the robot controller. In step 7, the robot controller parses the host computer data, and the robot adjusts the welding gun posture based on the welding posture data. The robot determines the arc shape and welding speed based on the welding process parameters. The welding robot performs arc starting, welding, or arc ending operations based on the weld feature point data type. The robot controller sends the welding process parameters to the digital intelligent welder via DeviceNet to control the welding current and voltage. This invention enables real-time tracking of welds and real-time adjustment of welding posture.

[0004] Although the above methods can meet most scenarios, research and practical application of the above methods and existing technologies have revealed that the above methods and existing technologies have at least the following defects:

[0005] Monocular line laser sensors have data limitations when weld features change dynamically, and are insufficient to support accurate judgment and adjustment of the welding process. The local coordinate system of the weld posture is established only based on the coordinates of the weld feature points in the two frames of images to generate the welding path and adjust the welding gun posture. This makes it difficult to achieve accurate path planning and real-time correction when faced with complex weld shapes or large deviations during the welding process.

[0006] In view of this, the present invention proposes a welding robot motion path optimization planning method based on multi-sensor fusion to solve the above problems. Summary of the Invention

[0007] In order to overcome the above-mentioned defects of the prior art and achieve the above-mentioned purpose, the present invention provides the following technical solution: a welding robot motion path optimization planning method based on multi-sensor fusion, comprising the following steps:

[0008] Collect and obtain sensor data, including the three-dimensional point cloud of the weld, the weld pool image, the acceleration and angular velocity of the welding gun end in three axes, the welding current, and the welding voltage;

[0009] Convert the sensor data into a unified coordinate system to obtain converted sensor data;

[0010] The converted weld 3D point cloud is filtered to obtain a filtered point cloud, and the feature extraction of the filtered point cloud is performed based on the RANSAC algorithm to obtain the weld centerline feature;

[0011] Combining the converted weld 3D point cloud and weld centerline features, the initial welding path is obtained based on the improved A* algorithm, and the initial welding path is smoothed to obtain a smooth welding path.

[0012] The welding gun posture characteristics are obtained by fusing the weld centerline characteristics, the converted acceleration and angular velocity of the welding gun end in three axes through the extended Kalman filter.

[0013] During the welding process of the welding robot according to the smooth welding path, the welding seam deviation prediction model is used to calculate the welding seam prediction deviation in combination with the welding gun posture characteristics, and the welding path in the same layer of welding space is corrected based on the weld seam prediction deviation. The welding robot then performs welding based on the corrected welding path in the same layer of welding space.

[0014] The converted melt pool image is used as the input of the convolutional neural network to obtain the segmented melt pool contour, and the segmented melt pool contour is analyzed to obtain the melt pool characteristics;

[0015] The molten pool features, converted welding current and converted welding voltage are fused through attention convolution to obtain fused features; the fused features are compared with the data stored in the preset process knowledge base, and the welding parameters of the welding passes of different layers are adaptively adjusted; the welding path in the corrected welding space of the same layer is calculated through the spiral scanning algorithm to obtain the optimal overlap position of the next layer, and the welding path of the welding space of different layers is corrected; the corrected welding path of the welding space of different layers is used as the optimized motion path of the welding robot.

[0016] Furthermore, the method for obtaining the initial welding path includes:

[0017] Step 1: Obtain the robot's welding space based on the weld 3D point cloud, divide the welding space according to the preset layer height to obtain N-layer welding space, and traverse the boundary of the N-layer welding space to divide the N-layer welding space into discrete grid nodes, each node representing a welding gun position;

[0018] Step 2: Select the starting and ending positions of the weld according to the direction of the weld centerline feature. Select the starting and ending points of each layer of weld space in the grid node. The ending point of the previous layer of weld space is the same as the starting point of the next layer of weld space. Repeat steps 3 to 9 for each layer of weld space.

[0019] Step 3: Preset and initialize the open list and closed list: the open list is used to store the nodes to be evaluated, and the initial open list contains the starting point; the closed list is used to store the nodes that have been evaluated, and the initial closed list is empty;

[0020] Step 4: Define the current node Heuristic function to the end point;

[0021] Step 5: Select the node with the smallest evaluation function value from the open list through the evaluation function of the heuristic function ;

[0022] Step 6: Node Move from the open list to the closed list;

[0023] Step 7: Check the nodes Is it the end point: If the node If it is the end point, a path from the starting point to the end point is found and the algorithm ends; otherwise, continue to step 8;

[0024] Step 8: Generate nodes All adjacent nodes , adjacent nodes Indicates that the node The reachable nodes in the up, down, left, right and diagonal directions in the grid;

[0025] Step 9: When the end point is found, start from the end point and trace back to the parent node to generate the welding path from the starting point to the end point of the weld space of the corresponding layer;

[0026] Step 10: Connect the welding paths of each layer of weld space from the starting point to the end point in sequence to obtain the initial welding path.

[0027] Furthermore, in step 8, for each adjacent node , execute AD:

[0028] A. If the node In the closing list, skip the node ;

[0029] B. Calculate the number of nodes from the starting point based on the evaluation function of the heuristic function Arrival Node The actual cost ;

[0030] C. If the node Not in the open list, the node Join the open list and get the node The corresponding actual cost, heuristic function value and evaluation function value, while recording the node The parent node is ;

[0031] D. If the node Already in the open list, compare the currently calculated The value is compared with the actual cost in the open list; if the current calculated actual cost is smaller, the node in the open list is updated The actual cost, evaluation function value and parent node are .

[0032] Furthermore, the method for obtaining a smooth welding path includes:

[0033] The path node coordinates are smoothed using a B-spline curve formula; all smoothed path node coordinates are sequentially connected to obtain a smooth welding path.

[0034] Furthermore, the method for obtaining the weld prediction deviation during the welding process includes:

[0035] A weld deviation prediction model is established based on the current layer height, reference layer height, actual groove width, expected groove width and welding gun posture deviation value, and the weld prediction deviation is calculated based on the weld deviation prediction model; among which, the welding gun posture deviation value is the difference between the welding gun posture feature and the preset welding gun posture feature.

[0036] Furthermore, the method for obtaining the optimal overlap position of the next layer includes:

[0037] By expanding outward the spiral equation regarding the spiral radius, spiral angle and pitch, the optimal overlap position of the next layer is searched; by adjusting the values of the spiral radius, spiral angle and pitch, the overlap position that meets the welding quality requirements is searched, and the overlap position that meets the preset welding quality requirements is used as the optimal overlap position of the next layer.

[0038] Furthermore, the molten pool characteristics include molten pool length, molten pool width and molten pool back angle; the method for obtaining the molten pool length includes:

[0039] The melt pool contour is used as the input of the image segmentation model to obtain the probability that each pixel belongs to the melt pool contour; pixels Probability of belonging to the melt pool contour Perform binarization to obtain the pixels The binarization processing result is obtained; the pixel points with a value of 1 in the binarization processing result are extracted to obtain the binary processed molten pool contour; the molten pool contour after the binarization processing is traversed to obtain the boundary points of the molten pool contour, the Euclidean distance between any two points on the molten pool contour is calculated as the contour distance, and the value with the largest contour distance is selected as the molten pool length; is the number of pixels in the melt pool outline;

[0040] Methods for obtaining the melt pool width include:

[0041] Get the contour points at both ends corresponding to the contour length and , calculate the melt pool contour length direction vector formed by the contour points at both ends and the modulus corresponding to the melt pool contour length direction vector; for any contour point , calculated by the contour points With contour points The resulting vector The projection length of the molten pool is calculated based on the projection length. The distance between the contour point and the straight line in the direction of the molten pool contour length is calculated based on the vector perpendicular to the molten pool length equation. The maximum value of the distance between the contour point and the straight line in the direction of the molten pool contour length is selected as the molten pool width.

[0042] Methods for obtaining the molten pool drag angle include:

[0043] Obtain the preset welding direction, select W contour points at the end far away from the welding starting point, fit them through R-order polynomial, calculate the coefficients of the polynomial by minimizing the square sum of errors combined with the coordinates of the selected contour points, substitute the calculated coefficients of the polynomial into the R-order polynomial to obtain the fitting curve equation, calculate the derivative equation of the fitting curve equation, and put the first Substitute the coordinates of the contour points into the derivative equation to obtain the corresponding tangent slope , calculate the first The angle between the tangent direction of each contour point and the positive direction of the horizontal axis is calculated based on the angle to obtain the molten pool drag angle;

[0044] The angles between the tangent direction of the selected W contour points and the welding direction are calculated and the average value is taken as the molten pool drag angle.

[0045] Furthermore, the method of obtaining fusion features includes:

[0046] The melt pool length, melt pool width and melt pool drag angle are spliced as the visual feature vector , the current and voltage are spliced as the arc characteristic vector , the visual feature vector and arc eigenvector Map to the same dimension and obtain the mapped visual feature vector and the mapped arc eigenvector ; Based on the mapped visual feature vector and the mapped arc feature vector The attention weight is calculated based on the similarity score between the two, and the fusion feature is obtained based on the attention weight calculation.

[0047] Furthermore, the method for obtaining the weld centerline feature includes:

[0048] Step a, randomly select three points in the filtered point cloud to form a plane to be measured, and construct a plane equation of the plane to be measured;

[0049] Step b: Calculate the first Other points The distance to the plane to be measured;

[0050] Step c: determining the relationship between the distances of all other points to the plane to be measured and a preset distance threshold; if the number of other points whose distances are less than the preset distance threshold exceeds a preset number threshold, then determining that the points whose distances are less than the preset distance threshold belong to the same relevant plane;

[0051] Step d, repeating steps a to c K times, and selecting the relevant plane containing the largest number of points in the filtered point cloud as the weld plane;

[0052] Step e: converting the filtered point cloud contained in the weld plane from the base coordinate system to the parameter space through a conversion formula;

[0053] Step f, substitute the coordinates of the points located on the weld centerline in the filtered point cloud into the parametric equation, fit and solve the parametric equation by the least squares method to obtain the solution result, convert the solution result back to a unified coordinate system, obtain the position equation of the weld centerline, substitute the preset value into the position equation of the weld centerline, obtain the weld centerline feature point set, calculate the feature vector composed of adjacent weld centerlines by the vector method, and splice all the feature vectors as the weld centerline feature.

[0054] Furthermore, the method of converting the sensor data into a unified coordinate system includes:

[0055] Build a base coordinate system with the robot base as the origin to obtain the original coordinate system corresponding to the sensor data;

[0056] Use homogeneous transformation matrix to describe the joint motion of a robot with n joints, and express the transformation matrix from the base coordinate system to the original coordinate system as the product of the transformation matrices of each joint;

[0057] The coordinates of the sensor data in the original coordinate system are converted to the base coordinate system through the conversion formula to obtain the converted sensor data.

[0058] Furthermore, the method of obtaining the filtered point cloud includes:

[0059] For each point in the weld 3D point cloud ,in, , and Respectively points of the weld 3D point cloud The horizontal coordinate value, vertical coordinate value and vertical coordinate value in the base coordinate system; the point is calculated by the adaptive Gaussian function Filtered points ; The set of all filtered weld 3D point clouds is taken as the filtered point cloud.

[0060] Furthermore, the method for obtaining the welding gun posture feature includes:

[0061] Define the system state equation;

[0062] The weld centerline feature, the acceleration and angular velocity of the welding gun end in three axes are spliced together as the current moment The corresponding observation value;

[0063] Predict the state and covariance at the next moment based on the system state equation;

[0064] In the updating step, the state and covariance at the next moment are updated according to the observation value to obtain the updated state estimate and updated covariance, and the updated state estimate is used as the welding gun posture feature.

[0065] The technical effects and advantages of the welding robot motion path optimization planning method based on multi-sensor fusion of the present invention are as follows:

[0066] The present invention realizes the all-round and multi-dimensional accurate perception of the welding process by collecting multi-source sensor data such as weld 3D point cloud, molten pool image, acceleration and angular velocity of the end of welding gun in three axes, welding current and voltage and converting them into a unified coordinate system, providing a basis for subsequent processing; the converted weld 3D point cloud is filtered and the weld centerline feature is extracted based on the RANSAC algorithm, and the improved A* algorithm is used to plan and smooth the initial welding path in combination with the weld 3D point cloud and the feature, so that the welding path planning is more accurate and more in line with actual welding requirements; the weld centerline feature and the welding gun end are fused by the extended Kalman filter The end acceleration and angular velocity are used to obtain the welding gun posture characteristics. During welding, the characteristics are combined with the weld deviation prediction model and the spiral scanning algorithm to respectively correct the welding space paths of the same layer and different layers, effectively improving the real-time adaptability and accuracy of the welding path in the actual welding process; the converted molten pool image is processed by a convolutional neural network to obtain the molten pool characteristics, which are fused with the welding current and voltage through attention convolution, and compared with the preset process knowledge base to adaptively adjust the welding parameters, realizing intelligent and precise adjustment of welding parameters, ensuring the stability of welding quality, and comprehensively improving the performance of the welding robot's motion path planning and welding operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 This is a flow chart of a welding robot motion path optimization planning method based on multi-sensor fusion according to the present invention;

[0068] Figure 2 A schematic flow chart of a method for obtaining an initial welding path according to the present invention;

[0069] Figure 3 Schematic diagram of the parameter self-updating method of the present invention;

[0070] Figure 4 Schematic diagram of the sudden interference response method of the present invention. DETAILED DESCRIPTION

[0071] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0072] Example 1

[0073] See also Figure 1 As shown, the welding robot motion path optimization planning method based on multi-sensor fusion described in this embodiment includes the following steps:

[0074] The sensor data is collected and obtained, and the sensor data includes the three-dimensional point cloud of the weld, the molten pool image, the acceleration and angular velocity of the welding gun end in three axes, the welding current and the welding voltage; the three-dimensional point cloud of the weld can be collected by a depth camera, and the collection of the three-dimensional point cloud of the weld can accurately obtain the spatial position and shape information of the weld, providing a basis for planning the welding path; the molten pool image is collected by a high-definition camera, and the collection of the molten pool image can intuitively reflect the state of the welding molten pool, which helps to adjust the welding parameters and path according to the changes in the molten pool; the acceleration of the welding gun end in three axes is measured by an accelerometer, and the angular velocity of the welding gun end in three axes is measured by a gyroscope. The collection of the acceleration and angular velocity of the welding gun end in three axes can grasp the movement posture and dynamic changes of the welding gun in real time, and ensure the accuracy and stability of the welding path; the welding current and welding voltage can be collected by an ammeter and a voltmeter. The collection of welding current and welding voltage can understand the energy input of the welding, which is convenient for optimizing the welding parameters according to the actual welding state, and then indirectly optimize the movement path of the welding robot to ensure the welding quality.

[0075] Based on the welding robot motion model, the sensor data is converted into a unified coordinate system to obtain the converted sensor data;

[0076] Methods for converting sensor data into a unified coordinate system include:

[0077] Build a base coordinate system with the robot base as the origin to obtain the original coordinate system corresponding to the sensor data;

[0078] Using a homogeneous transformation matrix Describe the joint motion of a robot with n joints, where for The rotation matrix is used to express the rotation relationship of the coordinate system; for The translation vector is used to describe the translation relationship of the coordinate system; the transformation matrix from the base coordinate system to the original coordinate system is expressed as the product of the transformation matrices of each joint, such as ; is the transformation matrix from the base coordinate system to the original coordinate system; , and For the 1st, 2nd and The joint transformation matrix.

[0079] The coordinates of the sensor data in the original coordinate system are converted to the base coordinate system through the conversion formula, such as ,in, is the converted sensor data; is the sensor data before conversion, and obtains the sensor data after conversion.

[0080] Adaptive Gaussian filtering is performed on the weld 3D point cloud to obtain a filtered point cloud, and then feature extraction is performed on the filtered point cloud based on the RANSAC algorithm to obtain the weld centerline feature;

[0081] Methods for obtaining filtered point clouds include:

[0082] For each point in the weld 3D point cloud ,in, , and Respectively points of the weld 3D point cloud The horizontal coordinate value, vertical coordinate value and vertical coordinate value in the base coordinate system; the point is calculated by the adaptive Gaussian function Filtered points :

[0083] ;

[0084] in, for point The neighborhood point set of ; for point With neighboring points The weights between them are adaptively adjusted according to the distribution and distance of the neighborhood points; for point No. neighborhood points; the set of all filtered weld 3D point clouds is taken as the filtered point cloud.

[0085] Methods for obtaining weld centerline features include:

[0086] Step a: Randomly select three points in the filtered point cloud to form the plane to be measured, and build the plane equation of the plane to be measured ;in, 、 、 and are all coefficients of the plane equation of the plane to be measured;

[0087] Step b: Calculate the first Other points Distance to the plane to be measured :

[0088] ;

[0089] in, , and For the Other points The horizontal coordinate value, vertical coordinate value and vertical coordinate value in the base coordinate system;

[0090] Step c: determining the relationship between the distances of all other points to the plane to be measured and a preset distance threshold; if the number of other points whose distances are less than the preset distance threshold exceeds a preset number threshold, then determining that the points whose distances are less than the preset distance threshold belong to the same relevant plane;

[0091] Step d, repeating steps a to c K times, and selecting the relevant plane containing the largest number of points in the filtered point cloud as the weld plane;

[0092] Step e: The filtered point cloud contained in the weld plane is converted into Transform from the base coordinate system to the parameter space; where, is a fixed point on the line; is the direction vector of the line; is a parameter;

[0093] Step f: The point on the weld centerline in the point cloud Substitute the coordinates into the parametric equation, fit the parametric equation by the least square method to obtain the solution, convert the solution back to the unified coordinate system, obtain the position equation of the weld centerline, and set the preset The value is substituted into the position equation of the weld centerline to obtain the weld centerline feature point set. The feature vector composed of adjacent weld centerlines is calculated by the vector method, and all feature vectors are spliced as the weld centerline feature.

[0094] Combining the converted weld 3D point cloud and weld centerline features, the initial welding path is obtained based on the improved A* algorithm, and the initial welding path is smoothed to obtain a smooth welding path.

[0095] Reference Figure 2 , the methods for obtaining the initial welding path include:

[0096] Step 1: Obtain the robot's welding space based on the weld 3D point cloud, divide the welding space according to the preset layer height to obtain N-layer welding space, and traverse the boundary of the N-layer welding space to divide the N-layer welding space into discrete grid nodes, each node representing a welding gun position;

[0097] Step 2: Select the starting and ending positions of the weld according to the direction of the weld centerline feature. Select the starting and ending points of each layer of weld space in the grid node. The ending point of the previous layer of weld space is the same as the starting point of the next layer of weld space. Repeat steps 3 to 9 for each layer of weld space.

[0098] Step 3: Preset and initialize the open list and closed list: the open list is used to store the nodes to be evaluated, and the initial open list contains the starting point; the closed list is used to store the nodes that have been evaluated, and the initial closed list is empty.

[0099] Step 4: Set the current node to , the end point is , define the heuristic function ;in, is a node To the end The straight-line distance, is a measure of the difference between the current welding direction and the direction pointing to the end point, and It is the weight coefficient, which can be adjusted according to actual welding requirements.

[0100] Step 5: From the open list, pass the evaluation function on the heuristic function Select the node with the smallest evaluation function value, where From the starting point to the node the actual cost; For nodes The corresponding heuristic function value.

[0101] Step 6: Node Move from the open list to the closed list.

[0102] Step 7: Check the nodes Is it the end point: If the node If it is the end point, a path from the starting point to the end point is found and the algorithm ends; otherwise, continue to step 8;

[0103] Step 8: Generate nodes All adjacent nodes , adjacent nodes Indicates that the node The reachable nodes in the up, down, left, right and diagonal directions of the grid; for each adjacent node , execute AD:

[0104] A. If the node In the closing list, skip the node ;

[0105] B. Calculate the number of nodes from the starting point based on the evaluation function of the heuristic function Arrival Node The actual cost ,in, It is a slave node To Node The cost of movement is considered, taking into account factors such as the moving distance and whether the welding posture needs to be changed. For example, the cost of horizontal or vertical movement is 1, and the cost of diagonal movement is If the welding posture needs to be changed significantly, the cost will increase accordingly.

[0106] C. If the node Not in the open list, the node Join the open list and get the node Corresponding 、 and Value, where For nodes The corresponding heuristic function value; For nodes The corresponding evaluation function value; at the same time record the node The parent node is .

[0107] D. If the node Already in the open list, compare the currently calculated Values are the same as those already in the open list value; if the currently calculated If the value is smaller, update the node in the open list of 、 The value and parent node are This step reflects the dynamic optimization of the path by the improved A* algorithm when considering the actual welding situation, ensuring that the found path is not only short but also meets the welding process requirements.

[0108] Step 9: When the end point is found or the open list is empty, if the end point is found, the welding path from the start point to the end point of the weld space of the corresponding layer is generated by backtracking the parent node.

[0109] Step 10: Connect the welding paths of each layer of weld space from the starting point to the end point in sequence to obtain the initial welding path.

[0110] Methods for achieving a smooth weld path include:

[0111] Through the B-spline curve formula Smoothing path node coordinates; where, is a point on the B-spline curve, i.e. the smoothed Path node coordinates; For the The coordinates of the path nodes Sub-B-spline basis function; The first welding path path node coordinates; V is the number of path nodes; all smoothed path node coordinates are connected in sequence to obtain a smooth welding path.

[0112] The welding gun posture characteristics are obtained by fusing the weld centerline characteristics, the converted acceleration and angular velocity of the welding gun end in three axes through the extended Kalman filter.

[0113] Methods for obtaining welding gun posture features include:

[0114] Define the system state equation ;in, For the current moment The corresponding system status; is the state transition function; For the previous moment The corresponding system status; For the previous moment Corresponding control input; is the process noise;

[0115] The weld centerline feature, the acceleration and angular velocity of the welding gun end in three axes are spliced together as the current moment The corresponding observations ;in, is the observation function; is the observation noise;

[0116] Predict the state at the next moment based on the system state equation ;in, For Moment, based on The optimal estimated system state obtained from all observation data at and before the moment; the covariance at the next moment is predicted based on the system state equation ;in, is the state transfer matrix; For Moment, based on The covariance matrix of the system state estimate obtained from all observation data at and before time; is the transpose of the vector; is the process noise;

[0117] In the update step, the state estimate is updated based on the observations:

[0118] ;in, is the Kalman gain, for Always The estimated value obtained by predicting the state at the moment; the covariance is updated according to the observed value ;in, is the identity matrix; for The observation matrix at time t, is the prior covariance matrix, indicating The uncertainty degree of the updated state estimate As a welding gun posture feature.

[0119] During the welding process of the welding robot according to the smooth welding path, the welding seam deviation prediction model is used to calculate the welding seam prediction deviation in combination with the welding gun posture characteristics, and the welding seam prediction deviation is obtained. The welding path in the same layer of welding space is corrected based on the weld seam prediction deviation. The welding robot adjusts the welding robot posture based on the PID control method based on the corrected welding path in the same layer of welding space to perform welding;

[0120] Methods for obtaining weld prediction deviation during welding include:

[0121] According to the current floor height , reference floor height , actual groove width , expected groove width And welding gun posture deviation value Establishing a weld deviation prediction model , the weld prediction deviation is obtained based on the weld deviation prediction model; wherein, To predict weld deviation; welding gun posture deviation value is the difference between the welding gun posture feature and the preset welding gun posture feature; 、 and The adaptive weight coefficient can be trained and adjusted according to the actual welding data through the least square method or gradient descent method.

[0122] The melt pool image is used as the input of the convolutional neural network to obtain the segmented melt pool contour. The segmented melt pool contour is analyzed to obtain the melt pool features, which include melt pool length, melt pool width and melt pool drag angle.

[0123] Convolutional neural network training methods include:

[0124] D groups of training data are collected in advance, and the training data include melt pool images and corresponding melt pool contours.

[0125] The melt pool image is used as the input of the convolutional neural network, and the melt pool contour is used as the output of the convolutional neural network. The goal is to minimize the error between the output melt pool contour and the actual melt pool contour. The network parameters of the convolutional neural network are optimized through the nature-inspired optimization algorithm to obtain the network parameters that minimize the error between the melt pool contour output by the convolutional neural network and the actual melt pool contour. The convolutional neural network constructed with the corresponding network parameters is used as the trained convolutional neural network.

[0126] Methods for obtaining the melt pool length include:

[0127] The melt pool contour is used as the input of the image segmentation model to obtain the probability that each pixel belongs to the melt pool contour. The training method of the image segmentation model is similar to that of the convolutional neural network. pixels Probability of belonging to the melt pool contour Perform binarization , and obtain the pixels The binarization result of ;in, is the preset contour threshold; is the number of pixels in the molten pool contour; the binarized molten pool contour is obtained by extracting the pixels with a value of 1 in the binarization processing result; the molten pool contour after binarization processing is traversed to obtain the boundary points of the molten pool contour, and the Euclidean distance between any two points on the molten pool contour is calculated as the contour distance, and the value with the largest contour distance is selected as the molten pool length.

[0128] Methods for obtaining the melt pool width include:

[0129] Get the contour points at both ends corresponding to the contour length and , calculate the length direction vector of the molten pool contour , and the modulus corresponding to the length direction vector of the molten pool contour; for any contour point , calculated by the contour points With contour points The resulting vector Projection length , is the length direction vector of the melt pool contour Corresponding unit vector; calculate the vector perpendicular to the melt pool length equation ; Calculate the distance from the contour point to the straight line in the direction of the melt pool contour length based on the vector perpendicular to the melt pool length equation ; Select the maximum value of the distance from the contour point to the straight line in the length direction of the molten pool contour as the molten pool width.

[0130] Methods for obtaining the molten pool drag angle include:

[0131] Obtain the preset welding direction, select W contour points at the end far away from the welding starting point, fit them through R-order polynomial, calculate the coefficients of the polynomial by minimizing the square sum of errors combined with the coordinates of the selected contour points, substitute the calculated coefficients of the polynomial into the R-order polynomial to obtain the fitting curve equation, calculate the derivative equation of the fitting curve equation, and put the first Substitute the coordinates of the contour points into the derivative equation to obtain the corresponding tangent slope , calculate the first The angle between the tangent direction of the contour point and the positive direction of the horizontal axis , calculate the molten pool drag angle ;in, ; It is the angle between the welding direction and the positive direction of the horizontal axis.

[0132] The angles between the tangent direction of the selected W contour points and the welding direction are calculated and the average value is taken as the molten pool drag angle.

[0133] The molten pool features, converted welding current and converted welding voltage are fused through attention convolution to obtain fused features; the fused features are compared with the data stored in the preset process knowledge base to adaptively adjust the welding parameters; the welding path in the corrected welding space of the same layer is calculated through the spiral scanning algorithm to obtain the optimal overlap position of the next layer, and the welding path of the welding space of different layers is corrected; the corrected welding path of the welding space of different layers is used as the optimized motion path of the welding robot; the welding parameters include welding speed, welding current and welding inclination.

[0134] Methods for obtaining fusion features include:

[0135] The melt pool length, melt pool width and melt pool drag angle are spliced as the visual feature vector , the current and voltage are spliced as the arc characteristic vector , the visual feature vector and arc eigenvector Map to the same dimension and obtain the mapped visual feature vector and the mapped arc eigenvector ; Calculate the attention weight ;in, is the similarity function; is the mapped visual feature vector and the mapped arc feature vector Similarity score of is the arc feature vector after mapping and the mapped visual feature vector Similarity score based on attention weight Calculate the fusion features .

[0136] Methods for adaptively adjusting welding parameters include:

[0137] The preset process knowledge base stores the optimal welding parameters corresponding to the intervals corresponding to different fusion features. The fusion features are matched with the intervals corresponding to the fusion features to obtain the matching intervals. The current welding parameters are compared with the optimal welding parameters corresponding to the matching intervals. If the current welding parameters are different from the optimal welding parameters corresponding to the matching intervals, the current welding parameters are adaptively adjusted to be the same as the optimal welding parameters corresponding to the matching intervals.

[0138] Methods for obtaining the optimal overlap position of the next layer include:

[0139] Expand outward through the spiral equation to search for the optimal overlap position of the next layer. The spiral equation is as follows:

[0140] ;

[0141] ;

[0142] ;

[0143] in, , and is the coordinate of the optimal overlap position of the next layer in the unified coordinate system, that is, the corrected path node coordinate; , and are the path node coordinates in a unified coordinate system; is the helix radius; is the spiral angle; is the pitch; by adjusting 、 and The value of is used to search for the overlap position that meets the welding quality requirements, and the overlap position that meets the preset welding quality requirements is used as the optimal overlap position for the next layer.

[0144] Example 2

[0145] See also Figure 3 As shown, this embodiment provides a parameter self-updating method for a welding robot motion path optimization planning method based on multi-sensor fusion, comprising the following steps:

[0146] Define the state space , state space Contains welding speed, welding current, layer height and weld prediction deviation;

[0147] Defining the action space , action space Contains the adjustment amount of welding parameters;

[0148] Define the reward function ;in, For forming quality; is welding efficiency (such as welding time); is the welding energy consumption; , and is the weight coefficient, which can be adjusted according to actual needs.

[0149] Initialize a Q network and a target Q network. The Q network is used to select actions, and the target Q network is used to calculate the target Q value to stabilize the learning process.

[0150] During the welding process, the welding robot interacts with the environment, continuously collecting sample data. During each interaction, the welding robot selects an action based on its current state through the Q network. After executing the selected action, the environment returns the new state and reward function value. The current state, selected action, reward function value, and the new state returned by the environment are stored in an experience replay pool. When the experience replay pool contains a certain amount of data, a batch of data is randomly sampled from it to train the Q network. The parameters of the Q network are updated by minimizing the loss function, enabling the Q network to more accurately estimate the Q value and guide the robot to select more optimal actions.

[0151] After completing every five welds, the welding parameter library is updated according to the current welding data and the reward function value.

[0152] Regularly evaluate the trained model by running it in actual welding scenarios or simulated environments to observe indicators such as welding quality, efficiency, and energy consumption. Based on the evaluation results, optimize the model's parameters, state space, action space, or reward function.

[0153] Example 3

[0154] See also Figure 4 As shown, this embodiment provides a sudden interference response method applied to a welding robot motion path optimization planning method based on multi-sensor fusion, comprising the following steps:

[0155] The rate of change of the welding current is calculated by the ratio of the difference in welding current at different moments to the corresponding time interval. When the rate of change of the welding current exceeds the preset current threshold, the welding robot is immediately triggered to stop.

[0156] The welding robot records the breakpoint position through the position sensor and generates the repair path based on the breakpoint position and welding process requirements using the improved A* algorithm.

[0157] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

[0158] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A welding robot motion path optimization planning method based on multi-sensor fusion, characterized in that: The steps include: Collect and obtain sensor data, including the three-dimensional point cloud of the weld, the weld pool image, the acceleration and angular velocity of the welding gun end in three axes, the welding current, and the welding voltage; Convert the sensor data into a unified coordinate system to obtain converted sensor data; The converted weld 3D point cloud is filtered to obtain a filtered point cloud, and the feature extraction of the filtered point cloud is performed based on the RANSAC algorithm to obtain the weld centerline feature; Combining the converted weld 3D point cloud and weld centerline features, the initial welding path is obtained based on the improved A* algorithm, and the initial welding path is smoothed to obtain a smooth welding path. The welding gun posture characteristics are obtained by fusing the weld centerline characteristics, the converted acceleration and angular velocity of the welding gun end in three axes through the extended Kalman filter. During the welding process of the welding robot according to the smooth welding path, the welding seam deviation prediction model is used to calculate the welding seam prediction deviation in combination with the welding gun posture characteristics, and the welding path in the same layer of welding space is corrected based on the weld seam prediction deviation. The welding robot then performs welding based on the corrected welding path in the same layer of welding space. The converted melt pool image is used as the input of the convolutional neural network to obtain the segmented melt pool contour, and the segmented melt pool contour is analyzed to obtain the melt pool characteristics; The molten pool features, converted welding current and converted welding voltage are fused through attention convolution to obtain fused features; the fused features are compared with the data stored in the preset process knowledge base, and the welding parameters of the welding passes of different layers are adaptively adjusted; the welding path in the corrected welding space of the same layer is calculated through the spiral scanning algorithm to obtain the optimal overlap position of the next layer, and the welding path of the welding space of different layers is corrected; the corrected welding path of the welding space of different layers is used as the optimized motion path of the welding robot.

2. The welding robot motion path optimization planning method based on multi-sensor fusion according to claim 1 is characterized in that: Methods for obtaining the initial weld path include: Step 1: Obtain the robot's welding space based on the weld 3D point cloud, divide the welding space according to the preset layer height to obtain N-layer welding space, and traverse the boundary of the N-layer welding space to divide the N-layer welding space into discrete grid nodes, each node representing a welding gun position; Step 2: Select the starting and ending positions of the weld according to the direction of the weld centerline feature. Select the starting and ending points of each layer of weld space in the grid node. The ending point of the previous layer of weld space is the same as the starting point of the next layer of weld space. Repeat steps 3 to 9 for each layer of weld space. Step 3: Preset and initialize the open list and closed list: the open list is used to store the nodes to be evaluated, and the initial open list contains the starting point; the closed list is used to store the nodes that have been evaluated, and the initial closed list is empty; Step 4: Define the current node Heuristic function to the end point; Step 5: Select the node with the smallest evaluation function value from the open list through the evaluation function of the heuristic function ; Step 6: Node Move from the open list to the closed list; Step 7: Check the nodes Is it the end point: If the node If it is the end point, a path from the starting point to the end point is found and the algorithm ends; otherwise, continue to step 8; Step 8: Generate nodes All adjacent nodes , adjacent nodes Indicates that the node The reachable nodes in the up, down, left, right and diagonal directions in the grid; Step 9: When the end point is found, start from the end point and trace back to the parent node to generate the welding path from the starting point to the end point of the weld space of the corresponding layer; Step 10: Connect the welding paths of each layer of weld space from the starting point to the end point in sequence to obtain the initial welding path.

3. The welding robot motion path optimization planning method based on multi-sensor fusion according to claim 2 is characterized in that: In step 8, for each adjacent node , execute AD: A. If the node In the closing list, skip the node ; B. Calculate the number of nodes from the starting point based on the evaluation function of the heuristic function Arrival Node The actual cost ; C. If the node Not in the open list, the node Join the open list and get the node The corresponding actual cost, heuristic function value and evaluation function value, while recording the node The parent node is ; D. If the node Already in the open list, compare the currently calculated Value and actual cost already in the open list; If the currently calculated actual cost is smaller, update the node in the open list The actual cost, evaluation function value and parent node are .

4. The welding robot motion path optimization planning method based on multi-sensor fusion according to claim 3 is characterized in that: Methods for achieving a smooth weld path include: The path node coordinates are smoothed using a B-spline curve formula; all smoothed path node coordinates are sequentially connected to obtain a smooth welding path.

5. The welding robot motion path optimization planning method based on multi-sensor fusion according to claim 4 is characterized in that: Methods for obtaining weld prediction deviation during welding include: A weld deviation prediction model is established based on the current layer height, reference layer height, actual groove width, expected groove width and welding gun posture deviation value, and the weld prediction deviation is calculated based on the weld deviation prediction model; among which, the welding gun posture deviation value is the difference between the welding gun posture feature and the preset welding gun posture feature.

6. The welding robot motion path optimization planning method based on multi-sensor fusion according to claim 5 is characterized in that: Methods for obtaining the optimal overlap position of the next layer include: By expanding outward the spiral equation regarding the spiral radius, spiral angle and pitch, the optimal overlap position of the next layer is searched; by adjusting the values of the spiral radius, spiral angle and pitch, the overlap position that meets the welding quality requirements is searched, and the overlap position that meets the preset welding quality requirements is used as the optimal overlap position of the next layer.

7. The welding robot motion path optimization planning method based on multi-sensor fusion according to claim 1 is characterized in that: The molten pool characteristics include molten pool length, molten pool width and molten pool drag angle; Methods for obtaining the melt pool length include: The melt pool contour is used as the input of the image segmentation model to obtain the probability that each pixel belongs to the melt pool contour; The probability that a pixel belongs to the melt pool contour Perform binarization to obtain the The binary processing results of the pixels are obtained; the pixel points with a value of 1 in the binary processing results are extracted to obtain the binary processed molten pool contour; the molten pool contour after the binary processing is traversed to obtain the boundary points of the molten pool contour, the Euclidean distance between any two points on the molten pool contour is calculated as the contour distance, and the value with the largest contour distance is selected as the molten pool length; Methods for obtaining the melt pool width include: Get the contour points at both ends corresponding to the contour length and , calculate the melt pool contour length direction vector formed by the contour points at both ends and the modulus corresponding to the melt pool contour length direction vector; for any contour point , calculated by the contour points With contour points The resulting vector The projection length of the molten pool is calculated based on the projection length. The distance between the contour point and the straight line in the direction of the molten pool contour length is calculated based on the vector perpendicular to the molten pool length equation. The maximum value of the distance between the contour point and the straight line in the direction of the molten pool contour length is selected as the molten pool width. Methods for obtaining the molten pool drag angle include: Obtain the preset welding direction, select W contour points at the end far away from the welding starting point, fit them through R-order polynomial, calculate the coefficients of the polynomial by minimizing the square sum of errors combined with the coordinates of the selected contour points, substitute the calculated coefficients of the polynomial into the R-order polynomial to obtain the fitting curve equation, calculate the derivative equation of the fitting curve equation, and put the first Substitute the coordinates of the contour points into the derivative equation to obtain the corresponding tangent slope , calculate the first The angle between the tangent direction of each contour point and the positive direction of the horizontal axis is calculated based on the angle to obtain the molten pool drag angle; The angles between the tangent direction of the selected W contour points and the welding direction are calculated and the average value is taken as the molten pool drag angle.

8. The welding robot motion path optimization planning method based on multi-sensor fusion according to claim 7 is characterized in that: Methods for obtaining fusion features include: The melt pool length, melt pool width and melt pool drag angle are spliced as the visual feature vector , the current and voltage are spliced as the arc characteristic vector , the visual feature vector and arc eigenvector Map to the same dimension and obtain the mapped visual feature vector and the mapped arc eigenvector ; Based on the mapped visual feature vector and the mapped arc feature vector The attention weight is calculated based on the similarity score between the two, and the fusion feature is obtained based on the attention weight calculation.

9. The welding robot motion path optimization planning method based on multi-sensor fusion according to claim 1 is characterized in that: Methods for obtaining weld centerline features include: Step a, randomly select three points in the filtered point cloud to form a plane to be measured, and construct a plane equation of the plane to be measured; Step b: Calculate the first Other points The distance to the plane to be measured; Step c: determining the relationship between the distances of all other points to the plane to be measured and a preset distance threshold; if the number of other points whose distances are less than the preset distance threshold exceeds a preset number threshold, then determining that the points whose distances are less than the preset distance threshold belong to the same relevant plane; Step d, repeating steps a to c K times, and selecting the relevant plane containing the largest number of points in the filtered point cloud as the weld plane; Step e: converting the filtered point cloud contained in the weld plane from the base coordinate system to the parameter space through a conversion formula; Step f, substitute the coordinates of the points located on the weld centerline in the filtered point cloud into the parametric equation, fit and solve the parametric equation by the least squares method to obtain the solution result, convert the solution result back to a unified coordinate system, obtain the position equation of the weld centerline, substitute the preset value into the position equation of the weld centerline, obtain the weld centerline feature point set, calculate the feature vector composed of adjacent weld centerlines by the vector method, and splice all the feature vectors as the weld centerline feature.

10. The welding robot motion path optimization planning method based on multi-sensor fusion according to claim 1, characterized in that: Methods for converting sensor data into a unified coordinate system include: Build a base coordinate system with the robot base as the origin to obtain the original coordinate system corresponding to the sensor data; Use homogeneous transformation matrix to describe the joint motion of a robot with n joints, and express the transformation matrix from the base coordinate system to the original coordinate system as the product of the transformation matrices of each joint; The coordinates of the sensor data in the original coordinate system are converted to the base coordinate system through the conversion formula to obtain the converted sensor data.

11. The welding robot motion path optimization planning method based on multi-sensor fusion according to claim 1, characterized in that: Methods for obtaining filtered point clouds include: For each point in the weld 3D point cloud ,in, , and Respectively points of the weld 3D point cloud The horizontal coordinate value, vertical coordinate value and vertical coordinate value in the base coordinate system; the point is calculated by the adaptive Gaussian function Filtered points ; The set of all filtered weld 3D point clouds is taken as the filtered point cloud.

12. The welding robot motion path optimization planning method based on multi-sensor fusion according to claim 1, characterized in that: Methods for obtaining welding gun posture features include: Define the system state equation; The weld centerline feature, the acceleration and angular velocity of the welding gun end in three axes are spliced together as the current moment The corresponding observation value; Predict the state and covariance at the next moment based on the system state equation; In the updating step, the state and covariance at the next moment are updated according to the observation value to obtain the updated state estimate and updated covariance, and the updated state estimate is used as the welding gun posture feature.

Images