A teaching-free welding robot path planning system based on visual autonomous learning

Abstract

The present invention provides a teaching-free welding robot path planning system based on visual autonomous learning, which relates to the field of industrial welding technology. The system comprises: a posture compensation module for arranging three infrared reflective marking points in an L-shape on the surface of a fixture base, using a stereo camera to collect the original point cloud of the marking points and the workpiece, and preprocessing the point cloud; generating a posture compensation matrix based on the offset between the actual coordinates and the theoretical coordinates of the marking points, and outputting a denoised point cloud data set and the compensation matrix; a posture correction module for fitting the workpiece plane equation based on the denoised point cloud data set using a random sampling consistency algorithm, applying the posture compensation matrix to the workpiece plane equation, correcting the posture error caused by fixture drift, calculating the coordinates of the weld trajectory endpoints, and outputting the corrected weld posture parameters and scanning posture instructions. The present invention can compensate for fixture drift, correct weld posture, extract three-dimensional features, and plan a collision-free path, thereby improving welding accuracy, efficiency, and automation.

Description

Technical Field

[0001] The present invention relates to the technical field of industrial welding, and in particular to a teaching-free welding robot path planning system based on visual autonomous learning. Background Art

[0002] The traditional teaching programming model (i.e., manually guiding the robot to record path points) has the problems of low efficiency, reliance on experience, and difficulty in adapting to changes in the position of the workpiece or fixture (such as drift caused by welding thermal deformation), which restricts the improvement of the automation level.

[0003] Vision-guided welding technologies, such as online laser scanning, offer some improvements in adaptability by providing real-time perception of weld seam location. However, existing systems still face significant challenges in coping with the subtle drift of the fixture base that can occur during welding, which can cause deviations in the global coordinate system.

[0004] Since existing methods mainly rely on local scanning information of the weld itself, there is a lack of an efficient and robust mechanism to detect and compensate for such global fixture posture changes in real time. This may cause the final planned welding path to produce absolute position errors, affecting welding accuracy and quality. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide a teaching-free welding robot path planning system based on visual autonomous learning, which improves the accuracy of the welding path.

[0006] In order to solve the above technical problems, the technical solutions of the present invention are as follows:

[0007] In the first aspect, a teaching-free welding robot path planning system based on visual autonomous learning includes:

[0008] The pose compensation module is used to arrange three infrared reflective markers in an L-shape on the surface of the fixture base, use a stereo camera to collect the original point cloud of the markers and the workpiece, and pre-process the point cloud. It generates a pose compensation matrix based on the offset between the actual coordinates and the theoretical coordinates of the markers, and outputs a denoised point cloud dataset and compensation matrix.

[0009] The pose correction module is used to fit the workpiece plane equation based on the denoised point cloud dataset using a random sampling consistency algorithm, and then applies the pose compensation matrix to the workpiece plane equation to correct the pose error caused by fixture drift, calculate the coordinates of the weld trajectory endpoints, and output the corrected weld pose parameters and scanning pose instructions;

[0010] The extraction module is used to control the robot to drive the line structured light scanning camera to move along the correction weld trajectory according to the scanning posture instruction, collect the line laser image sequence and extract the center line of the light strip, and output the two-dimensional coordinate set of the center point of the light strip;

[0011] The point cloud reconstruction module is used to reconstruct a 3D point cloud based on the 2D coordinate set of the center point of the light strip and the light plane calibration parameters. It outputs a 3D point cloud of the weld by constructing topological relationships and extracting weld feature points.

[0012] The path planning module is used to implement a bidirectional growth search based on the three-dimensional point cloud of the weld using an improved star-shaped rapid expanding tree algorithm, optimize the path smoothness through a greedy strategy, and output collision-free welding path instructions; according to the welding path instructions, the welding operation is performed in combination with the robot collision detection model.

[0013] In a second aspect, a computing device includes:

[0014] one or more processors;

[0015] The storage device is used to store one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the system.

[0016] According to a third aspect, a computer-readable storage medium stores a program, which implements the system when executed by a processor.

[0017] The above solution of the present invention includes at least the following beneficial effects:

[0018] By generating a pose compensation matrix through the infrared reflective marking points on the fixture base, the pose error caused by fixture drift can be effectively corrected, the stability and accuracy of weld positioning can be improved, and the welding deviation caused by slight displacement of the equipment can be reduced. The dual-vision solution combining stereo vision and line structured light scanning first realizes rough positioning of the weld through a random sampling consistency algorithm, and then extracts feature points through high-precision scanning of line structured light, taking into account both large-scale detection efficiency and local high-precision positioning requirements, thereby improving the reliability of weld identification. The improved star-shaped rapid expansion tree algorithm is used for bidirectional growth search, and the path is optimized through a greedy strategy. It can efficiently generate collision-free and smooth welding paths, improving the efficiency and safety of path planning. No manual teaching is required throughout the process. The welding process is automated through visual autonomous learning and dynamic adjustment, reducing dependence on operator experience and improving the adaptability and production efficiency of small-batch and multi-variety workpiece welding. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is a schematic diagram of a teaching-free welding robot path planning system based on visual autonomous learning provided by an embodiment of the present invention.

[0020] Figure 2This is a flow chart of an embodiment of the present invention for reconstructing a three-dimensional point cloud based on a two-dimensional coordinate set of the center point of a light strip, combined with light plane calibration parameters, and outputting a three-dimensional point cloud of a weld by constructing a topological relationship and extracting weld feature points. DETAILED DESCRIPTION

[0021] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0022] like Figure 1 As shown, an embodiment of the present invention proposes a teaching-free welding robot path planning system based on visual autonomous learning, comprising:

[0023] The pose compensation module is used to arrange three infrared reflective markers in an L-shape on the surface of the fixture base, use a stereo camera to collect the original point cloud of the markers and the workpiece, and pre-process the point cloud. It generates a pose compensation matrix based on the offset between the actual coordinates and the theoretical coordinates of the markers, and outputs a denoised point cloud dataset and compensation matrix.

[0024] The pose correction module is used to fit the workpiece plane equation based on the denoised point cloud dataset using a random sampling consistency algorithm, and then applies the pose compensation matrix to the workpiece plane equation to correct the pose error caused by fixture drift, calculate the coordinates of the weld trajectory endpoints, and output the corrected weld pose parameters and scanning pose instructions;

[0025] The extraction module is used to control the robot to drive the line structured light scanning camera to move along the correction weld trajectory according to the scanning posture instruction, collect the line laser image sequence and extract the center line of the light strip, and output the two-dimensional coordinate set of the center point of the light strip;

[0026] The point cloud reconstruction module is used to reconstruct a 3D point cloud based on the 2D coordinate set of the center point of the light strip and the light plane calibration parameters. It outputs a 3D point cloud of the weld by constructing topological relationships and extracting weld feature points.

[0027] The path planning module is used to implement a bidirectional growth search based on the three-dimensional point cloud of the weld using an improved star-shaped rapid expanding tree algorithm, optimize the path smoothness through a greedy strategy, and output collision-free welding path instructions; according to the welding path instructions, the welding operation is performed in combination with the robot collision detection model.

[0028] In an embodiment of the present invention, infrared reflective markers arranged in an L-shape are used to capture fixture drift in real time, effectively correcting posture errors caused by slight displacements of the fixture; the posture compensation matrix is ​​used to further correct errors, accurately calculate weld endpoints, and ensure that the output weld posture parameters and scanning instructions accurately adapt to the actual workpiece state, thereby improving the stability of weld positioning. The two-dimensional coordinates are reconstructed into a three-dimensional point cloud in combination with the light plane calibration parameters. By constructing topological relationships and extracting feature points, the three-dimensional shape of the weld is accurately restored, providing a high-precision spatial basis for path planning. An improved algorithm is used to implement bidirectional growth search and optimize path smoothness, quickly generating a collision-free welding path. Combined with a collision detection model, the welding process is safe and reliable, improving the automation and efficiency of welding and reducing manual intervention.

[0029] In a preferred embodiment of the present invention, three infrared reflective marking points are arranged in an L-shape on the surface of the fixture base, a stereo camera is used to collect the original point cloud of the marking points and the workpiece, and the point cloud is preprocessed; a pose compensation matrix is ​​generated based on the offset between the actual coordinates and the theoretical coordinates of the marking points, and a denoised point cloud dataset and a compensation matrix are output, which may include:

[0030] Based on the spatial range of the fixture coordinate system, the original point cloud is filtered directly to intercept the valid area point cloud containing the workpiece and the marking points;

[0031] Perform voxel grid processing on the point cloud of the valid area to generate a downsampled point cloud;

[0032] Compute the distance distribution characteristics of each point in the neighborhood of the downsampled point cloud to generate a denoised point cloud dataset;

[0033] In the denoised point cloud dataset, the marker points are identified by the reflection intensity threshold and the measured 3D coordinate values ​​are extracted. The theoretical coordinates of the marker points are preset and the least squares method is used to calculate the rigid body transformation parameters from the measured coordinates to the theoretical coordinates.

[0034] The rigid body transformation parameters are converted into a 4×4 homogeneous transformation matrix as the pose compensation matrix, and the denoised point cloud dataset and pose compensation matrix are output.

[0035] In an embodiment of the present invention, a three-dimensional spatial range is defined: first, the physical dimensions of the fixture (such as length, width, and height) are measured, and combined with the maximum placement range of the workpiece and the infrared reflective marking point, a closed three-dimensional rectangular area is delineated in the fixture coordinate system; for example, if the fixture base is 500 mm long along the X-axis, 300 mm wide along the Y-axis, and 200 mm high along the Z-axis, and the workpiece and the marking point are both located within a range of 0-100 mm on the fixture surface and above, then the X-axis range is set to [0, 500 mm], the Y-axis range is set to [0, 300 mm], and the Z-axis range is set to [0, 300 mm] (including the fixture itself and the workpiece above it).

[0036] From the original point cloud collected by the stereo camera, extract the 3D coordinates (X, Y, Z) of each point one by one and compare them with the preset 3D range:

[0037] If the X coordinate of a point is between [0, 500 mm], the Y coordinate is between [0, 300 mm], and the Z coordinate is between [0, 300 mm], then the point is determined to be a valid point (belonging to the workpiece or the mark point) and is retained;

[0038] If any of the X, Y, and Z coordinates of a point exceeds the above range (such as the Z of a workbench point below 0, or the X of a distant point in the environment > 500mm), it is considered an invalid point and is directly removed.

[0039] Output valid area point cloud: After the above screening, the remaining point cloud only contains the fixture, workpiece and marking points, excluding points in irrelevant areas such as the workbench, ground, surrounding equipment, etc.

[0040] Voxel gridding of the valid area point cloud to generate a downsampled point cloud:

[0041] Based on the point cloud density requirements (e.g., if the welding accuracy requirement is 1mm, the voxel size can be set to 5mm), the valid area point cloud space obtained in the first step is evenly divided into multiple cubes (voxels) with a side length of 5mm. For example, a 500mm range on the X axis can be divided into 100 voxels (500 / 5=100), a 300mm range on the Y axis can be divided into 60 voxels, and a 300mm range on the Z axis can be divided into 60 voxels, forming a 100×60×60 voxel grid. For each voxel, all points in the valid area point cloud are traversed to determine whether the point belongs to the current voxel (i.e., whether the point's X, Y, and Z coordinates fall within the X, Y, and Z intervals of the voxel):

[0042] If the X range of voxel A is [0, 5mm], the Y range is [0, 5mm], and the Z range is [0, 5mm], then all points with X∈[0, 5), Y∈[0, 5), and Z∈[0, 5) belong to voxel A;

[0043] Collect all points within voxel A, calculate the average X coordinate (Xavg = sum of all points X / number of points), average Y coordinate (Yavg), and average Z coordinate (Zavg) of these points, and obtain the centroid coordinates of the voxel (Xavg, Yavg, Zavg);

[0044] All original points in voxel A are replaced by the barycentric coordinates as representative points after downsampling. The above operation is repeated for all voxels. The number of point clouds finally obtained is only 1 / (5×5×5)=1 / 125 of the original valid point cloud (assuming that the original point cloud is evenly distributed within the voxel). This reduces the amount of data while preserving the overall shape of the workpiece and the markers.

[0045] Calculate the neighborhood distance distribution characteristics and generate a denoised point cloud dataset:

[0046] Search the K nearest neighbors of each point: For the downsampled point cloud obtained in the second step, process each point P one by one:

[0047] With point P as the center, calculate the Euclidean distance between point P and all other points in the point cloud, that is, the straight-line distance, such as the distance between point P (x1, y1, z1) and point Q (x2, y2, z2); sort all distances and select the points corresponding to the first 50 smallest distances as the neighborhood points of point P (K=50), where K is the number of neighborhood points.

[0048] Calculate the average neighborhood distance: For each point P, add the distance between it and its 50 neighboring points, and then divide by 50 to obtain the average neighborhood distance of point P (for example, if the sum of the distances of the 50 neighbors of point P is 100 mm, the average distance is 2 mm).

[0049] Count global distance features and denoise:

[0050] Calculate the overall mean (e.g., the average distance of all points is 2.5 mm) and variance (e.g., the variance is 0.8 mm, reflecting the degree of distance dispersion) of the average neighborhood distance of all points; set the denoising threshold to "mean ± 1 × variance" (i.e., 2.5 ± 0.8 mm, range 1.7-3.3 mm); check the average neighborhood distance of each point one by one: if the distance is within the range of 1.7-3.3 mm, retain the point; if the distance is < 1.7 mm (e.g., dense noise points) or > 3.3 mm (e.g., isolated interference points), determine it as a noise point and remove it; after screening, the isolated noise and dense interference points in the point cloud are removed, and only the continuous point cloud of the workpiece and the marking points is retained.

[0051] Identify the markers and calculate the rigid body transformation parameters:

[0052] The reflection intensity of the infrared reflective marking point is much higher than that of the workpiece surface (for example, the reflection intensity of the marking point is 200-255, and that of the workpiece surface is 50-100, assuming the intensity range is 0-255). Set the reflection intensity threshold to 150:

[0053] Traverse all points in the denoised point cloud and retain only those with reflection intensity > 150 to obtain a set of candidate marker points (usually containing 3 true marker points and a small number of falsely detected high-reflection points, such as reflective areas on the workpiece surface).

[0054] Determine the true marker points: Since the marker points are arranged in an L-shape (the three points in space are not collinear, and the line connecting any two points is perpendicular), the geometric relationship of the candidate marker point set is verified:

[0055] Calculate the distance and angle between any three points in the candidate point set and select three points that meet the L-shaped characteristics (for example, the distance between two points is 100mm, the distance between the other two points is 100mm, and the angle between them is 90°, which is consistent with the preset marking point size);

[0056] Eliminate candidate points that do not meet the L-shaped characteristics (such as three points are collinear, distance or angle are inconsistent), and finally determine the three-dimensional coordinate measured values ​​of the three real marking points (P1 measured, P2 measured, P3 measured).

[0057] Given the preset theoretical coordinates of the three markers in the fixture coordinate system (P1 theory, P2 theory, P3 theory), the rotation matrix (R) and translation vector (T) are solved by the least squares method:

[0058] Assume that the measured point coincides with the theoretical point after rotation and translation, that is, Ptheoretical = R × Pmeasured + T;

[0059] Substitute the measured and theoretical values ​​of P1, P2, and P3 to construct the error equation (error = the difference between the actual transformed coordinates and the theoretical coordinates); iteratively adjust the parameters of R and T to minimize the sum of the squared errors of all points (such as (P1 transformation - P1 theory)² + (P2 transformation - P2 theory)² + (P3 transformation - P3 theory)²), and finally obtain the optimal rotation matrix R (3×3) and translation vector T (3×1).

[0060] Convert the rigid body transformation parameters into a pose compensation matrix and output the result:

[0061] Convert the rotation matrix R and translation vector T into a 4×4 homogeneous transformation matrix (posture compensation matrix). The first to third rows and the first to third columns of the matrix are filled with the rotation matrix R to describe the rotation correction of the coordinate system; the first to third rows and the fourth column of the matrix are filled with the translation vector T to describe the translation correction of the coordinate system; the fourth row of the matrix is ​​fixed to [0, 0, 0, 1] to meet the homogeneous transformation rules of three-dimensional coordinates (such as the homogeneous coordinates of the point (x, y, z) are (x, y, z, 1)).

[0062] The final output is two data: denoised point cloud dataset (the workpiece and marker point cloud after filtering, downsampling, and denoising); pose compensation matrix.

[0063] Through-filtering and voxel downsampling quickly eliminate invalid point clouds, reducing data volume while retaining key information. Neighborhood distance denoising effectively filters out isolated noise points, improving the purity of the point cloud and laying a solid foundation for marker recognition and workpiece feature extraction. The characteristics of infrared reflective markers are utilized for precise identification. Combined with the rigid body transformation parameters calculated using the least squares method, the pose error caused by fixture drift can be accurately quantified. The rigid body transformation parameters are converted into a homogeneous matrix to effectively offset the effects of fixture drift and improve weld positioning accuracy. The combination of the denoised point cloud and the pose compensation matrix reduces welding deviations caused by small equipment displacements.

[0064] In a preferred embodiment of the present invention, based on the denoised point cloud data set, a random sampling consistency algorithm is used to fit the workpiece plane equation, and a posture compensation matrix is ​​applied to the workpiece plane equation to correct the posture error caused by fixture drift, calculate the coordinates of the weld trajectory endpoints, and output the corrected weld posture parameters and scanning posture instructions, which may include:

[0065] The pose compensation matrix is ​​fused with the denoised point cloud dataset to achieve point cloud coordinate space transformation and generate pose-corrected point cloud;

[0066] Based on the pose-corrected point cloud, the random sampling consistency algorithm is used to fit the vertical plane equation of the workpiece, and based on the same corrected point cloud, the bottom plane equation of the workpiece is fitted;

[0067] Solve the spatial intersection equation by simultaneously solving the vertical plane equation and the bottom plane equation, and take the two intersection points of the spatial intersection line and the actual boundary of the workpiece point cloud as the starting and ending points of the weld;

[0068] According to the direction vector from the starting point of the weld to the end point, the scanning basic posture of the direction vector of the Z axis of the robot end tool coordinate system parallel to the end point is calculated;

[0069] The scanning path points are discretized at equal intervals based on the length of the weld line segment. The basic posture is superimposed on the scanning path points to generate a scanning pose sequence, and the weld endpoint coordinates and scanning pose sequence instructions are output.

[0070] In an embodiment of the present invention, each three-dimensional point in the denoised point cloud data set is traversed, and the coordinates are corrected point by point through the posture compensation matrix: for any point (x, y, z) in the point cloud, its original coordinates are subjected to matrix multiplication with the posture compensation matrix (i.e., the coordinates are transformed by rotation and translation of the matrix) to obtain the corrected coordinates (x', y', z'); after the correction is completed, the point cloud is screened again: the average distance between each corrected point and the surrounding neighborhood points is calculated. If the distance exceeds a preset threshold (such as 1mm), it is determined to be a deviated point caused by a calculation error and is eliminated; finally, all retained points are integrated to generate a posture-corrected point cloud to ensure that the coordinates of each point can reflect the actual position after the fixture drifts.

[0071] The calculation process of fitting the plane equation based on the pose-corrected point cloud and solving the weld endpoint:

[0072] When fitting a plane equation using the Random Sampling Consensus Algorithm (RANSAC), three non-collinear points are randomly selected from the pose-corrected point cloud to construct the initial plane equation: Ax+By+Cz+D=0. The distances from all points in the point cloud to this plane are then calculated, and the number of inliers with distances less than a threshold (e.g., 0.5 mm) is counted. This iteration is repeated at least 50 times, and the plane with the largest number of inliers is selected as the reference plane for the workpiece's vertical and bottom surfaces. When the pose compensation matrix is ​​applied to the plane equation, the plane's normal vector (A, B, C) and constant term D are corrected through matrix transformation. Specifically, the rotational component of the compensation matrix adjusts the normal vector's direction, and the translational component corrects the constant term D to offset plane tilt or offset caused by fixture drift. The corrected vertical and bottom plane equations are combined to solve the intersection equation of the two planes. The points on the intersection line are traversed and combined with the physical boundaries of the workpiece (such as the intersection of the intersection line and the edge of the workpiece) to determine the starting point (x1, y1, z1) and end point (x2, y2, z2) of the weld. According to the direction vector of the line connecting the end points, the posture of the scanning camera is set (such as the optical axis is perpendicular to the intersection line, and the camera rotation angle is consistent with the direction of the intersection line) to ensure that the scanning range completely covers the weld.

[0073] The process of extracting the center line of the light strip according to the scanning instruction:

[0074] The captured laser image is a color image, containing three color channels: red, green, and blue. The pixel brightness value of each channel reflects the vividness of the corresponding color: the higher the red channel value, the more pronounced the red; the higher the green channel value, the brighter the green; and the higher the blue channel value, the more prominent the blue. During the conversion, weights are assigned based on the human eye's sensitivity to color: since green is the most sensitive color to human vision, the green channel has the highest weight; red is second; and blue has the lowest. At each pixel, the red channel's brightness value is converted according to the first weighting ratio, the green channel's brightness value is converted according to the second weighting ratio, and the blue channel's brightness value is converted according to the third weighting ratio. The three conversion results are then added together to obtain the pixel's grayscale value. For example, if a pixel has a red brightness of 100, a green brightness of 200, and a blue brightness of 50, the added weighting will result in a grayscale value closer to the green brightness, resulting in a single-channel grayscale image. The grayscale value of each pixel is calculated by weighting the three channels.

[0075] Separate the initial area of ​​the light bar:

[0076] In a grayscale image, the grayscale value of the laser light stripe region is significantly higher than that of the background (e.g., the light stripe grayscale value is approximately 200-255, while the background is approximately 0-50). A grayscale threshold is manually set (e.g., 150), and all pixels in the image are traversed: pixels with grayscale values ​​above the threshold are marked as light stripe candidates, and pixels with grayscale values ​​below the threshold are marked as background points. Morphological processing is performed on the candidate points: consecutive adjacent candidate points are checked. If a group of candidate points has fewer than 20 (considered isolated noise), the group is discarded. Candidate points with more than 20 consecutive points are retained to form a preliminary continuous light stripe region, which is then output as the light stripe candidate region.

[0077] Line-by-line scanning and center point coarse positioning:

[0078] Starting from the first row at the top of the grayscale image, scan down to the bottom row by row. In each row, check each pixel from left to right: if the pixel belongs to the light stripe candidate area, it is marked as a valid pixel. Within a row, continuous valid pixels form a light stripe segment, and the horizontal coordinates (horizontal position) of the starting and ending points of the segment are recorded; the middle position of the segment is calculated and used as the center point of the light stripe coarse positioning of the row, and the horizontal coordinates are recorded. After traversing all rows, the center points of each row are integrated to form a longitudinally arranged light stripe coarse positioning sequence, and the horizontal coordinates of the center point of each row and the basic posture (such as the angle between the optical axis and the horizontal direction) are output.

[0079] On the horizontal coordinates of the coarse positioning center point, candidate points are selected along the vertical direction (1 pixel above and below), the grayscale values ​​of these points are collected, and the point with the highest grayscale value is selected as the sub-pixel center point; the basic posture (such as the direction of the optical axis) is superimposed on each sub-pixel point to generate a scanning posture containing position and angle: the basic posture is superimposed on the endpoint coordinates to ensure that the scanning optical axis is aligned with the weld area; the scanning points are divided into equal intervals according to the length of the weld, and a scanning posture is generated at each point. Finally, the weld endpoint coordinates and the scanning posture sequence instructions are output.

[0080] The process of converting two-dimensional coordinates into three-dimensional point cloud:

[0081] For the two-dimensional coordinates (u, v) of the center point of the light strip, its pixel position in the camera image is first determined: the horizontal coordinate u corresponds to the column position in the horizontal direction of the image, and the vertical coordinate v corresponds to the row position in the vertical direction of the image. In combination with the camera's intrinsic parameter information (such as focal length, pixel size, etc.), the pixel coordinates are converted into ray directions in the camera coordinate system: starting from the camera's optical center, a spatial ray is extended along the direction corresponding to the pixel coordinate. This ray is intersected with the pre-calibrated light plane, and the intersection point is the three-dimensional spatial point corresponding to the two-dimensional coordinate. For each intersection point, the spatial posture of the light plane is superimposed: Based on the three-dimensional point, combined with the basic scanning posture (such as the angle between the camera optical axis and the horizontal direction, the rotation angle, etc.), a three-dimensional coordinate containing position and posture is generated; by traversing all two-dimensional coordinates, the corresponding three-dimensional point cloud is calculated and generated one by one, and finally integrated into a complete weld three-dimensional point cloud dataset.

[0082] The process of building topological relationships:

[0083] For the generated 3D weld point cloud, each point is selected as a target point. A fixed search radius (e.g., 5mm) is set around it. All other points within this radius are searched, forming the neighborhood point set for the target point. The spatial distribution direction of each point in the neighborhood point set is analyzed: if the neighborhood points are densely arranged in a certain direction and the angle between them and the target point varies little, this direction is considered the extension direction of the weld. If the neighborhood points form a clear angle turning point at the target point (e.g., the angle between the lines connecting adjacent points is greater than 30°), this point is marked as a weld feature point. All feature points are integrated and connected in the order of their arrangement in 3D space to form a continuous weld trajectory, ensuring that the trajectory fully reflects the spatial form of the weld.

[0084] Path planning process:

[0085] Using the starting point of the weld's 3D point cloud as the initial node and the end point as the target node, two exploration paths, forward and reverse, are established. In the forward path, a new node is generated in an unexplored direction, starting from the starting point. The distance between each new node and an existing node must be within a reasonable range (e.g., no more than 2 mm). The reverse path begins at the end point and continues in the same manner, extending toward the starting point. When the straight-line distance between a node in the forward and reverse paths is less than a set threshold (e.g., 3 mm), the two nodes are connected to form the initial path. This initial path is smoothed by checking whether three consecutive nodes are collinear. If so, the intermediate node is deleted. If a broken line exists, the turning angle is calculated. If the angle is less than 5°, the broken line is replaced with a straight line. During collision detection, a safe zone (e.g., a sphere with a radius of 5 mm) is defined with the welding torch tip as the center. The distance between this zone and the surrounding point cloud is checked one by one to ensure that no other objects are within the safe zone at all nodes. Finally, a planning instruction is generated, containing the position and posture of each node.

[0086] By fusing the pose compensation matrix with the plane equation, the tiny displacements of the fixture caused by thermal stress and vibration can be corrected in real time, avoiding the accumulation of global pose deviations and improving the long-term stability of weld positioning. The RANSAC algorithm is highly robust to noise points and can accurately extract weld endpoints when combined with pose correction. The scanning posture is planned based on the corrected weld trajectory to ensure that the line structured light camera is always aligned with the weld area, reducing scan omissions or redundant data and improving the effectiveness of light bar center extraction. Through 3D point cloud reconstruction and topological relationship construction, the detailed features of the weld are accurately captured, providing high-precision data support for path planning. The improved algorithm's bidirectional growth and greedy strategy can quickly generate a smooth, collision-free path, reducing the impact of robot movement, and combined with the collision detection model to further ensure the safety of the welding process.

[0087] In a preferred embodiment of the present invention, according to the scanning posture instruction, the robot is controlled to drive the line structured light scanning camera to move along the correction weld trajectory, collect the line laser image sequence and extract the center line of the light strip, and output the two-dimensional coordinate set of the center point of the light strip, which may include:

[0088] Based on the spatial posture data in the scanned posture sequence, combined with the preset structural parameters of the robot body, inverse kinematics solution is performed to generate a joint angle instruction set;

[0089] The joint angle instruction set is input into the robot control platform to drive the robot end to move according to the instruction; when the robot end reaches each scanning posture, the position arrival signal triggers the line structured light camera to collect the original image;

[0090] The original image captured by the camera is grayscaled using a fixed weight coefficient to generate a grayscale image. On the grayscale image, the connected area is expanded based on the grayscale similarity threshold of the neighboring pixels, starting from the pixel with the maximum grayscale value, to form a binary mask of the light stripe area.

[0091] Scan pixels row by row along the normal direction of the binary mask in the light stripe area, calculate the horizontal gradient amplitude, and locate the pixel coordinates with the largest gradient amplitude in each row as the coarse positioning center point;

[0092] Fit the local surface function with the coarse positioning center point as the center, calculate the sub-pixel center offset, and superimpose the pixel coordinates and sub-pixel offset to generate the light bar center coordinates. Specifically, it includes:

[0093] Based on the coarse positioning center point, the grayscale value distribution data of the surrounding 3×3 pixel area is obtained;

[0094] Construct a two-dimensional quadratic surface function model based on gray value distribution data;

[0095] The coefficient parameters of the surface function are determined by least square fitting of grayscale values, and the coordinates of the extreme points of the surface function are calculated as the sub-pixel center position;

[0096] Perform vector superposition of the sub-pixel center position and the coarse positioning center point coordinates to generate the final light stripe center sub-pixel coordinates, and output the sub-pixel coordinates of all light stripe centers in the current frame in the order of image row scanning;

[0097] The coordinates of the center points of each frame are integrated according to the execution order of the scanning posture to form a two-dimensional coordinate sequence set.

[0098] In an embodiment of the present invention, after receiving a scanning posture instruction, the robot parses the spatial coordinates and angle parameters in the instruction, converts these parameters into the rotation angles of each joint through an internal motion control module, and drives the end of the robot to drive the line structured light scanning camera to move along the corrected weld trajectory; during the movement, every time the robot reaches a preset path point (such as a point every 2 mm), it triggers the camera's shooting switch to capture the line laser image of that position; at the same time, it monitors in real time whether the end position is consistent with the reference position in the instruction. If there is any deviation, it is corrected by fine-tuning the joint angle to ensure the accuracy of the image acquisition position.

[0099] Image preprocessing and grayscale:

[0100] The collected color laser images are first preprocessed: the highlight spots caused by lens reflections are removed (the spot area is manually marked and filled with the average grayscale value of the surrounding pixels), and the invalid areas at the edge of the image are cropped (such as pixels within 10 pixels of the edge). After that, the image is converted into a grayscale image. By comparing the brightness of the red, green, and blue channels, the high grayscale value of the light strip area (the laser light strip has the highest brightness in the green channel) is retained, and the low grayscale value of the background area is weakened, making the grayscale difference between the light strip and the background more significant.

[0101] Light strip area segmentation and boundary extraction:

[0102] On the grayscale image, grayscale thresholds are set for the light stripes and background (e.g., light stripe grayscale ≥ 200, background ≤ 50). Regions above the threshold are marked as light stripe candidates. Morphological processing is performed on these candidate regions: adjacent small regions are merged (e.g., regions with a distance ≤ 2 pixels are merged), and isolated points with an area ≤ 10 pixels are deleted to form a continuous light stripe mask. The mask is scanned row by row, recording the left edge (the first pixel with a grayscale value ≥ 200) and right edge (the last pixel with a grayscale value ≥ 200) of the light stripe in each row. The midpoint of the left and right edges is calculated as the coarse positioning center point for that row.

[0103] Sub-pixel center point extraction:

[0104] Within the pixel range of the roughly located center point (such as the pixel and 1 pixel above, below, left, and right), grayscale value distribution data is collected. By analyzing the grayscale change trend (high in the middle, low on both sides), the sub-pixel position with the highest grayscale value (such as 0.3 pixels above the pixel center) is found. The sub-pixel points of all rows are arranged in sequence to form the center point coordinates including position deviations, ensuring that each point falls precisely on the center line of the light strip, and finally outputting a complete two-dimensional coordinate set.

[0105] The robot moves along a preset trajectory, correcting deviations in real time to ensure accurate image acquisition. Highlights and edge noise are removed, and grayscale processing enhances the contrast between the light strip and the background. Morphological processing eliminates isolated noise, and boundary midpoint calculation ensures accurate centerline positioning, laying the foundation for sub-pixel extraction. Sub-pixel center point superposition and deviation correction keep the light strip center positioning error within 0.1mm, meeting high-precision welding requirements.

[0106] like Figure 2 As shown, in another preferred embodiment of the present invention, based on the two-dimensional coordinate set of the center point of the light strip, combined with the light plane calibration parameters, a three-dimensional point cloud is reconstructed, and by constructing a topological relationship and extracting weld feature points, a three-dimensional point cloud of the weld is output, which may include:

[0107] Receive the sub-pixel coordinates of all light strip center points, combine them with the pre-calibrated line structured light plane equation parameters, and convert each two-dimensional coordinate into the corresponding spatial ray equation;

[0108] Solve the spatial ray equation and the light plane equation simultaneously to calculate the coordinates of the three-dimensional intersection points; integrate all the three-dimensional intersection coordinates according to the scanning sequence to generate the initial three-dimensional point cloud of the weld;

[0109] Based on the spatial distribution of the initial weld 3D point cloud, a 3D binary index tree structure is constructed. The 3D binary index tree is used to accelerate the neighborhood search and calculate the normal vector direction of each point within the preset radius neighborhood. Specifically, the following steps are performed:

[0110] Receive the initial weld 3D point cloud, recursively segment the point cloud along the spatial coordinate axis, calculate the position variance of the point cloud along each coordinate axis, and select the coordinate axis with the largest variance as the segmentation dimension;

[0111] Based on the segmentation dimension, the median point of the segmentation dimension is used as the segmentation plane. The point cloud in front of the segmentation plane is stored in the left subtree, and the point cloud behind the plane is stored in the right subtree. Each subtree is recursively processed until the leaf node to generate a three-dimensional binary index tree.

[0112] Based on the three-dimensional binary index tree, the target point coordinates and search radius parameters are input, and the neighborhood point set within the radius sphere is located by recursively traversing from the root node.

[0113] Calculate the three-dimensional coordinate covariance matrix of the neighborhood point set, solve the eigenvalues ​​and eigenvectors of the covariance matrix, and take the eigenvector corresponding to the minimum eigenvalue as the normal vector direction;

[0114] Calculate the spatial angle between the normal vector of each point and the average normal vector of the neighborhood, delete abnormal points with angles exceeding 25°, and output the optimized complete weld 3D point cloud.

[0115] In this embodiment of the present invention, the sub-pixel coordinates of the centers of all light stripes are received (including the precise positions in the horizontal and vertical directions, such as (u+0.3, v-0.2), where 0.3 and -0.2 are sub-pixel offsets). Combined with the pre-calibrated parameters of the line structured light plane equation (such as the tilt angle of the light plane in space, its relative position to the camera coordinate system, etc.), a corresponding spatial ray equation is generated for each two-dimensional coordinate:

[0116] Taking the camera's optical center as the starting point of the ray (fixed as the origin of the camera coordinate system), the direction of the ray is determined according to the horizontal and vertical positions of the two-dimensional coordinates and the camera's intrinsic parameters (such as focal length and pixel size): the horizontal coordinate determines the deflection angle of the ray in the horizontal direction, and the vertical coordinate determines the deflection angle in the vertical direction. Ultimately, a ray equation is formed that starts from the camera's optical center and points to the spatial direction corresponding to the two-dimensional coordinate.

[0117] Solve the simultaneous equations to find the 3D intersection and generate the initial point cloud:

[0118] Combine each spatial ray equation with the light plane equation and calculate the coordinates of the intersection between the two through geometric relationships:

[0119] The ray extends in a fixed direction, while the light plane assumes a fixed position in space (e.g., a 30° tilt). By determining the spatial relationship between the ray and the light plane, a unique intersection point (i.e., the point where the ray crosses the light plane) is found. This intersection point is the 3D spatial point corresponding to the 2D coordinates. The coordinates of all 3D intersection points are arranged sequentially according to the robot's scanning sequence (e.g., from the weld start point to the weld end point), forming a continuous initial 3D weld point cloud. The points in the point cloud are distributed according to the scanning path, reflecting the spatial shape of the weld.

[0120] Construct a three-dimensional binary index tree:

[0121] Based on the spatial distribution of the initial weld 3D point cloud, an index tree is constructed through recursive segmentation to accelerate neighborhood search:

[0122] First, calculate the position variance of the point cloud on the three coordinate axes x, y, and z: the larger the variance, the more dispersed the points are on that axis (for example, the x-axis has the largest variance, indicating that the points are more dispersed in the horizontal direction). Select the coordinate axis with the largest variance as the dimension for the first segmentation (for example, the x-axis).

[0123] Use the median of all point coordinates along that dimension as the boundary and create a splitting plane perpendicular to that axis (e.g., if the median of the x-axis is x = 50 mm, then the splitting plane is the vertical plane at x = 50 mm). Store points on one side of the plane (e.g., x < 50 mm) in the left subtree, and points on the other side (x > 50 mm) in the right subtree.

[0124] Repeat the above operation for the left and right subtrees: calculate the variance of the points in the subtree on the three axes, select the axis with the largest variance and split it again until the number of points in the subtree is too small to split further (for example, each subtree contains only 10 points or less), forming leaf nodes, and finally generating a complete three-dimensional binary index tree.

[0125] Use the three-dimensional binary index tree to quickly locate the neighborhood points of each point and calculate the normal vector:

[0126] For each target point in the point cloud, input its coordinates and preset search radius (such as 5mm), and start traversal from the root node of the index tree: determine whether the search sphere of the target point (a sphere with the target point as the center and a radius of 5mm) intersects with the current segmentation plane. If so, traverse both the left and right subtrees at the same time. If not, only traverse the subtree containing the sphere. Finally, collect all the points in the sphere to form the neighborhood point set of the target point.

[0127] Calculate the covariance matrix of the neighborhood point set: By analyzing the distribution relationship of the neighborhood points on the x, y, and z axes (for example, if the points are densely arranged in a certain direction, the covariance value in that direction is large), solve the eigenvalues ​​and eigenvectors of the covariance matrix; among them, the direction of the eigenvector corresponding to the minimum eigenvalue is the normal vector of the local plane where the point is located (because the minimum eigenvalue corresponds to the direction with the sparsest distribution, that is, the direction perpendicular to the plane).

[0128] Improve point cloud quality by filtering valid points through normal vector consistency:

[0129] The spatial angle between the normal vector of each point and the average normal vector of the neighboring points is calculated (such as the angle between the normal vector of point A and the average normal vector of the surrounding 50 points). If the angle exceeds 25°, it means that the local plane direction of the point is too different from the surrounding area (it may be a noise point), and the point is removed. If the angle is ≤25°, the point is retained. All retained points are integrated to form an optimized complete weld 3D point cloud. The point distribution in the point cloud is continuous and the normal vectors are consistent, accurately reflecting the spatial characteristics of the weld.

[0130] By calculating the intersection of rays and light planes, 2D coordinates are precisely converted to 3D points, ensuring minimal deviation between the point cloud coordinates and the actual weld location. A 3D binary index tree reduces the computational complexity of neighborhood searches, avoiding redundant point-by-point traversals and rapidly locating neighborhood points, improving processing speed. Based on covariance matrix analysis of local point distribution, the normal vector direction truly reflects the orientation of the weld surface, providing a precise directional basis for feature point identification. Points with abnormal normal vectors are eliminated to reduce noise interference, making the optimized point cloud continuous and consistent, while fully preserving the spatial characteristics of the weld.

[0131] In a preferred embodiment of the present invention, based on the three-dimensional point cloud of the weld, an improved star-shaped rapid expanding tree algorithm is used to implement a bidirectional growth search, and a greedy strategy is used to optimize the path smoothness to output a collision-free welding path instruction. Executing the welding operation according to the welding path instruction in combination with the robot collision detection model may include:

[0132] Extract the first and last endpoint coordinates of the weld 3D point cloud as the starting and ending points of the welding path;

[0133] Based on the coordinates of the starting and ending points, the forward exploration tree is initialized with the starting point as the root node, and the reverse exploration tree is initialized with the end point as the root node to form a bidirectional exploration tree;

[0134] Based on a bidirectional exploration tree structure, the regional expansion priority weight is calculated according to the spatial density distribution of the point cloud, and the dual-tree nodes are driven to expand synchronously according to the priority. When the Euclidean distance between the dual-tree nodes is less than the connection threshold, the initial welding path is generated by connection.

[0135] Based on the initial welding path, a merging operation is performed on the continuous collinear nodes to generate a simplified welding path;

[0136] Based on the simplified welding path, the joint angle of each node is solved by the robot inverse kinematics to verify whether the joint angle exceeds the physical motion limit of the robot;

[0137] Based on the nodes that have passed the reachability verification, a safety sphere space is constructed with the welding gun end as the center. The minimum spatial distance between the safety sphere and the environmental point cloud is calculated. All continuous nodes that are reachable and have no collision risk are screened to form an executable path segment.

[0138] Based on the executable path segments, time parameterized interpolation processing is performed to generate a time-space-joint angle control instruction sequence;

[0139] Based on the spatial trajectory of the control instruction sequence, the reference value of the welding parameters is queried in combination with the workpiece material, the spatial curvature of the path is analyzed, and the welding parameters are dynamically corrected to obtain the corrected welding parameters;

[0140] Based on the corrected welding parameters and control instruction sequence, the robot is driven to move, and the welding operation is triggered synchronously when the welding gun reaches the path point.

[0141] In an embodiment of the present invention, the first and last endpoints are extracted from the three-dimensional point cloud of the weld as the starting point and end point of the path: along the scanning sequence of the point cloud (such as the order from the start to the end of the robot scanning), the first point is selected as the starting point and the last point is selected as the end point; if there are branches in the point cloud, the two endpoints of the longest continuous line segment are determined by analyzing the spatial connection relationship of the points (such as the distance between consecutive points ≤ 0.5 mm is a valid connection) to ensure that the starting and ending points cover the entire weld.

[0142] Initialize the bidirectional exploration tree:

[0143] The forward exploration tree is initialized with the starting point as the root node, and the root node stores the three-dimensional coordinates and initial posture of the starting point (such as the welding gun is perpendicular to the weld surface); the reverse exploration tree is initialized with the end point as the root node, and the root node stores the three-dimensional coordinates and end posture of the end point; both trees use a linked list structure, and each node contains its own coordinates, parent node index (recording the connection relationship with the previous node) and expansion priority (initial value is 0).

[0144] Bidirectional growth search and initial path generation:

[0145] Calculate expansion priority: Count the point cloud density (e.g., number of points per cubic millimeter) of each region in the weld's 3D point cloud. Regions with higher density (e.g., denser points at weld corners) receive a higher priority weight (e.g., a weight of 1.2 for a density of 5 points / cubic millimeter, and 0.8 for a density of 2 points / cubic millimeter). Each node's priority is calculated by multiplying the region's density weight by the distance weight from the root node (closer distances increase the weight).

[0146] Dual-tree synchronous expansion: Each iteration selects the node with the highest priority from both the forward tree and the reverse tree, randomly generates new sampling points around the node (the sampling range is within 5 mm around the node), calculates the Euclidean distance between the sampling point and all nodes in the tree, selects the node with the closest distance as the parent node, adds the sampling point to the corresponding tree, and updates its priority.

[0147] Connect to generate the initial path: Continue to expand until the Euclidean distance between a node in the forward tree and a node in the reverse tree is ≤3mm (connection threshold). At this time, connect the two nodes and trace back to the root node along the parent node index to form the initial welding path from the starting point to the end point.

[0148] Simplified welding path:

[0149] Traverse the consecutive nodes of the initial path and check the spatial position relationship of the three adjacent nodes: calculate the vector from the first node to the second node, and the vector from the second node to the third node. If the angle between the two vectors is 180° (collinear), delete the middle node; if the angle is close to 180° (such as 175°-185°), replace the broken line with a straight line and retain the nodes at both ends; by merging collinear nodes, reduce redundant nodes in the path and generate a simplified path.

[0150] Joint angle verification (reachability check):

[0151] Based on the robot's structural parameters (such as the length of each joint and the direction of the rotation axis), calculate the angle that each joint needs to rotate (such as the waist joint rotation angle, the upper arm joint lifting angle, etc.) when the end reaches the node coordinates; compare the solved angle with the robot's physical motion limits (such as the waist joint rotation range of -180° to 180°, and the upper arm joint lifting range of 0° to 90°). If all joint angles are within the limit, the node passes the reachability verification; if a joint angle exceeds the limit, adjust the node coordinates (such as fine-tuning 0.2mm along the path) and recalculate until it passes.

[0152] Collision detection and executable path segment generation:

[0153] For nodes that pass reachability verification, a safety sphere with a radius of 5mm (the size is determined by the welding torch diameter plus a safety margin) is constructed with the welding torch tip as the center. The minimum spatial distance between the safety sphere and the surrounding point cloud (workpiece, fixture, workbench, etc.) is calculated. If the distance is ≥1mm (safety threshold), the node is considered collision-free. If the distance is <1mm, the node is marked as a dangerous node and removed. All consecutive nodes without collision risk are integrated to form an executable path segment, in which the nodes in the path segment are continuous and without jumps.

[0154] Time parameterized interpolation:

[0155] Interpolation points are inserted at equal time intervals (e.g., 0.1 seconds) between adjacent nodes of an executable path segment. The number of interpolation points is calculated based on the spatial distance between the two nodes and the preset motion speed (e.g., 5 mm / s) (e.g., 10 points are inserted if the distance is 10 mm). The coordinates of each interpolation point are obtained through linear interpolation (e.g., from node A to node B, the coordinate of the nth interpolation point = A coordinate + (B coordinate - A coordinate) × n / total number of points). The joint angle corresponding to each interpolation point is also calculated (through interpolation of the joint angles of adjacent nodes), generating a control instruction sequence consisting of "time-space coordinate-joint angle" to ensure continuous and smooth robot motion.

[0156] Dynamic correction of welding parameters:

[0157] A "workpiece material - welding parameter" database is pre-established. The database stores corresponding benchmark parameters by material category (e.g., for steel, a benchmark current of 150A, a benchmark voltage of 25V, and a benchmark welding speed of 10mm / s is stored; for aluminum, a benchmark current of 120A, a benchmark voltage of 20V, and a benchmark welding speed of 8mm / s is stored). Based on the material of the workpiece to be welded (e.g., if it is low-carbon steel as determined by preliminary testing), the corresponding benchmark parameters are retrieved from the database.

[0158] Pick adjacent interpolation points and calculate the direction vector:

[0159] Three consecutive interpolation points (denoted as point A, point B, and point C) are selected in sequence from the control instruction sequence. These three points are distributed along the welding path, and the spatial distance between two adjacent points is equal (for example, both are 0.5 mm, corresponding to a time interval of 0.1 second).

[0160] Calculate the direction vector from point A to point B: Take point A as the starting point and point B as the end point, and get vector AB (reflecting the spatial direction from A to B);

[0161] Calculate the direction vector from point B to point C: Take point B as the starting point and point C as the end point to obtain vector BC (reflecting the spatial direction from B to C).

[0162] Calculate the path space curvature:

[0163] First, calculate the spatial angle between vectors AB and BC: By comparing the directions of the two vectors, if AB points horizontally to the right and BC tilts upward and to the right, the angle is the deflection angle between the two vectors (e.g., 30°). Then, combine the distance between adjacent points to calculate the curvature: curvature = angle ÷ distance between adjacent points (e.g., if the angle is 30° and the distance between adjacent points is 0.5mm, then curvature = 30° ÷ 0.5mm = 60° / mm). The larger this value, the more pronounced the curvature of the path at point B (e.g., corners typically have greater curvature, while straight segments have near-zero curvature).

[0164] Correction of welding parameters according to curvature:

[0165] Set the curvature threshold to 0.5° / mm and compare the calculated curvature with the threshold:

[0166] If the curvature is ≥ 0.5° / mm (e.g. at a corner, the curvature = 60° / mm):

[0167] Welding speed correction: Based on the reference speed of 10 mm / s, reduce it by 20%, that is, 10 mm / s - (10 mm / s × 20%) = 8 mm / s;

[0168] Welding current correction: Based on the reference current of 150A, increase by 5%, that is, 150A + (150A × 5%) = 157.5A;

[0169] The welding voltage remains at the base value of 25V (or is fine-tuned in proportion to the current change, such as increasing by 5% to 26.25V).

[0170] If the curvature is less than 0.5° / mm (e.g., for a straight line segment, the curvature is 0.3° / mm): Keep all parameters at the baseline values ​​(current 150A, voltage 25V, speed 10mm / s).

[0171] Each interpolation point corresponds to a curvature value (calculated by its adjacent points before and after it), and the corrected parameters (or benchmark parameters) are bound to the control instructions of the interpolation point:

[0172] In the control instruction sequence, in addition to the "time-space coordinates-joint angle" information, each interpolation point also has an additional "corresponding welding current, voltage, and speed" field. For example, if the curvature of point B is ≥0.5° / mm, the current in the corresponding instruction field is recorded as 157.5A and the speed is recorded as 8mm / s. If the curvature of point A is <0.5° / mm, the field is recorded as the baseline parameter. Finally, a complete instruction sequence of "time-space coordinates-joint angles-welding parameters" is formed to ensure that when the robot reaches each interpolation point, the welding equipment can perform operations according to the matching parameters.

[0173] To perform a welding operation:

[0174] The control instruction sequence and corrected welding parameters are sent to the robot controller: the controller drives the movement of each joint in chronological order and provides real-time feedback on the end position (error ≤ 0.1mm); when the welding gun reaches each interpolation point on the path, the welding switch is triggered synchronously (for example, the current and voltage are output according to the corrected parameters) to complete the welding at that point; the robot status is monitored throughout the process. If any abnormality occurs (such as the joint angle exceeds the limit), the welding is stopped immediately and an alarm is issued.

[0175] Bidirectional growth search combined with priority weights can quickly find a path that covers the entire weld, reducing blind expansion and improving planning efficiency. The combined use of collinear nodes and collision detection makes the path concise and free of redundancy, avoiding interference with the environment and ensuring stable robot motion. Joint angle verification ensures that the path complies with the robot's physical limits, avoiding motion freezes or failures and improving operational reliability. Dynamic correction of welding parameters to adapt to path curvature ensures uniform welds on straight sections and corners, reducing welding defects. No human intervention is required from path planning to parameter correction, adapting to different weld seam shapes and improving the welding adaptability of small batches of multi-variety workpieces.

[0176] An embodiment of the present invention further provides a computing device comprising: a processor and a memory storing a computer program, wherein the computer program, when executed by the processor, executes the system described above. All implementations in the above system embodiments are applicable to this embodiment and can achieve the same technical effects.

[0177] The embodiment of the present invention further provides a computer-readable storage medium storing instructions, which, when executed on a computer, causes the computer to execute the system described above. All implementations in the above system embodiments are applicable to this embodiment and can achieve the same technical effects.

[0178] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A teaching-free welding robot path planning system based on visual autonomous learning, characterized in that: include: The posture compensation module is used to arrange three infrared reflective markers in an L shape on the surface of the fixture base, use a stereo camera to collect the original point cloud of the markers and the workpiece, and pre-process the point cloud; Generate a pose compensation matrix based on the offset between the actual coordinates and theoretical coordinates of the marker points, and output a denoised point cloud dataset and compensation matrix; The pose correction module is used to fit the workpiece plane equation based on the denoised point cloud dataset using a random sampling consistency algorithm, and then applies the pose compensation matrix to the workpiece plane equation to correct the pose error caused by fixture drift, calculate the coordinates of the weld trajectory endpoints, and output the corrected weld pose parameters and scanning pose instructions; The extraction module is used to control the robot to drive the line structured light scanning camera to move along the correction weld trajectory according to the scanning posture instruction, collect the line laser image sequence and extract the center line of the light strip, and output the two-dimensional coordinate set of the center point of the light strip; The point cloud reconstruction module is used to reconstruct a 3D point cloud based on the 2D coordinate set of the center point of the light strip and the light plane calibration parameters. It outputs a 3D point cloud of the weld by constructing topological relationships and extracting weld feature points. The path planning module is used to implement a bidirectional growth search based on the three-dimensional point cloud of the weld using an improved star-shaped rapid expanding tree algorithm, optimize the path smoothness through a greedy strategy, and output collision-free welding path instructions; according to the welding path instructions, the welding operation is performed in combination with the robot collision detection model.

2. The teaching-free welding robot path planning system based on visual autonomous learning according to claim 1 is characterized in that: Three infrared reflective marking points are arranged in an L shape on the surface of the fixture base. A stereo camera is used to collect the original point cloud of the marking points and the workpiece, and the point cloud is pre-processed. Generate a pose compensation matrix based on the offset between the actual coordinates and theoretical coordinates of the markers, and output a denoised point cloud dataset and compensation matrix, including: Based on the spatial range of the fixture coordinate system, the original point cloud is filtered directly to intercept the valid area point cloud containing the workpiece and the marking points; Perform voxel grid processing on the point cloud of the valid area to generate a downsampled point cloud; Compute the distance distribution characteristics of each point in the neighborhood of the downsampled point cloud to generate a denoised point cloud dataset; In the denoised point cloud dataset, the marker points are identified by the reflection intensity threshold and the measured 3D coordinate values ​​are extracted. The theoretical coordinates of the marker points are preset and the least squares method is used to calculate the rigid body transformation parameters from the measured coordinates to the theoretical coordinates. The rigid body transformation parameters are converted into a 4×4 homogeneous transformation matrix as the pose compensation matrix, and the denoised point cloud dataset and pose compensation matrix are output.

3. The teaching-free welding robot path planning system based on visual autonomous learning according to claim 2 is characterized in that: Based on the denoised point cloud dataset, a random sampling consistency algorithm is used to fit the workpiece plane equation. The pose compensation matrix is ​​then applied to the workpiece plane equation to correct the pose error caused by fixture drift. The coordinates of the weld trajectory endpoints are calculated, and the corrected weld pose parameters and scanning pose instructions are output, including: The pose compensation matrix is ​​fused with the denoised point cloud dataset to achieve point cloud coordinate space transformation and generate pose-corrected point cloud; Based on the pose-corrected point cloud, the random sampling consistency algorithm is used to fit the vertical plane equation of the workpiece, and based on the same corrected point cloud, the bottom plane equation of the workpiece is fitted; Solve the spatial intersection equation by simultaneously solving the vertical plane equation and the bottom plane equation, and take the two intersection points of the spatial intersection line and the actual boundary of the workpiece point cloud as the starting and ending points of the weld; According to the direction vector from the starting point of the weld to the end point, the scanning basic posture of the direction vector of the Z axis of the robot end tool coordinate system parallel to the end point is calculated; The scanning path points are discretized at equal intervals based on the length of the weld line segment. The basic posture is superimposed on the scanning path points to generate a scanning pose sequence, and the weld endpoint coordinates and scanning pose sequence instructions are output.

4. The teaching-free welding robot path planning system based on visual autonomous learning according to claim 3 is characterized in that: According to the scanning posture instruction, the robot is controlled to drive the line structured light scanning camera to move along the correction weld trajectory, collect the line laser image sequence and extract the center line of the light strip, and output the two-dimensional coordinate set of the center point of the light strip, including: Based on the spatial posture data in the scanned posture sequence, combined with the preset structural parameters of the robot body, inverse kinematics solution is performed to generate a joint angle instruction set; The joint angle instruction set is input into the robot control platform to drive the robot end to move according to the instruction; when the robot end reaches each scanning posture, the position arrival signal triggers the line structured light camera to collect the original image; The original image captured by the camera is grayscaled using a fixed weight coefficient to generate a grayscale image. On the grayscale image, the connected area is expanded based on the grayscale similarity threshold of the neighboring pixels, starting from the pixel with the maximum grayscale value, to form a binary mask of the light stripe area. Scan pixels row by row along the normal direction of the binary mask in the light stripe area, calculate the horizontal gradient amplitude, and locate the pixel coordinates with the largest gradient amplitude in each row as the coarse positioning center point; Fit the local surface function with the coarse positioning center point as the center, calculate the sub-pixel center offset, and superimpose the pixel coordinates and sub-pixel offset to generate the light strip center coordinates; The coordinates of the center points of each frame are integrated according to the execution order of the scanning posture to form a two-dimensional coordinate sequence set.

5. The teaching-free welding robot path planning system based on visual autonomous learning according to claim 4 is characterized in that: Fit the local surface function with the coarse positioning center point as the center, calculate the sub-pixel center offset, and superimpose the pixel coordinates and sub-pixel offset to generate the light bar center coordinates, including: Based on the coarse positioning center point, the grayscale value distribution data of the surrounding 3×3 pixel area is obtained; Construct a two-dimensional quadratic surface function model based on gray value distribution data; The coefficient parameters of the surface function are determined by least square fitting of grayscale values, and the coordinates of the extreme points of the surface function are calculated as the sub-pixel center position; Perform vector superposition of the sub-pixel center position and the coarse positioning center point coordinates to generate the final light stripe center sub-pixel coordinates, and output the sub-pixel coordinates of all light stripe centers in the current frame in the order of image row scanning.

6. The teaching-free welding robot path planning system based on visual autonomous learning according to claim 5 is characterized in that: Based on the 2D coordinate set of the center point of the light strip, the 3D point cloud is reconstructed in combination with the light plane calibration parameters. By building the topological relationship and extracting the weld feature points, the 3D point cloud of the weld is output, including: Receive the sub-pixel coordinates of all light strip center points, combine them with the pre-calibrated line structured light plane equation parameters, and convert each two-dimensional coordinate into the corresponding spatial ray equation; Solve the spatial ray equation and the light plane equation simultaneously to calculate the coordinates of the three-dimensional intersection points; integrate all the three-dimensional intersection coordinates according to the scanning sequence to generate the initial three-dimensional point cloud of the weld; Based on the spatial distribution of the initial weld 3D point cloud, a 3D binary index tree structure is constructed. The 3D binary index tree is used to accelerate the neighborhood search and calculate the normal vector direction of each point within the preset radius neighborhood. Calculate the spatial angle between the normal vector of each point and the average normal vector of the neighborhood, delete abnormal points with angles exceeding 25°, and output the optimized complete weld 3D point cloud.

7. The teaching-free welding robot path planning system based on visual autonomous learning according to claim 6 is characterized in that: Based on the spatial distribution of the initial weld 3D point cloud, a 3D binary index tree structure is constructed. The 3D binary index tree is used to accelerate the neighborhood search and calculate the normal vector direction of each point within the preset radius neighborhood, including: Receive the initial weld 3D point cloud, recursively segment the point cloud along the spatial coordinate axis, calculate the position variance of the point cloud along each coordinate axis, and select the coordinate axis with the largest variance as the segmentation dimension; Based on the segmentation dimension, the median point of the segmentation dimension is used as the segmentation plane. The point cloud in front of the segmentation plane is stored in the left subtree, and the point cloud behind the plane is stored in the right subtree. Each subtree is recursively processed until the leaf node to generate a three-dimensional binary index tree. Based on the three-dimensional binary index tree, the target point coordinates and search radius parameters are input, and the neighborhood point set within the radius sphere is located by recursively traversing from the root node. Calculate the three-dimensional coordinate covariance matrix of the neighborhood point set, solve the eigenvalues ​​and eigenvectors of the covariance matrix, and take the eigenvector corresponding to the minimum eigenvalue as the normal vector direction.

8. The teaching-free welding robot path planning system based on visual autonomous learning according to claim 7 is characterized in that: Based on the 3D point cloud of the weld, an improved star-shaped rapid expansion tree algorithm is used to implement a bidirectional growth search. Path smoothness is optimized through a greedy strategy to output collision-free welding path instructions. Based on the welding path instructions, the robot collision detection model is combined to execute welding operations, including: Extract the first and last endpoint coordinates of the weld 3D point cloud as the starting and ending points of the welding path; Based on the coordinates of the starting and ending points, the forward exploration tree is initialized with the starting point as the root node, and the reverse exploration tree is initialized with the end point as the root node to form a bidirectional exploration tree; Based on a bidirectional exploration tree structure, the regional expansion priority weight is calculated according to the spatial density distribution of the point cloud, and the dual-tree nodes are driven to expand synchronously according to the priority. When the Euclidean distance between the dual-tree nodes is less than the connection threshold, the initial welding path is generated by connection. Based on the initial welding path, a merging operation is performed on the continuous collinear nodes to generate a simplified welding path; Based on the simplified welding path, the joint angle of each node is solved by the robot inverse kinematics to verify whether the joint angle exceeds the physical motion limit of the robot; Based on the nodes that have passed the reachability verification, a safety sphere space is constructed with the welding gun end as the center. The minimum spatial distance between the safety sphere and the environmental point cloud is calculated. All continuous nodes that are reachable and have no collision risk are screened to form an executable path segment. Based on the executable path segments, time parameterized interpolation processing is performed to generate a time-space-joint angle control instruction sequence; Based on the spatial trajectory of the control instruction sequence, the reference value of the welding parameters is queried in combination with the workpiece material, the spatial curvature of the path is analyzed, and the welding parameters are dynamically corrected to obtain the corrected welding parameters; Based on the corrected welding parameters and control instruction sequence, the robot is driven to move, and the welding operation is triggered synchronously when the welding gun reaches the path point.

9. A computing device, characterized in that include: one or more processors; A storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the system according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a program, which, when executed by a processor, implements the system according to any one of claims 1 to 8.

Images