Welding robot path planning system cooperating with industrial vision and collision detection

Abstract

The invention discloses a welding robot path planning system cooperating with industrial vision and collision detection, and relates to the technical field of robot automatic welding, the welding robot path planning system comprises an upper computer, the upper computer is in communication connection with the following modules: an environment sensing module used for collecting welding environment information in real time, comprising three-dimensional point cloud data of a workpiece and obstacle information; and the data processing and fusion module is used for fusing the visual identification result and the obstacle detection data. By integrating industrial visual recognition and collision detection technologies, the dynamic change of the welding environment can be sensed in real time, an accurate environment model is constructed based on three-dimensional point cloud data, compared with a traditional welding robot with a preset path, the welding track can be dynamically adjusted, welding defects caused by environment interference are avoided, and meanwhile the welding efficiency is improved. In combination with an assembly error compensation mechanism, the deviation between a theoretical model and an actual workpiece can be corrected, the welding position precision is ensured, and the high-precision ship manufacturing requirement is met.

Description

Technical Field

[0001] This invention relates to the field of robotic automated welding technology, and more specifically to a welding robot path planning system that integrates industrial vision and collision detection. Background Technology

[0002] With the expansion of shipbuilding scale and the increase in complexity, modern ships have increasingly higher requirements for welding quality. They not only need to meet shipbuilding standards, but also relevant national standards. The application of welding robot systems can significantly improve welding production efficiency. Through automated and intelligent welding path planning, non-welding time, such as workpiece handling, assembly and adjustment, is reduced, thereby improving overall production efficiency.

[0003] In existing technologies, current welding robot path planning systems mostly rely on preset programs and fixed paths, making it difficult to adapt to changes in workpiece size, shape, and assembly errors in actual production. Moreover, static path planning lacks real-time environmental perception and obstacle avoidance capabilities, which not only leads to reduced efficiency or welding defects in complex or changing welding environments but also poses collision risks. Therefore, how to integrate industrial vision recognition results and collision detection data and incorporate them into path planning algorithms so that the robot can generate feasible paths in real time based on the actual welding environment and obstacle distribution is the problem that this invention aims to solve. To this end, a welding robot path planning system that integrates industrial vision and collision detection is proposed. Summary of the Invention

[0004] The purpose of this invention is to provide a welding robot path planning system that integrates industrial vision and collision detection to solve the problems mentioned in the background art.

[0005] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows:

[0006] A welding robot path planning system that integrates industrial vision and collision detection, comprising a host computer, characterized in that the host computer is communicatively connected to the following modules, wherein:

[0007] The environmental perception module is used to collect welding environment information in real time, including the three-dimensional point cloud data of the workpiece and obstacle information, and to provide the basic data required for path planning.

[0008] The data processing and fusion module is used to fuse visual recognition results and obstacle detection data to form a unified environment model;

[0009] The path planning and optimization module is used to analyze the visual recognition results and obstacle detection data contained in the environmental model, and generate feasible welding paths in combination with welding process requirements;

[0010] The multi-axis collaborative control module is used to decompose the planned path into motion commands for each axis of the robot based on the robot's joint kinematics model, convert them into control signals that the robot can recognize, drive the robot to weld according to the planned path, and ensure that the robot accurately executes the path planning results.

[0011] The visualization operation module provides an industrial control computer interface to display real-time path, welding parameters, robot execution status, and abnormal alarms, and supports manual intervention.

[0012] A further improvement of the technical solution of the present invention is that the environmental perception module includes an industrial vision recognition unit and an obstacle detection unit;

[0013] The industrial vision recognition unit is used to capture and analyze images of the welding area in real time using a high-definition camera and image processing algorithms, identify the size, shape, weld position and assembly error of the workpiece, and generate three-dimensional point cloud data.

[0014] The obstacle detection unit is used to detect the position and distance of obstacles in the welding area in real time using lidar, and to discover and locate dynamic and static obstacles in the welding process.

[0015] A further improvement to the technical solution of the present invention is that the industrial vision recognition unit includes:

[0016] High-definition cameras deployed in the welding area capture real-time images of the welding area, obtaining two-dimensional image data including workpiece size, shape, weld position, and assembly errors.

[0017] The image processing algorithm is used to denoise, enhance and segment the two-dimensional image to extract the workpiece feature information, and combined with stereo vision technology, the two-dimensional image is converted into three-dimensional point cloud data.

[0018] A 3D model of the workpiece is constructed based on 3D point cloud data, and the size, shape, weld position and assembly error of the workpiece are identified, providing spatial coordinates and geometric references for path planning.

[0019] A further improvement of the technical solution of the present invention is that the obstacle detection unit includes:

[0020] The welding robot uses a laser radar mounted on it to emit laser beams in real time with parameters of 360° horizontal scanning and 0.5° vertical resolution. This measures the distance and azimuth of each point on the surface of the obstacle in the welding area, generating three-dimensional spatial point cloud data with a sampling frequency of ≥20Hz. At the same time, the original point cloud is denoised to remove invalid noise points caused by welding spatter or environmental dust. The data is then unified to the robot's base coordinate system through coordinate transformation to ensure spatial consistency.

[0021] Cluster analysis is performed on point cloud data. Dynamic and static obstacles are distinguished by the DBSCAN algorithm. False detection points are filtered out by combining the workpiece model library. The obstacle types are classified and labeled. The center coordinates, size and motion state of the obstacles are determined. A structured obstacle list is output.

[0022] Based on the robot's current position and the spatial relationship with obstacles, the minimum safe distance and collision time are calculated. If a potential collision risk is detected, the system immediately triggers an early warning and synchronizes obstacle information to the data processing and fusion module and the path planning and optimization module to ensure the safety of the welding process.

[0023] A further improvement to the technical solution of the present invention is that the data processing and fusion module includes:

[0024] The welding environment information output by the environmental perception module, including 3D point cloud data and obstacle information, is unified into a coordinate system. Data frames are synchronized by timestamp matching to eliminate spatiotemporal misalignment caused by differences in acquisition frequency.

[0025] Based on spatial proximity and feature similarity, the size, shape, weld position and assembly error of the workpiece contained in the 3D point cloud data are associated and mapped with the dynamic / static obstacles marked in the obstacle information. Semantic labels are used to achieve data semantic consistency and construct an intermediate model containing environmental geometry and semantic information.

[0026] By integrating the geometric features and dynamic attributes of workpiece and obstacle data, a unified environment model is generated using the octree method. The obstacle status (addition / disappearance / movement) is updated in real time and output to the path planning and optimization module.

[0027] A further improvement of the technical solution of the present invention is that the path planning optimization module includes a dynamic path planning unit and a path optimization adjustment unit;

[0028] The dynamic path planning unit uses an RRT-based path planning algorithm to generate an initial path based on a real-time environment model.

[0029] The path optimization and adjustment unit is used to optimize the initial path by taking into account energy consumption and collision risk factors, adjust the path to avoid obstacles and meet process requirements, reduce pauses and repetitive actions during welding, and improve welding quality and efficiency.

[0030] A further improvement to the technical solution of the present invention is that the dynamic path planning unit includes:

[0031] The system receives the environmental model output by the data processing and fusion module, extracts the welding environment information, including workpiece geometric features, obstacle positions and motion states, and constructs a searchable space through an octree structure. At the same time, it initializes the starting point (the robot's current position) and the target point (the end point of the weld) of the Rapid Expanding Random Tree (RRT) algorithm, and generates candidate nodes in free space through random sampling.

[0032] Based on the current robot pose, the random tree is iteratively expanded in the sampling space. Each iteration generates a new node and checks whether the connection between the new node and its parent node collides with an obstacle. If the new node overlaps with the obstacle volume element, the node is discarded and resampled. If the expansion is successful, the path segments between nodes are recorded, the new node is added to the tree structure, and the robot gradually approaches the target point.

[0033] When the random tree is first connected to the target point, the initial path is extracted and optimized. Redundant nodes are removed by pruning algorithm to shorten the path length, and B-spline curves are used to fit the smooth trajectory of discrete nodes to ensure compliance with robot kinematic constraints, thereby obtaining the initial path.

[0034] A further improvement to the technical solution of the present invention is that the path optimization and adjustment unit includes:

[0035] The path optimization and adjustment unit establishes a multi-objective function that includes energy consumption, collision risk and process constraints. Based on the node sequence of the initial path, it quantifies the joint motion energy consumption and obstacle distance penalty term of each segment of the path, and constructs an optimization problem in combination with welding process parameters to ensure that the path is balanced between safety and efficiency.

[0036] The obstacle status is updated by querying the octree in real time, and the initial path is verified by collision detection. If the current path fails due to the movement or addition of obstacles, a local RRT replanning is triggered, and an alternative path is generated only in the affected area.

[0037] The discrete path is fitted with B-spline curves to generate a high-order continuous trajectory. The safety of the final path is verified again by collision detection. At the same time, the feasibility of the motion parameters of each axis is analyzed, and a smooth, low-consumption welding path that meets the process requirements is output for the multi-axis collaborative control module to call.

[0038] A further improvement to the technical solution of the present invention is that the multi-axis cooperative control module includes:

[0039] The optimized welding path is received, and inverse kinematics calculation is performed based on the robot joint kinematics model. The Cartesian space path is decomposed into a sequence of rotation angles of each joint axis to ensure that the end effector accurately tracks the planned trajectory while meeting the joint motion range and speed limits.

[0040] High-order interpolation is performed on the rotation angle sequence of each joint axis after decomposition to generate a continuous and smooth time-parameterized trajectory. The position, velocity and acceleration command of each axis at each sampling time are calculated to ensure the synchronization of multi-axis motion and avoid trajectory tracking errors or mechanical vibration.

[0041] The interpolated trajectory command is converted into a pulse signal that the robot controller can recognize. The output torque of each axis is adjusted in real time through PID to compensate for dynamic errors and drive the robot to complete the welding task according to the planned path. At the same time, the joint status feedback is continuously monitored to ensure execution safety.

[0042] A further improvement to the technical solution of the present invention is that the visualization operation module includes:

[0043] The system acquires joint position, speed, acceleration, welding current, and voltage parameters from the robot controller in real time, synchronizes the data stream through a multi-threaded communication protocol, and dynamically renders the end effector trajectory, weld pool status, and motion curves of each axis in the graphical interface, ensuring that operators can intuitively monitor the execution process.

[0044] The system has built-in monitoring logic that continuously analyzes data. When an anomaly is detected, an alarm pop-up window appears on the interface, triggering simultaneous audio and visual alerts. It also automatically marks the anomaly timestamp and associated parameters, allowing operators to quickly locate the problem.

[0045] Operators can directly intervene in the execution process through the interface control, pause the robot and manually correct welding parameters or path points. After confirmation, the system seamlessly switches to manual intervention mode, replans the remaining trajectory and ensures multi-axis synchronization, and finally resumes automated execution.

[0046] Due to the adoption of the above technical solution, the technical progress achieved by this invention compared to the prior art is as follows:

[0047] 1. This invention provides a welding robot path planning system that integrates industrial vision and collision detection. By integrating industrial vision recognition and collision detection technologies, it can perceive the dynamic changes of the welding environment in real time and build an accurate environmental model based on three-dimensional point cloud data. Compared with traditional welding robots with preset paths, it can dynamically adjust the welding trajectory to avoid welding defects caused by environmental interference. At the same time, combined with the assembly error compensation mechanism, it can correct the deviation between the theoretical model and the actual workpiece, ensure the accuracy of the welding position, and meet the requirements of high-precision shipbuilding.

[0048] 2. This invention provides a welding robot path planning system that integrates industrial vision and collision detection. It uses LiDAR to detect static / dynamic obstacles in the welding area in real time, combines the DBSCAN clustering algorithm to distinguish obstacle types, and calculates the minimum safe distance and collision time. When a potential collision risk is detected, an early warning is immediately triggered and the path is dynamically adjusted to avoid direct contact between the robot and obstacles. At the same time, it reduces mechanical wear caused by sudden stops, extends the service life of the equipment, and ensures safety in human-robot collaboration scenarios. Attached Figure Description

[0049] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0050] Figure 1 This is a schematic diagram of the workflow of the present invention;

[0051] Figure 2 This is a schematic diagram of the data flow of the system functional modules of the present invention. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0053] Example 1, such as Figure 1 , Figure 2 As shown, this invention provides a welding robot path planning system that integrates industrial vision and collision detection, including a host computer. The host computer is communicatively connected to the following modules, wherein:

[0054] The host computer, as the core processing unit of the system, is used to communicate with other modules, receive, process and send control commands, so that operators can monitor and adjust the system's operating status.

[0055] The environmental perception module is used to collect welding environment information in real time, including the three-dimensional point cloud data of the workpiece and obstacle information, and to provide the basic data required for path planning. The environmental perception module includes an industrial vision recognition unit and an obstacle detection unit.

[0056] The industrial vision recognition unit utilizes high-definition cameras and image processing algorithms to perform real-time imaging and image analysis of the welding area, identify the workpiece's size, shape, weld position, and assembly errors, and generate 3D point cloud data. Through high-definition cameras deployed in the welding area, it captures real-time images of the welding area, acquiring 2D image data containing the workpiece's size, shape, weld position, and assembly errors. Image processing algorithms are used to denoise, enhance, and segment the 2D images, extracting workpiece feature information. Combined with stereo vision technology, the 2D images are converted into 3D point cloud data. Based on the 3D point cloud data, a 3D model of the workpiece is constructed, identifying the workpiece's size, shape, weld position, and assembly errors, providing spatial coordinates and geometric references for path planning.

[0057] The specific tasks of the industrial vision recognition unit are as follows: Around the robotic welding production line in the first span of the eight-span factory building, eight 20-megapixel industrial cameras (Basler ace acA2440-75uc) are used in a 45° angled circular pattern to cover a 60m panoramic view of the production line, in conjunction with a DLP structured light projector (TI LightCrafter). The 4500) system achieves active stereo vision, with each camera equipped with a 650nm anti-welding arc light interference filter and a dynamic range ≥120dB, ensuring coverage of the entire welding area with no blind spots. Through the deployment of high-definition cameras, it acquires real-time images of the welding environment, obtaining two-dimensional image data including workpiece size, shape, weld position, and assembly errors. The use of high-resolution cameras ensures the clarity and detail of the image data. During acquisition, the system simultaneously records timestamps and spatial coordinate information. This system is used for robotic automatic welding of deck sections, deployed in the first span of an eight-span workshop. The total design length of the entire deck section robotic welding production line is 60m. Applicable workpiece size parameters are as follows: materials are marine grade A, B, D, and E steel, as well as AH32, AH36, DH32, DH36, and EH36 steel; the maximum workpiece size is 15000mm long * 15000mm wide; the workpiece passage height is 3300mm, and the robot's Z-axis travel is not less than 1.2 meters; workpiece height ≤ 1000mm, maximum allowable height for vertical welding 1000mm, maximum allowable height for corner vertical welding 1000mm; single-stop welding distance for bridge welding gantry not less than 5m, gap between plates ≤ 2mm, angle between stiffener and main plate 90°±1°, workpiece dimensional error ≤ ±3mm, workpiece assembly dimensional error ≤ ±3mm; plate surface condition: anti-corrosion primer thickness 15μm~25μm; welding types include flat fillet weld, vertical fillet weld, arc weld, corner weld, planar zigzag fillet weld, and end elbow plate flat fillet weld, excluding assembly welding. It features multi-layer, multi-pass welding capabilities, eliminating the need for overhead and butt welding. After acquiring 2D images, image processing algorithms are used for further analysis. Noise reduction algorithms eliminate interference factors such as welding spatter and uneven lighting. Enhancement algorithms improve contrast and edge sharpness. Segmentation algorithms separate the workpiece area from the background, extracting key feature information. Combined with stereo vision technology, multi-view images are acquired through structured light projection, and 3D point cloud data is generated based on disparity calculation. Specifically, a deep learning-based semantic segmentation network processes the acquired 2D images, using pre-trained... A convolutional neural network model (U-Net) automatically identifies and separates the workpiece area from the background, extracts multi-scale features, and then restores spatial details through a decoder, ultimately outputting pixel-level classification results. The segmentation results undergo morphological post-processing (opening and closing operations) to eliminate discrete noise points, obtaining a continuous and complete workpiece mask. An optical pattern combining Gray code and phase-shifted fringes is projected by a DLP structured light projector. Multiple sets of coded patterns are projected sequentially through time multiplexing, while a camera synchronously acquires deformed fringe images. The absolute phase value is calculated using a phase unwrapping algorithm to establish the correspondence between camera and projector pixels. Combined with pre-calibrated system parameters including camera intrinsic and extrinsic parameters and lens distortion coefficients, parallax is calculated based on the principle of triangulation, converting the two-dimensional image coordinates into three-dimensional spatial points. The original point cloud is filtered to remove isolated noise points, and then voxel grid downsampling is used to maintain geometric features while reducing data volume. For large-scale deck segments, a partitioned scanning and global optimization strategy is adopted: first, high-precision reconstruction of local areas is performed, and then sub-region alignment is achieved through feature points. Finally, a complete point cloud model is synthesized. The point cloud density is dynamically adjusted according to welding process requirements, while the weld area is kept at 0.A 2mm spacing is used, which can be relaxed to 1mm in non-critical areas. Welding-related geometric features are extracted from the 3D point cloud, including weld positioning, assembly error detection, and surface condition analysis. Specifically, curvature analysis identifies Gaussian curvature extrema at plate joints, and a region growing algorithm is used to extract the centerline of continuous welds. The measured point cloud is non-rigidly registered with the CAD design model, and local deviations are calculated using the distance field to identify out-of-tolerance areas. The uniformity of anti-corrosion paint coverage is evaluated using the point cloud normal vector distribution, and damaged areas of the paint film are detected. Based on the converted 3D point cloud data, a 3D model of the workpiece is further constructed. Through point cloud registration and surface reconstruction algorithms, the actual size, shape, and weld position of the workpiece are restored. Simultaneously, combined with an assembly error analysis algorithm, the deviation between the theoretical design value and the actual workpiece is dynamically corrected, generating spatial coordinates and geometric references that conform to actual production conditions.

[0058] The obstacle detection unit is used to detect the position and distance of obstacles in the welding area in real time using LiDAR, discovering and locating dynamic and static obstacles during the welding process. Using LiDAR deployed on the welding robot, it emits laser beams in real time with parameters of 360° horizontal scanning and 0.5° vertical resolution, measuring the distance and azimuth of each point on the obstacle surface within the welding area, generating three-dimensional spatial point cloud data with a sampling frequency ≥20Hz. Simultaneously, it performs noise reduction processing on the original point cloud, removing invalid noise points caused by welding spatter or environmental dust, and unifies the data to the robot's base coordinate system through coordinate transformation to ensure spatial consistency. It performs cluster analysis on the point cloud data, distinguishing between dynamic and static obstacles using the DBSCAN algorithm, filtering false detection points using a workpiece model library, classifying and labeling obstacle types, determining the obstacle's center coordinates, size, and motion state, and outputting a structured obstacle list. Based on the robot's current position and the spatial relationship with the obstacles, it calculates the minimum safe distance and collision time. If a potential collision risk is detected, the system immediately triggers an early warning and synchronizes the obstacle information to the data processing and fusion module and the path planning and optimization module to ensure welding process safety.

[0059] The specific tasks of the obstacle detection unit are as follows: The obstacle detection unit achieves environmental perception through the lidar deployed on the welding robot, using a 16-line lidar (Velodyne). The VLP-16 laser radar emits a laser beam in real time with a 360° horizontal scan, a vertical field of view of ±15°, and a vertical resolution of 0.5°. It measures the distance and azimuth of each point on the surface of the obstacle within the welding area, generating a high-density point cloud. The sampling frequency is no less than 20Hz. The welding area is divided into three sectors (each sector is 120°). Priority is given to detecting the 120° area in front of the robot's movement path. The built-in "welding mode" of the laser radar is activated to suppress arc reflection interference. The minimum detection distance is set to 0.3m (to avoid near-field noise). The raw point cloud data needs to be preprocessed. Noise reduction processing is used to remove invalid noise caused by welding spatter, fumes, or environmental dust to improve detection accuracy. Subsequently, the point cloud data is uniformly converted to the robot's base coordinate system through coordinate transformation to ensure spatial consistency with the robot's motion control system. The anti-interference design of the laser radar includes welding spatter protection and electromagnetic shielding. A high-temperature resistant (≥300℃) ceramic protective cover is installed, and a double-layer metal shielding layer with an internal air curtain isolates dust and suppresses high-frequency interference from the welding power supply. The preprocessed point cloud data is clustered using the DBSCAN (density-based spatial clustering) algorithm to distinguish discrete point groups and identify potential obstacles. By adapting to point cloud distributions of different densities, dynamic obstacles (such as mobile devices and personnel) and static obstacles (such as tooling fixtures and unmoved workpieces) are effectively separated. Combined with a pre-set workpiece model library, false detection points (such as temporary splashes or reflective interference) are further filtered out, and obstacle types (such as personnel, equipment, tools, etc.) are classified and labeled. Finally, a structured obstacle list is output, including the center coordinates, geometric dimensions, and motion status (stationary or moving) of the obstacles. Based on the structured obstacle list and the robot's current pose, the system calculates the minimum safe distance and collision time between the robot and obstacles in real time. When a potential collision risk is detected, the system triggers an early warning mechanism, alerting the operator through sound and light. At the same time, the obstacle information is synchronized to the data processing and fusion module and the path planning and optimization module to dynamically adjust the welding path or replan the obstacle avoidance strategy, ensuring the safety and continuity of the welding process.

[0060] The expression for calculating the minimum safe distance is:

[0061] D safe =v robot ×t r +D buffer ;

[0062] In the formula, D safe For the minimum safe distance, v robot t represents the maximum speed of the robot's end effector. r D represents the system response time. buffer For buffer distance;

[0063] The expression for calculating the collision time is:

[0064]

[0065] In the formula, TTC is the collision time, Δd(t) is the relative distance, the real-time Euclidean distance between the robot and the obstacle, which changes dynamically with motion, Δv(t) is the relative velocity, the relative velocity (scalar) between the robot and the obstacle along the line connecting them, which is positive when they are close, and t horizon The time window represents the prediction time range. The system only calculates the collision time within this time period (to avoid predictions indefinitely far into the future). min(·) is the minimum collision time, calculated from all future times t. The minimum value of Δv(t) is used. If Δv(t) is negative (e.g., the obstacle is moving away from the robot), then TTC is meaningless (no warning is triggered). It is only used in the calculation when Δv(t) > 0 (the distance between the two is shortened).

[0066] The data processing and fusion module is used to fuse visual recognition results and obstacle detection data to form a unified environment model. It unifies the welding environment information, including 3D point cloud data and obstacle information, output by the environment perception module into a coordinate system, synchronizes data frames through timestamp matching to eliminate spatiotemporal misalignment caused by differences in acquisition frequency, and associates and maps the size, shape, weld position, and assembly error of the workpiece contained in the 3D point cloud data with the dynamic / static obstacles marked in the obstacle information based on spatial proximity and feature similarity. It achieves data semantic consistency through semantic tags, constructs an intermediate model containing environmental geometry and semantic information, fuses the geometric features and dynamic attributes of workpiece and obstacle data, generates a unified environment model using the octree method, updates obstacle status (addition / disappearance / movement) in real time, and outputs it to the path planning and optimization module to ensure that the robot's motion decisions are consistent with the actual environment dynamics.

[0067] The specific tasks of the data processing and fusion module are as follows: For the 3D point cloud data (based on the LiDAR coordinate system) and obstacle information (based on the robot's base coordinate system) output by the environmental perception module, a rigid body transformation matrix is ​​used to unify them into the same reference coordinate system, eliminating spatial misalignment caused by differences in sensor installation positions. Simultaneously, a timestamp matching algorithm is used to interpolate and synchronize data from different acquisition frequencies, ensuring that the point cloud frame and obstacle detection frame are aligned in the time dimension. The data buffer window size is dynamically adjusted to adapt to asynchronous input from 20Hz to 100Hz, outputting a spatiotemporally consistent environmental data stream. After completing coordinate system one, workpiece data and obstacle information are fused based on spatial proximity and feature similarity. Specifically, the geometric features of the workpiece surface normal vector and curvature in the point cloud are calculated and matched with the geometric attributes of the obstacle cluster centers. Combined with preset semantic tags, a workpiece-obstacle association mapping table is constructed. At the same time, an assembly error compensation mechanism is introduced to analyze the theoretical CAD model and... If the deviation of the actual point cloud data continues to exceed a preset threshold, semantic label reclassification is triggered to ensure that the dynamic update of semantic labels is synchronized with the actual environmental changes, generating an intermediate model containing geometric dimensions, position, motion state, and semantic attributes. Based on the intermediate model, an octree structure is used to construct a hierarchical environment model, discretizing the continuous space of the welding area into volume elements with a side length of 0.2m. Each node stores the geometric features of the workpiece (size, weld position) and the dynamic attributes of obstacles (type, speed, trajectory). Through a sliding window mechanism, the data of the most recent 10 frames is maintained, and the changes of volume elements are monitored in real time. The obstacle status (addition / disappearance / movement) is dynamically updated. Added obstacles are detected as unmatched cluster centers, disappeared obstacles are detected as not detected for 3 consecutive frames, and moving obstacles are when the position change exceeds the size of the volume element, triggering local model reconstruction. When the obstacle status changes, the relevant volume elements are reconstructed with an update frequency of ≤50ms. The fused environment model is then output to the path planning module.

[0068] The path planning and optimization module is used to analyze the visual recognition results and obstacle detection data contained in the environmental model, and generate feasible welding paths in combination with welding process requirements;

[0069] The multi-axis collaborative control module is used to decompose the planned path into motion commands for each axis of the robot based on the robot's joint kinematics model, convert them into control signals that the robot can recognize, drive the robot to weld according to the planned path, and ensure that the robot accurately executes the path planning results.

[0070] The visualization operation module provides an industrial control computer interface to display real-time path, welding parameters, robot execution status, and abnormal alarms, and supports manual intervention.

[0071] Example 2, as Figure 1 , Figure 2As shown, based on Embodiment 1, the present invention provides a technical solution: preferably, the path planning optimization module includes a dynamic path planning unit and a path optimization adjustment unit;

[0072] The dynamic path planning unit employs a path planning algorithm based on RRT (Rapidly Expanding Random Tree). It generates an initial path based on a real-time environment model, receives the environment model output by the data processing and fusion module, extracts welding environment information including workpiece geometry, obstacle positions, and motion states, and constructs a searchable space using an octree structure. Simultaneously, it initializes the starting point (robot's current position) and target point (weld end point) of the Rapidly Expanding Random Tree (RRT) algorithm. Candidate nodes are generated in free space through random sampling, ensuring the sampling process avoids known obstacle areas. Based on the current robot pose, the random tree is iteratively expanded in the sampling space. Each iteration generates a new node and checks whether its connection to its parent node collides with an obstacle. If a new node overlaps with an obstacle volume element, the node is discarded and resampled. If expansion is successful, the path segments between nodes are recorded, and the new node is added to the tree structure, gradually approaching the target point. When the random tree first connects to the target point, the initial path is extracted and optimized. Redundant nodes are removed using a pruning algorithm to shorten the path length, and B-spline curves are used to fit the smooth trajectory of discrete nodes, ensuring compliance with robot kinematic constraints, thus obtaining the initial path.

[0073] The specific tasks of the dynamic path planning unit are as follows: It receives the environment model output by the data processing and fusion module, analyzes the welding environment information, including workpiece geometry, obstacle positions, and motion states, and discretizes the continuous space into searchable volume elements using an octree structure. Each node stores the obstacle occupancy status and dynamic attributes, supporting efficient spatial querying and collision detection. Based on the robot's current pose (starting point) and weld endpoint (target point), it initializes the Rapidly Expanding Random Tree (RRT) algorithm, randomly samples candidate nodes in free space, and uses the octree to quickly determine whether the sampled point is within an obstacle volume element. If a conflict occurs, resampling is performed to ensure that tree expansion always occurs within a feasible area. The path planning unit uses the robot's current pose as the root node and iteratively expands the RRT in the sampled space. Each iteration randomly generates candidate nodes and queries their neighboring volume elements using the octree. If a candidate node overlaps with an obstacle volume element, it is considered a collision risk, discarded, and resampled. If safe, the connection between the candidate node and the nearest node in the tree is calculated, and the connection is checked to see if it penetrates the obstacle (verified by volume element traversal). If the connection does not collide, the candidate node is added to the tree structure, and the path segments between nodes are recorded. If a collision occurs, the sampling direction or step size is adjusted, and the attempt is repeated. Through continuous iteration, the random tree gradually approaches the target point until the target point is connected for the first time, forming an initial path framework. Based on the initial path framework, the optimization process is initiated. A pruning algorithm is applied to remove redundant nodes, merge collinear or nearest-neighbor path segments, and shorten the total path length. Then, B-spline curves are used to fit the discrete nodes to generate a continuous and smooth trajectory curve, so that the kinematic parameters of the robot's acceleration and velocity meet the constraints, and an initial path that conforms to the robot's motion capabilities is output.

[0074] The path optimization and adjustment unit is used to optimize the initial path by comprehensively considering energy consumption and collision risk factors. The path is adjusted to avoid obstacles and meet process requirements, reduce pauses and repetitive actions during welding, improve welding quality and efficiency, and reduce energy consumption and collision risk. The path optimization and adjustment unit establishes a multi-objective function that includes energy consumption, collision risk and process constraints. Based on the node sequence of the initial path, it quantifies the joint motion energy consumption and obstacle distance penalty of each path segment. Combined with welding process parameters, it constructs an optimization problem to ensure that the path is balanced between safety and efficiency. The obstacle status is updated by querying and updating the obstacle status in real time through octree. The initial path is verified by collision detection. If the current path fails due to the movement or addition of obstacles, a local RRT replanning is triggered. An alternative path is generated only in the affected area to avoid global recalculation. The path segment is kept away from dynamic obstacles and meets the robot kinematic constraints to avoid sudden stops or jitter. B-spline curves are applied to fit the discrete path to generate a high-order continuous trajectory. The safety of the final path is verified again by collision detection. At the same time, the feasibility of motion parameters of each axis is analyzed, and a smooth, low-energy welding path that meets process requirements is output for the multi-axis collaborative control module to call.

[0075] The specific work of the path optimization and adjustment unit is as follows: Based on the node sequence of the initial path, the path optimization and adjustment unit establishes a multi-objective optimization function that includes energy consumption, collision risk, and process constraints. It quantifies the performance indicators of each segment of the initial path. Specifically, the energy consumption term calculates the integral of the driving torque and displacement of each axis based on the robot's joint kinematics model, reflecting mechanical energy consumption. The collision risk term queries the minimum distance between path points and obstacles using an octree, introducing the reciprocal of the distance as a penalty term. The process constraint term combines parameters such as welding speed and welding torch posture to ensure weld quality. This transforms the multi-objective problem into a constrained optimization problem, which is then balanced through weighted summation. Safety and efficiency are ensured by maintaining a safe distance from obstacles while reducing energy consumption, thus meeting the requirements of welding processes for trajectory continuity and stability. The obstacle status updated by the octree is continuously monitored, and collision detection is performed on the current path. If a dynamic obstacle intrusion or a new obstacle causes the path to fail, a hierarchical response strategy is triggered. The first level adjustment is to use the elastic band method to fine-tune the path for intruding obstacles, maintaining the original path topology. The second level replanning is to start the local RRT* algorithm when the intrusion makes the path infeasible, and establish a new sampling space within 50cm around the affected path segment. Collision detection is accelerated using an octree to ensure that new nodes meet the following requirements: a minimum distance of 30cm from dynamic obstacles, a turning angle of no more than 15° between adjacent nodes, and a single adjustment length of no more than 20% of the original path. Replanning is limited to the affected path segment, generating a sequence of alternative nodes around the obstacle. The octree is used to quickly verify the collision-free nature of the new path segment, avoiding global recalculation and improving efficiency. Random sampling and collision detection iteratively expand the tree structure to ensure that the new path segment stays away from dynamic obstacles, while avoiding sudden stops or jitter, ensuring the smoothness and safety of robot movement. The optimized discrete path is fitted with B-spline curves to generate a smooth trajectory of second order or higher. The fitting process constrains the spacing and curvature of control points, ensuring that the velocity and acceleration of each axis meet the upper limit of robot dynamics. The final trajectory is then subjected to global collision detection again using an octree to ensure no missed collisions. Simultaneously, the feasibility of the position, velocity, and acceleration curves of each axis is analyzed, and path segments exceeding the execution capability are eliminated. The final output is a welding path that combines smoothness, low energy consumption, and process compliance, which is directly sent to the multi-axis collaborative control module for execution.

[0076] The multi-axis collaborative control module includes: receiving the optimized welding path, performing inverse kinematics calculation based on the robot joint kinematics model, decomposing the Cartesian space path into a sequence of rotation angles for each joint axis to ensure that the end effector accurately tracks the planned trajectory while meeting the joint motion range and speed limits, performing high-order interpolation on the decomposed rotation angle sequence of each joint axis to generate a continuous and smooth time-parameterized trajectory, calculating the position, velocity, and acceleration commands of each axis at each sampling moment to ensure multi-axis motion synchronization and avoid trajectory tracking errors or mechanical vibrations, converting the interpolated trajectory commands into pulse signals recognizable by the robot controller, adjusting the output torque of each axis in real time through PID control to compensate for dynamic errors, driving the robot to complete the welding task according to the planned path, and continuously monitoring joint status feedback to ensure execution safety.

[0077] The specific tasks of the multi-axis collaborative control module are as follows: It receives the optimized welding path (including position, attitude, and time parameters), performs inverse kinematics calculations based on the robot's joint kinematics model, and iteratively solves the nonlinear equations to decompose the trajectory of the end effector into a sequence of rotation angles for each joint axis. This ensures that the joint angle combination at each sampling moment uniquely corresponds to the Cartesian space pose. Simultaneously, it verifies whether the joint angles are within physical constraints and performs real-time singular configuration avoidance (by monitoring the Jacobian matrix condition number; when a near-singular configuration is detected (condition number > 10^4), a joint angle optimization term is automatically introduced to adjust to a more operable configuration). This ensures that the decomposed joint angle sequence satisfies the speed consistency during multi-axis linkage, avoiding trajectory deviations caused by single-axis overspeed or asynchrony. Specifically, an improved DH parameter method is used to establish the geometric transformation matrix of the Kawasaki RA005L six-axis robot, constructing the forward kinematic equations. A hybrid algorithm combining analytical and numerical methods is used to solve the inverse kinematics, ensuring that feasible solutions can still be obtained near singular points. Based on the joint rotation angle sequence, high-order interpolation is performed to generate continuous and smooth time-series data. The parameterized trajectory discretizes the time axis based on the sampling period and calculates the joint position, velocity, and acceleration commands at each moment to ensure multi-axis motion synchronization. During interpolation, C3 continuity (position, velocity, acceleration, and jerk are all continuous) is enforced to avoid trajectory tracking errors or mechanical vibrations. Simultaneously, the dynamic parameters of each axis are verified in real time, and out-of-limit segments are dynamically scaled or replanned to ensure that the trajectory conforms to the robot's dynamic performance. Finally, the interpolated trajectory command set is output, including a timestamp, joint angle, velocity, and acceleration quadruple. The interpolated trajectory command set is converted into pulse signals that the robot controller can recognize and is synchronously sent to each drive unit through the encoder interface. During execution, a PID control algorithm is used to adjust the output torque of each axis in real time to compensate for dynamic errors caused by friction, inertia, and external disturbances. By comparing the deviation between the actual joint position and the target trajectory, the control quantity is dynamically calculated to ensure that the end effector accurately tracks the planned path. At the same time, the joint current, temperature, and load status are continuously monitored. If overload or abnormal vibration is detected, the safety protection mechanism is immediately triggered to ensure the safety and stability of the execution process.

[0078] The visualization operation module includes: real-time acquisition of joint position, speed, acceleration, and welding current and voltage parameters fed back by the robot controller; synchronization of data streams through a multi-threaded communication protocol; and dynamic rendering of the end effector trajectory, weld pool status, and motion curves of each axis in the graphical interface, ensuring that operators can intuitively monitor the execution process. The system has built-in monitoring logic to continuously analyze data. When an anomaly is detected, an alarm pop-up window appears on the interface, triggering audio and visual prompts simultaneously, and automatically marking the anomaly timestamp and related parameters for operators to quickly locate the problem. Operators can directly intervene in the execution process through the interface interactive controls, pausing the robot and manually correcting welding parameters or path points. After confirmation, the system seamlessly switches to manual intervention mode, replans the remaining trajectory, ensures multi-axis synchronization, and finally resumes automated execution.

[0079] The specific functions of the visualization operation module are as follows: The visualization operation module establishes a real-time data connection with the robot controller via a multi-threaded communication protocol, periodically acquiring motion parameters such as joint position, speed, and acceleration, as well as process parameters such as welding current and voltage. After buffering, the module calls the graphics rendering engine to map the multi-dimensional data into dynamic visualization elements. The end effector trajectory is updated in real time as a 3D curve, the weld pool status is presented through color gradients or heat maps, and the motion curves of each axis are displayed as overlaid multi-channel waveforms. The interface adopts a layered layout, with key parameters highlighted on a digital dashboard, ensuring that operators can simultaneously monitor the overall status and detailed indicators, improving process transparency. The module also includes a built-in fault diagnosis function. The system continuously analyzes the matching of sensor data and process parameters, as well as the status of the electrical system. When an anomaly is detected, an alarm pop-up window appears on the interface, and a safety interlock warning is issued. The warning is achieved through a combination of a three-color bee warning light, limit switches, and a system emergency stop button. For the three-color bee warning light: Yellow light on: No anomaly, no emergency stop, non-automatic operation; Yellow light flashing: No anomaly, no emergency stop, non-automatic operation, information reminder available (manual / automatic switching); Green light on: Automatic operation; Green light flashing: Automatic operation paused; Red light on: Emergency stop present, non-automatic operation; Red light flashing: Anomaly present, no emergency stop, non-automatic operation; Green light flashing + Red light on + beeping: Automatic operation has an emergency stop signal (intermittent beeping, automatic stop after 15 seconds). Green light flashing + red light flashing + buzzer: Automatic operation malfunctions with no emergency stop signal (intermittent buzzer, automatic stop after 15 seconds); The limit switch is valid in automatic operation mode. Its trigger result is limit switch OFF, automatic operation program is prohibited from starting or pausing, robot pauses, red warning light illuminates; Recovery method is to check for abnormalities, manually turn servo ON, and allow automatic operation program to start or restart; For the system emergency stop button, the valid condition is all states; The trigger result is emergency stop button in non-automatic operation state, red warning light illuminates, all operations are prohibited; Emergency stop during automatic operation, automatic operation stops, red light illuminates + buzzer; Its recovery method is non-automatic operation state, emergency stop reset will release the prohibition; Emergency stop during automatic operation, emergency stop reset... After the robot is in position without any abnormalities, it automatically pauses. It automatically records the complete data sequence before and after the fault occurs, generating a traceable log file. It provides an interactive control interface, allowing operators to directly intervene in the execution process through interface buttons, sliders, or touch gestures. When it is necessary to pause the robot, clicking the "Emergency Stop" button will immediately freeze all axis movements while keeping the data communication channel active. Subsequently, the operator can manually adjust the welding parameters or modify the path point coordinates. The module verifies the legality of the input in real time. After confirming the modification, it automatically calls the trajectory replanning algorithm to generate a smooth transition of the remaining path based on the current robot state. It also ensures multi-axis collaboration through a synchronous control protocol, and finally seamlessly resumes automated execution, balancing safety and production efficiency.

[0080] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A welding robot path planning system that integrates industrial vision and collision detection, comprising a host computer, characterized in that, The host computer communication connection includes the following modules, wherein: The environmental perception module is used to collect welding environment information in real time, including the three-dimensional point cloud data of the workpiece and obstacle information; The data processing and fusion module is used to fuse visual recognition results and obstacle detection data to form a unified environment model; The path planning and optimization module is used to analyze the visual recognition results and obstacle detection data contained in the environmental model, and generate feasible welding paths in combination with welding process requirements; The multi-axis collaborative control module is used to decompose the planned path into motion commands for each axis of the robot based on the robot's joint kinematics model, convert them into control signals that the robot can recognize, and drive the robot to weld according to the planned path. The visualization module provides an industrial control computer interface to display real-time path, welding parameters, robot execution status, and abnormal alarms.

2. The welding robot path planning system with collaborative industrial vision and collision detection according to claim 1, characterized in that: The environmental perception module includes an industrial vision recognition unit and an obstacle detection unit; The industrial vision recognition unit is used to capture and analyze images of the welding area in real time using a high-definition camera and image processing algorithms, identify the size, shape, weld position and assembly error of the workpiece, and generate three-dimensional point cloud data. The obstacle detection unit is used to detect the position and distance of obstacles in the welding area in real time using lidar, and to discover and locate dynamic and static obstacles in the welding process.

3. The welding robot path planning system with collaborative industrial vision and collision detection according to claim 2, characterized in that: The industrial vision recognition unit includes: High-definition cameras deployed in the welding area capture real-time images of the welding area, obtaining two-dimensional image data including workpiece size, shape, weld position, and assembly errors. The image processing algorithm is used to denoise, enhance and segment the two-dimensional image to extract the workpiece feature information, and combined with stereo vision technology, the two-dimensional image is converted into three-dimensional point cloud data. A 3D model of the workpiece is constructed based on 3D point cloud data, and the size, shape, weld position and assembly error of the workpiece are identified, providing spatial coordinates and geometric references for path planning.

4. The welding robot path planning system with collaborative industrial vision and collision detection according to claim 2, characterized in that: The obstacle detection unit includes: The welding robot uses a laser radar mounted on it to emit laser beams in real time with parameters of 360° horizontal scanning and 0.5° vertical resolution. This measures the distance and azimuth of each point on the surface of the obstacle in the welding area, generating three-dimensional spatial point cloud data. At the same time, the original point cloud is denoised and the data is unified to the robot's base coordinate system through coordinate transformation. Cluster analysis is performed on point cloud data. Dynamic and static obstacles are distinguished by the DBSCAN algorithm. False detection points are filtered out by combining the workpiece model library. The obstacle types are classified and labeled. The center coordinates, size and motion state of the obstacles are determined. A structured obstacle list is output. Based on the robot's current position and the spatial relationship with obstacles, the minimum safe distance and collision time are calculated. If a potential collision risk is detected, the system immediately triggers an early warning and synchronizes the obstacle information to the data processing and fusion module and the path planning and optimization module.

5. The welding robot path planning system with collaborative industrial vision and collision detection according to claim 2, characterized in that: The data processing and fusion module includes: The welding environment information output by the environmental perception module, including 3D point cloud data and obstacle information, is unified into a coordinate system, and data frames are synchronized by timestamp matching. Based on spatial proximity and feature similarity, the size, shape, weld position and assembly error of the workpiece contained in the 3D point cloud data are associated and mapped with the dynamic / static obstacles marked in the obstacle information. Semantic labels are used to achieve data semantic consistency and construct an intermediate model containing environmental geometry and semantic information. By integrating the geometric features and dynamic attributes of workpiece and obstacle data, an octree method is used to generate a unified environment model, which updates the obstacle status in real time and outputs it to the path planning and optimization module.

6. The welding robot path planning system with collaborative industrial vision and collision detection according to claim 5, characterized in that: The path planning and optimization module includes a dynamic path planning unit and a path optimization and adjustment unit; The dynamic path planning unit uses an RRT-based path planning algorithm to generate an initial path based on a real-time environment model. The path optimization and adjustment unit is used to optimize the initial path by taking into account energy consumption and collision risk factors, and to adjust the path to avoid obstacles and meet process requirements.

7. The welding robot path planning system with collaborative industrial vision and collision detection according to claim 6, characterized in that: The dynamic path planning unit includes: The system receives the environmental model output by the data processing and fusion module, extracts the welding environment information, including workpiece geometric features, obstacle positions and motion states, and constructs a searchable space through an octree structure. At the same time, it initializes the starting point and target point of the fast expanding random tree algorithm and generates candidate nodes in free space through random sampling. Based on the current robot pose, the random tree is iteratively expanded in the sampling space. Each iteration generates a new node and checks whether the connection between the new node and its parent node collides with an obstacle. If the new node overlaps with the obstacle volume element, the node is discarded and resampled. If the expansion is successful, the path segments between nodes are recorded, the new node is added to the tree structure, and the robot gradually approaches the target point. When the random tree is first connected to the target point, the initial path is extracted and optimized. Redundant nodes are removed by pruning algorithm to shorten the path length, and B-spline curves are used to fit the smooth trajectory of discrete nodes to ensure compliance with robot kinematic constraints, thereby obtaining the initial path.

8. The welding robot path planning system with collaborative industrial vision and collision detection according to claim 6, characterized in that: The path optimization and adjustment unit includes: The path optimization and adjustment unit establishes a multi-objective function that includes energy consumption, collision risk and process constraints. Based on the node sequence of the initial path, it quantifies the joint motion energy consumption and obstacle distance penalty term of each path segment, and constructs an optimization problem in combination with welding process parameters. The obstacle status is updated by querying the octree in real time, and the initial path is verified by collision detection. If the current path fails due to the movement or addition of obstacles, a local RRT replanning is triggered, and an alternative path is generated only in the affected area. The discrete path is fitted with B-spline curves to generate a high-order continuous trajectory. The safety of the final path is verified again by collision detection. At the same time, the feasibility of the motion parameters of each axis is analyzed, and a smooth, low-consumption welding path that meets the process requirements is output for the multi-axis collaborative control module to call.

9. A welding robot path planning system with collaborative industrial vision and collision detection according to claim 6, characterized in that: The multi-axis collaborative control module includes: The optimized welding path is received, and inverse kinematics calculation is performed based on the robot joint kinematics model. The Cartesian space path is decomposed into a sequence of rotation angles of each joint axis, while satisfying the joint motion range and speed limits. The rotation angle sequence of each joint axis after decomposition is interpolated by high order to generate a continuous and smooth time parameterized trajectory, and the position, velocity and acceleration command of each axis at each sampling time are calculated. The interpolated trajectory command is converted into a pulse signal that the robot controller can recognize. The output torque of each axis is adjusted in real time through PID control to compensate for dynamic errors and drive the robot to complete the welding task according to the planned path.

10. A welding robot path planning system with collaborative industrial vision and collision detection according to claim 9, characterized in that: The visualization operation module includes: The system acquires joint position, velocity, acceleration, welding current, and voltage parameters from the robot controller in real time, synchronizes the data stream through a multi-threaded communication protocol, and dynamically renders the end effector trajectory, weld pool status, and motion curves of each axis in the graphical interface. The system has built-in monitoring logic that continuously analyzes data. When an anomaly is detected, an alarm pop-up window appears on the interface, triggering simultaneous audio and visual alerts. It also automatically marks the anomaly timestamp and associated parameters, allowing operators to quickly locate the problem. Operators can directly intervene in the execution process through the interface control, pause the robot and manually correct welding parameters or path points. After confirmation, the system seamlessly switches to manual intervention mode, replans the remaining trajectory, and finally resumes automated execution.

Images