Automatic welding seam tracking system of nondestructive testing robot based on multi-modal sensing

Abstract

The invention relates to the technical field of industrial nondestructive testing, in particular to an automatic welding seam tracking system of a nondestructive testing robot based on multi-modal sensing. Comprising a data acquisition module, a data preprocessing module, a data fusion module, a track generation module and a motion control module. Point cloud data are obtained through a laser scanner, image information is obtained through an RGB-D camera, attitude data are obtained through an inertial measurement unit, after multi-modal data are subjected to denoising, down-sampling and correction processing, data fusion is achieved through Kalman filtering, and continuous weld space trajectory points are output. And the track generation module generates a tail end pose sequence through three-dimensional B-spline fitting and surface normal estimation, and performs pose smoothing processing by adopting quaternion interpolation. The motion control module is combined with differential path keeping and mixed position-force control, it is ensured that the robot accurately tracks along the center line of the welding seam and maintains constant contact force, meanwhile, the process error correction function is achieved, and the stability and reliability of the system in the complex welding seam environment are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of industrial non-destructive testing, in particular to a non-destructive testing robot weld seam automatic tracking system based on multi-modal sensing. BACKGROUND

[0002] Weld seam quality is an important guarantee for the safe operation of industrial equipment and pipelines, so non-destructive testing of weld seams is of great significance in industrial production. Existing ultrasonic testing robots have low efficiency and complex operation in actual application. For example, ultrasonic testing robots for long-distance pipelines usually need to be installed along the pipeline track and disassembled and reinstalled in sections, resulting in long detection period, high labor intensity and low efficiency.

[0003] In addition, traditional crawling detection robots usually rely on preset trajectories for weld seam tracking, which is difficult to adapt to geometric deformation or installation errors of weld seams during construction or use. Especially for curved weld seams, the existing technology has not realized real-time trajectory correction, and the robot is difficult to accurately move along the weld centerline, thereby affecting the detection accuracy. On the other hand, the coupling state of the ultrasonic detection probe is unstable, which will cause the signal-to-noise ratio to fluctuate, and the detection data is prone to poor coupling, thereby reducing the reliability of the detection result. The existing technology cannot simultaneously meet the requirements of weld seam trajectory adaptability, automatic tracking of curved weld seams and stable coupling of the probe. SUMMARY

[0004] In view of the deficiencies of the prior art, the present application provides a non-destructive testing robot weld seam automatic tracking system based on multi-modal sensing, aiming to solve the problems of low detection accuracy and unstable trajectory tracking of traditional weld seams.

[0005] To achieve the above purpose, the present application realizes the following technical scheme: a non-destructive testing robot weld seam automatic tracking system based on multi-modal sensing, comprising:

[0006] A data acquisition module for acquiring point cloud data through a laser scanner, RGB-D image data through an RGB-D camera, and attitude data through an inertial measurement unit;

[0007] A data preprocessing module for preprocessing the point cloud data, RGB-D image data and attitude data to obtain preprocessed multi-modal data;

[0008] A data fusion module for synchronizing the preprocessed multi-modal data and outputting continuous weld seam spatial trajectory points through a Kalman fusion algorithm;

[0009] A trajectory generation module for three-dimensional B-spline fitting according to the weld seam spatial trajectory points with arc length parameterization, and generating end pose sequence in combination with the surface normal estimated by the point cloud to obtain a target motion trajectory;

[0010] The motion control module is used for driving the robot to perform a detection task according to a target motion trajectory, and includes differential path keeping control based on a lateral deviation and hybrid position-force control based on feedforward-PID and force feedback of an end of a mechanical arm, so as to realize automatic tracking of a weld seam and nondestructive detection.

[0011] Further, the step of obtaining the preprocessed multi-modal data comprises:

[0012] The point cloud data is subjected to denoising, downsampling and coordinate system calibration, and a random sample consensus method is used to fit the bevel slope to extract weld center line feature points;

[0013] The RGB-D image data is subjected to weld seam region edge detection, depth map denoising and feature point matching to supplement weld edge feature information;

[0014] The attitude data is subjected to zero offset correction and low-pass filtering to obtain stable attitude angle and angular velocity information.

[0015] Further, the step of outputting continuous weld seam spatial trajectory points comprises:

[0016] The preprocessed point cloud data and image data are subjected to spatial coordinate alignment and time synchronization to generate fused laser-vision multi-modal data;

[0017] The Kalman filtering algorithm is used to fuse the laser-vision multi-modal data and the attitude data collected by the inertial measurement unit to estimate the three-dimensional position and attitude state of the weld center line in real time;

[0018] The continuous weld seam spatial trajectory points are outputted through the state updating and prediction mechanism of the Kalman filtering.

[0019] Further, the step of obtaining the target motion trajectory is:

[0020] The continuous weld seam spatial trajectory points outputted by the Kalman filtering are sorted and parameterized according to arc length parameters;

[0021] Based on the parameterized trajectory points, a three-dimensional B-spline curve fitting method is used to generate a smooth and continuous weld seam trajectory curve;

[0022] The local normal information of the weld seam surface is estimated using the point cloud data, and corresponding end pose sequences are generated in combination with the fitted curve.

[0023] Further, when generating the end pose sequences, the trajectory generation module adopts a quaternion interpolation-based attitude smoothing processing method to perform spherical linear interpolation on adjacent poses, so as to avoid sudden changes in the attitude of the mechanical arm and ensure the continuity and stability of the trajectory attitude.

[0024] Further, the step of differential path keeping control comprises:

[0025] Real-time measurement of the transverse deviation of the weld centerline from the robot centerline using a laser sensor;

[0026] Calculate the deviation by a PID control algorithm to generate a speed difference value;

[0027] Adjust the PWM duty cycle of the left and right wheels according to the calculated speed difference value, and dynamically adjust the speed ratio of the left and right wheels;

[0028] Verify the stability of the adjusted wheel speed ratio to ensure that the robot accurately tracks the weld centerline.

[0029] Further, the step of hybrid position-force control comprises:

[0030] Provide the spatial deviation of the weld center by the laser vision module;

[0031] Calculate the joint angles using inverse kinematics of the robot to compensate for the position spatial deviation and achieve position adjustment of the probe center to align with the weld centerline;

[0032] Real-time acquisition of the probe contact force feedback value by a six-axis force sensor, and comparison with the preset target force value;

[0033] Adjust the Z-axis displacement of the robot using a PID controller to achieve constant contact force adjustment.

[0034] Further, the hybrid position-force control adjusts the control strategy according to the type of the weld, adopts constant force control mode when detecting a flat weld, and adopts hybrid control mode combining position adjustment and force adjustment when detecting a curved weld.

[0035] Further, the system further comprises a process error correction module, and the implementation logic of the process error correction is:

[0036] When the laser signal is lost, switch to pure vision mode, detect that the robot moving speed is reduced to 50% of the preset speed, and when the laser signal is restored and lasts for 2 seconds, restore the initial state;

[0037] When the contact force exceeds 50% of the preset initial value, the robot immediately retreats 3mm and re-couples, and when the force value stabilizes at the preset initial value for 1 second, the initial state is restored;

[0038] When communication interruption occurs, the robot immediately stops moving and is manually remotely controlled back to the original position by an external controller.

[0039] The present application also proposes a non-destructive testing robot weld automatic tracking method based on multi-modal sensing, comprising:

[0040] a data acquisition module configured to acquire point cloud data through a laser scanner, acquire RGB-D image data through an RGB-D camera, and acquire attitude data through an inertial measurement unit;

[0041] preprocess the point cloud data, the RGB-D image data, and the attitude data to obtain preprocessed multi-modal data;

[0042] synchronize the preprocessed multi-modal data and output continuous weld seam spatial trajectory points through a Kalman filtering algorithm;

[0043] perform three-dimensional B-spline fitting according to the weld seam spatial trajectory points and parameterize according to arc length, generate an end pose sequence in combination with surface normal estimated by the point cloud, and obtain a target motion trajectory;

[0044] drive the robot to perform a detection task according to the target motion trajectory, including differential path keeping control based on lateral deviation and hybrid position-force control of the robot end based on feedforward-PID and force feedback, to realize automatic tracking of the weld seam and non-destructive detection.

[0045] The present application provides a non-destructive detection robot weld seam automatic tracking system based on multi-modal sensing. The present application has the following advantages:

[0046] 1. The present application realizes high-precision and real-time perception of the weld seam center line through multi-modal sensing fusion of the laser scanner, the RGB-D camera, and the inertial measurement unit. The laser provides accurate geometric information, the vision supplements edge features, and the IMU provides stable attitude data, so that the system can stably acquire continuous spatial trajectories under complex weld seam shapes, and the weld seam detection precision and reliability are significantly improved.

[0047] 2. The present application adopts a Kalman filtering algorithm to fuse multi-modal data and realize continuous output of weld seam spatial trajectory points. Through a state prediction and update mechanism, the influence of sensor noise and short-time data loss is effectively suppressed, the stability and continuity of trajectory estimation are improved, a reliable foundation is provided for subsequent trajectory generation and motion control, and the robot is accurately tracked along the weld seam.

[0048] 3. The present application generates a smooth end pose sequence through three-dimensional B-spline fitting and surface normal estimation, and performs attitude smoothing processing in combination with attitude interpolation based on quaternions, effectively avoids attitude mutation of the robot arm, improves the continuity and stability of the motion trajectory, enables the probe to accurately align with the weld seam center, and ensures uniform contact force and stable detection quality during the non-destructive detection process.

[0049] 4、The application combines differential path keeping control and hybrid position-force control, and introduces a process error correction mechanism, to realize coordinated control of the robot mobile platform and the mechanical arm. In the case of complex weld or abnormal sensing, the system can automatically adjust the wheel speed or force feedback, ensure the accuracy and safety of the weld tracking, improve the overall detection efficiency, and significantly enhance the reliability and applicability of the system. BRIEF DESCRIPTION OF DRAWINGS

[0050] Figure 1 A structure diagram of the non-destructive testing robot weld automatic tracking system based on multi-modal sensing provided by the application is shown in the figure.

[0051] Figure 2 A step flow chart of outputting continuous weld space trajectory points provided by the application is shown in the figure.

[0052] Figure 3 A differential path keeping control step flow chart provided by the application is shown in the figure.

[0053] Figure 4 A hybrid position-force control step flow chart provided by the application is shown in the figure.

[0054] Figure 5 A flow chart of the non-destructive testing robot weld automatic tracking method based on multi-modal sensing provided by the application is shown in the figure. DETAILED DESCRIPTION

[0055] The technical solutions of the application will be described clearly and completely in combination with the accompanying drawings of the specification. Obviously, the described embodiments are only some of the embodiments of the application, but not all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the application.

[0056] Embodiment one

[0057] The embodiment of the application provides a non-destructive testing robot weld automatic tracking system based on multi-modal sensing, as shown in the figure, which comprises: Figure 1

[0058] A data acquisition module is configured to acquire point cloud data through a laser scanner, acquire RGB-D image data through an RGB-D camera, and acquire attitude data through an inertial measurement unit.

[0059] ​Specifically, the data acquisition module includes a laser scanner, an RGB-D camera and an inertial measurement unit (IMU), which are mounted on a mobile platform with a chassis structure combining crawler belts and magnetic wheels driven independently on the left and right sides. The left and right crawler belts are controlled by independent drive motors, allowing differential steering. The laser scanner is installed at the front end of the chassis and has a working wavelength of 650 nm, which can generate two-dimensional profile data of the weld and the surrounding surface, and then accumulate the weld three-dimensional point cloud through the displacement of the robot. The RGB-D camera has a resolution of 1280x1024 and can simultaneously collect color images and depth images of the weld area, which is used to supplement the weld feature information of the laser scanner under conditions such as occlusion and reflection. The inertial measurement unit is used to measure the attitude angle and angular velocity information of the robot during travel, providing attitude reference for multi-sensor data fusion, including roll angle, pitch angle and yaw angle.

[0060] The data preprocessing module is used for preprocessing the point cloud data, the RGB-D image data and the attitude data to obtain preprocessed multi-modal data.

[0061] Further, the step of obtaining preprocessed multi-modal data includes:

[0062] The point cloud data is denoised, down-sampled and calibrated in the coordinate system, and the random sample consensus method is used to fit the bevel surface to extract the weld centerline feature points.

[0063] The RGB-D image data is subjected to weld area edge detection, depth image denoising and feature point matching to supplement the weld edge feature information.

[0064] The attitude data is subjected to zero offset correction and low-pass filtering to obtain stable attitude angle and angular velocity information.

[0065] Specifically, the original point cloud data output by the line laser scanner is first subjected to statistical filtering denoising, i.e. calculating the average distance of each point neighborhood and removing outliers deviating from the mean value by more than twice the standard deviation, thereby removing isolated points caused by spot reflection, surface contamination or sensor jitter. Subsequently, a voxel grid down-sampling method is used to divide the point cloud into fixed-size cubic voxels, and the center point in each voxel represents the voxel, thereby reducing the number of points and the amount of calculation while maintaining the weld structure features. After completing the down-sampling, the point cloud data is converted from the laser scanner coordinate system to the robot body coordinate system through an external parameter calibration matrix, realizing the consistency of different sensor data. Finally, a random sample consensus plane fitting method is used to fit the bevel surface of the weld bevel in the point cloud, remove background points that do not belong to the bevel plane, and extract weld centerline feature points according to the fitted bevel geometric features, providing an accurate three-dimensional geometric basis for subsequent trajectory calculation.

[0066] The color image and depth map obtained from the RGB-D camera are first subjected to edge detection of the weld area by a Sobel operator or a Canny operator to obtain the preliminary contour of the weld in the image. For the depth map data, median filtering is used to remove depth noise caused by reflection, scattering or sensor ranging error, and the depth gradient edge features are preserved by bilateral filtering. Then, the feature points of the color image and the depth map are matched by ORB or SIFT algorithm, and the weld edge points detected by vision are associated with the point cloud feature points, so as to supplement the missing weld feature information of the laser scanning under certain attitude or shielding conditions.

[0067] The raw attitude data obtained from the inertial measurement unit are first subjected to zero offset correction, that is, the average values of acceleration and angular velocity are collected for a certain time in the stationary state of the robot, which are used as zero offset values and are deducted from the subsequent data to eliminate the influence of sensor drift. Then, a first-order low-pass filter or a complementary filtering algorithm is used to smooth the attitude angle and angular velocity signals to suppress high-frequency noise and retain the true motion trend, so as to obtain stable attitude information.

[0068] Through the above steps, the system obtains multi-modal data that has been subjected to noise suppression, feature extraction and coordinate alignment processing, which not only retains the detailed features of the weld space geometry, but also ensures the spatio-temporal consistency between different data sources, providing high-quality input data for subsequent multi-sensor fusion and trajectory generation.

[0069] The data fusion module is used to synchronize the preprocessed multi-modal data and output continuous weld space trajectory points through a Kalman fusion algorithm;

[0070] Further, the step of outputting continuous weld space trajectory points, as shown in Figure 2 , comprises:

[0071] aligning the preprocessed point cloud data and image data in space coordinates and synchronizing them in time to generate fused laser-vision multi-modal data;

[0072] using a Kalman filtering algorithm to fuse the laser-vision multi-modal data and the attitude data collected by the inertial measurement unit to estimate the three-dimensional position and attitude state of the weld center line in real time;

[0073] Through the state updating and prediction mechanism of Kalman filtering, continuous weld space trajectory points are output.

[0074] Specifically, first, the pre-processed point cloud data is converted with the RGB-D image data in the spatial coordinate system. The image feature point coordinates in the RGB-D camera coordinate system are converted to the laser scanner coordinate system through an external parameter calibration matrix, which is obtained from a calibration board or a known feature object. The ICP iterative closest point algorithm in the point cloud library is used to register the laser scanning point cloud and the dense point cloud generated by the RGB-D depth map back projection, so as to reduce the spatial deviation caused by installation errors and sensor drift.

[0075] To ensure the correspondence of multi-sensor data at the fusion moment, the laser scanning data, RGB-D data and IMU data are unified to the robot system clock by using hardware trigger synchronization or timestamp alignment. For sensor data with sampling frequency difference, linear interpolation is used to fill in the missing data points on the time axis, so as to ensure that the fusion input is the same time reference.

[0076] The spatially and temporally aligned laser-vision multi-modal data are input into the Kalman filter as observations, and the IMU attitude data are used as predictions for state estimation. The system state vector is defined as:

[0077] ;

[0078] wherein, is the three-dimensional position of the weld center line in the robot coordinate system, are the roll angle, pitch angle and yaw angle respectively, is the velocity component of the weld point. The prediction model is obtained by integrating the IMU angular velocity and linear acceleration to obtain the prior estimate of the position and attitude:

[0079] ;

[0080] wherein, denotes the state transition matrix, denotes the control matrix, denotes the control input, denotes the process noise. The observation model uses the weld center point coordinates in the laser-vision fusion point cloud and the attitude calculated from the RGB-D edge features as the observation vector wherein, denotes the observation matrix, denotes the observation noise. The Kalman filter is executed in two stages of prediction and update, wherein the prediction stage calculates the prior value of the weld center line position and attitude at the current moment based on the IMU data, and the update stage performs weighted fusion of the laser-vision fusion result and the prior value to obtain the optimal estimate. During the Kalman filter iteration process, the position in the state vector is output in real time by the system as a continuous weld space trajectory point.

[0081] Through the above processing, the obtained weld space trajectory points have high accuracy and continuity in both three-dimensional position and orientation, which can effectively reflect the spatial morphology of the real weld and provide reliable basic data for subsequent three-dimensional curve fitting and end pose generation.

[0082] The trajectory generation module is used to perform three-dimensional B-spline fitting based on the arc length parameterization of the weld space trajectory points, and combine it with the surface normal estimated by the point cloud to generate the end pose sequence to obtain the target motion trajectory.

[0083] Furthermore, the steps to obtain the target's trajectory are as follows:

[0084] The continuous weld space trajectory points output by the Kalman filter are sorted and parameterized according to the arc length parameter;

[0085] Based on parametric trajectory points, a smooth and continuous weld trajectory curve is generated using a three-dimensional B-spline curve fitting method.

[0086] The local normal information of the weld surface is estimated using point cloud data, and the corresponding end pose sequence is generated by combining the fitted curve.

[0087] Specifically, the continuous weld space trajectory points output by the Kalman filter Sort by acquisition order and calculate the three-dimensional Euclidean distance between adjacent trajectory points. The total arc length of the trajectory is obtained by accumulating the distance. And the cumulative distance of each trajectory point Normalized to parameters This yields the arc-length parameterized trajectory point set. .

[0088] Selecting the B-spline order and node vectors, fitting is performed based on the arc-length parameterized point set, respectively for... Perform independent spline fitting to obtain the set of control points for the 3D B-spline curve. The least squares method is used to solve for the control point positions during the fitting process, and a smoothing factor can be introduced. Noise interference is suppressed, ultimately generating a smooth three-dimensional weld seam trajectory curve. ,in This represents the spatial path along which the robot's end effector moves along the weld seam.

[0089] From the original point cloud data, a local point set within a certain radius near the fitted curve is extracted. The normal vector of the local point cloud is calculated using the normal estimation method of the PCL library, based on k-nearest neighbor search. The normal vector is corrected for directional consistency to ensure the continuity of the normal vector along the trajectory direction.

[0090] According to the fitted curve and the corresponding normal vector Determine the attitude of the end effector, where the position part is taken from... The attitude component is defined by the normal vector as the Z-axis direction at the end point, and the trajectory tangent vector... The X-axis direction, representing the endpoint, is used to determine the Y-axis direction via cross product, forming an orthogonal attitude matrix. Combine position and orientation into a homogeneous transformation matrix:

[0091] ;

[0092] With fixed step size Sampling is performed to output a continuous end-effector pose sequence. This is for the motion control module to call directly.

[0093] Through the above steps, discrete weld trajectory points that may contain noise can be transformed into a smooth and continuous three-dimensional path, and a precise end pose sequence can be generated by combining local surface normal vectors, thereby ensuring the stable fit and correct posture of the robot detection probe along the weld trajectory.

[0094] Furthermore, when generating the end pose sequence, the trajectory generation module employs a quaternion-based pose smoothing method to perform spherical linear interpolation on adjacent poses to avoid abrupt changes in the robot arm's pose and ensure the continuity and stability of the trajectory pose.

[0095] Specifically, each end pose in the trajectory is converted from a rotation matrix to a quaternion representation for interpolation calculation. Then, for two adjacent pose quaternions, the angle between them is calculated, and when the angle is large, the shortest path is selected for interpolation to prevent pose flipping. Next, several intermediate pose points are generated between adjacent poses using a spherical linear interpolation method according to a preset interpolation ratio, thus making the pose transition smoother. When the difference between adjacent poses is extremely small, linear interpolation is used to avoid numerical instability.

[0096] The generated intermediate interpolated quaternions are normalized before use to ensure that the rotational attitude they represent is valid and free of accumulated errors. Finally, the smoothed quaternions are converted back into rotation matrices and combined with the corresponding trajectory position coordinates to form a continuous and smooth end-effector pose sequence. This method effectively avoids shocks and trajectory instability caused by sudden changes in the robotic arm's attitude, ensuring stable contact and consistent attitude of the detection probe during weld tracking and non-destructive testing.

[0097] By using the above-mentioned attitude smoothing method based on quaternion interpolation, the present invention can significantly improve the continuity and stability of the attitude change of the robotic arm end effector, avoid the shaking or detachment of the detection probe from the weld seam caused by sudden attitude changes, and effectively ensure the accuracy of weld seam tracking and the quality of non-destructive testing.

[0098] a motion control module for driving the robot to perform the detection task according to a target motion trajectory, including differential path keeping control based on lateral deviation and hybrid position-force control of the end of the mechanical arm based on feedforward-PID and force feedback to realize automatic tracking of the weld seam and non-destructive detection.

[0099] Further, the steps of the differential path keeping control include, as shown in Figure 3

[0100] Real-time measurement of the lateral deviation of the weld centerline and the central axis of the robot by a laser sensor;

[0101] Calculation of the deviation by a PID control algorithm to generate a speed difference value;

[0102] Adjustment of the PWM duty cycle of the left and right wheels according to the calculated speed difference value to dynamically adjust the speed ratio of the left and right wheels;

[0103] Stability verification of the adjusted wheel speed ratio to ensure accurate tracking of the robot along the weld centerline.

[0104] Specifically, the laser sensor scans the front weld area in real time, calculates the lateral deviation Ay of the weld centerline in the robot coordinate system by a weld feature extraction algorithm, and the deviation reflects the relative position difference between the weld center and the central axis of the robot. Then, Ay is input into the PID controller as input, and the PID controller calculates the corresponding speed difference value Av according to the proportional, integral and differential parameters, which is used to correct the speed difference of the two sides of the track. Then, the control system dynamically adjusts the duty cycle of the PWM signal of the left and right drive wheels according to Av, so that the speed difference of the wheels reaches the compensation effect, thereby correcting the direction of the robot and gradually approaching and keeping on the weld centerline. Finally, during the motion process, the system verifies the stability of the path adjustment through the odometer and IMU attitude data, and when the robot attitude changes too fast or shows a trend of oscillation, the PID parameters are automatically adjusted or the Av change rate is reduced to ensure smooth operation and weld tracking accuracy.

[0105] The differential path keeping control detects the lateral deviation of the weld centerline and the central axis of the robot in real time, and dynamically adjusts the speed ratio of the left and right wheels by combining the PID algorithm, so that the robot can correct the drift in time and move stably along the weld centerline during the movement, thereby significantly improving the accuracy of automatic tracking of the weld seam and the stability of path keeping.

[0106] Further, the steps of the hybrid position-force control include, as shown in Figure 4

[0107] Providing the spatial deviation of the weld center by a laser vision module;

[0108] ​​The deviation is calculated by inverse kinematics of the robot arm, and the joint angles are calculated to compensate for the positional space deviation, so as to realize the position adjustment of the probe center to the weld center line.

[0109] The probe contact force feedback value is collected in real time by the six-axis force sensor, and compared with the preset target force value.

[0110] The PID controller is used to adjust the Z-axis displacement of the robot arm to realize constant contact force adjustment.

[0111] Specifically, the laser vision module collects three-dimensional point cloud and image information of the weld area in real time, calculates the spatial deviation of the weld center relative to the coordinate system of the robot arm end effector through weld feature extraction and spatial positioning algorithm. Then, the spatial deviation is input into the robot arm kinematics solving module, the forward and inverse kinematics equations are established based on the DH parameter model of the robot arm, and the Jacobian inverse matrix method is used to iteratively solve the joint angle change that can compensate for the deviation, and then the target angle of each joint is generated, realizing the end position modulation, so that the probe center accurately aligns with the weld center line.

[0112] On the basis of position modulation, the six-axis force sensor measures the contact force between the probe and the weld surface in real time, and compares the force values in each direction, especially the contact force in the Z-axis direction, with the preset target force value to obtain the force error signal. Subsequently, the PID controller calculates the displacement compensation amount in the Z-axis direction according to the error signal, and adds it to the motion command of the robot arm end, realizing dynamic adjustment of the contact force.

[0113] Through the mixed position-force control mode, the robot arm end probe can continuously maintain the position of aligning with the weld center line in the three-dimensional space, while ensuring the constant contact force between the probe and the weld surface through force feedback adjustment, so as to ensure the detection path accuracy and detection signal stability under complex weld morphology, and improve the reliability and continuity of non-destructive testing.

[0114] Further, the mixed position-force control adjusts the control strategy adaptively according to the weld type, adopts constant force control mode when detecting a planar weld, and adopts mixed control mode combining position adjustment and force adjustment when detecting a curved surface weld.

[0115] Through adaptive adjustment of the control strategy, the probe can accurately follow the weld path and maintain stable contact force, improving the reliability and data quality of non-destructive testing.

[0116] Further, the system further comprises a process error correction module, and the implementation logic of the process error correction is as follows:

[0117] When the laser signal is lost, switch to pure vision mode, detect the robot moving speed to 50% of the preset speed, when the laser signal is restored and lasts for 2 seconds, restore the initial state;

[0118] When the contact force exceeds 50% of the preset initial value, the mechanical arm immediately retreats 3mm and recouples, and when the force value is stable at the preset initial value for 1 second, the initial state is restored;

[0119] When communication interruption occurs, the robot immediately stops moving and is manually remotely controlled back to the original position by an external controller.

[0120] Process error correction can cope with sensor failure, force anomaly or communication interruption in real time during detection, avoid equipment damage and detection precision decline through automatic switching of working mode, adjustment of motion parameters or emergency stop, and ensure the safety, stability and task continuity of the system under complex working conditions.

[0121] Example two

[0122] The application also proposes a multi-modal sensor-based automatic weld tracking method for non-destructive testing robots, as shown in Figure 5. In a shipyard steel structure manufacturing workshop, this scheme is applied to the ultrasonic non-destructive testing task of vertical welds. After the robot starts, the line laser scanner installed at the front end of the mobile platform collects three-dimensional point cloud data of the weld area with an accuracy of ±0.1mm, while the RGB-D camera at the top acquires 1280×1024 resolution color images and depth information, and the IMU provides platform attitude angle and angular velocity data at a frequency of 100Hz.

[0123] The collected multi-modal data is preprocessed: the point cloud data is first downsampled by voxel grid filtering, then isolated noise is removed by statistical filtering, and the coordinate system is calibrated with the workpiece reference system; then the random sample consensus method is used to fit the bevel slope and extract the three-dimensional feature points of the weld centerline; the RGB-D data is edge detected and depth denoised to extract supplementary weld edge features; the IMU data is zero offset corrected and low pass filtered to obtain smooth and reliable attitude information.

[0124] The processed point cloud and image data are calibrated and spatially aligned, and synchronized using timestamps, forming fused laser-vision data, which are input into the extended Kalman filter algorithm together with the IMU attitude information. The filter estimates the three-dimensional position and attitude of the weld centerline in real time in the prediction-update cycle, and outputs a continuous and smooth trajectory point sequence.

[0125] The trajectory generation module sorts the trajectory points output by the Kalman filter according to arc length, fits a three-dimensional B-spline curve, and obtains a continuous and smooth weld seam curve; meanwhile, the corresponding end pose sequence is generated in combination with the local surface normal estimated by the point cloud, and the adjacent poses are smoothed by using a quaternion spherical linear interpolation method to avoid abrupt changes in the pose of the robot arm.

[0126] In the motion control stage, the mobile platform uses differential path keeping control, measures the lateral deviation Ay of the weld center line and the center axis in real time, calculates the speed difference Av by PID, adjusts the PWM duty cycle of the left and right tracks, and makes the platform always travel along the weld center line; the robot arm end performs hybrid position-force control, first compensates the spatial position deviation by inverse kinematics calculation to realize accurate alignment of the probe to the weld center; then the contact force between the probe and the workpiece surface is monitored in real time by a six-axis force sensor, and the Z-axis displacement is adjusted by PID to ensure constant contact force during detection.

[0127] In the whole weld detection process, the robot can stably follow the weld path and keep the constant contact force between the detection probe and the weld seam, the detection accuracy meets the preset requirements, and the process correction capability is provided when the laser signal is lost, the force value is abnormal, and the communication is interrupted, ensuring the safety and reliability of the vertical weld detection task in the complex workshop environment.

[0128] In the whole weld detection process, the robot can stably follow the weld path and keep the constant contact force between the detection probe and the weld seam, the detection accuracy meets the preset requirements, and the process correction capability is provided when the laser signal is lost, the force value is abnormal, and the communication is interrupted, ensuring the safety and reliability of the vertical weld detection task in the complex workshop environment.

[0129] Although embodiments of the present application have been shown and described, it is to be understood that various modifications, substitutions, replacements and changes can be made to these embodiments without departing from the principles and spirit of the present application, and the scope of the present application is defined by the appended claims and their equivalents.

Claims

1. A non-destructive testing robot weld seam automatic tracking system based on multi-modal sensing, characterized in that, The method comprises the following steps: a data acquisition module is used to acquire point cloud data through a laser scanner, acquire RGB-D image data through an RGB-D camera, and acquire attitude data through an inertial measurement unit; a data preprocessing module is used to preprocess the point cloud data, the RGB-D image data, and the attitude data, and obtain preprocessed multi-modal data; a data fusion module is used to synchronize the preprocessed multi-modal data, and output continuous weld seam spatial trajectory points through a Kalman fusion algorithm; a trajectory generation module is used to perform three-dimensional B-spline fitting according to the weld seam spatial trajectory points according to arc length parameterization, generate end pose sequences in combination with surface normal estimated by point cloud data, and obtain a target motion trajectory; a motion control module is used to drive a robot to perform a detection task according to the target motion trajectory, including differential path keeping control based on lateral deviation and hybrid position-force control of a robot end based on feedforward-PID and force feedback, so as to realize automatic tracking and non-destructive testing of a weld seam.

2. The multi-modal sensor based non-destructive inspection robotic weld seam automatic tracking system of claim 1, wherein, The step of obtaining the preprocessed multi-modal data comprises the following steps: the point cloud data is subjected to denoising, downsampling, and coordinate system calibration processing, and a random sample consensus method is used to fit the bevel surface to extract weld seam center line feature points; the RGB-D image data is subjected to weld seam region edge detection, depth image denoising, and feature point matching to supplement weld seam edge feature information; the attitude data is subjected to zero offset correction and low-pass filtering processing to obtain stable attitude angle and angular velocity information.

3. The multi-modal sensor based non-destructive inspection robotic weld seam automatic tracking system of claim 1, wherein, The step of outputting the continuous weld seam spatial trajectory points comprises the following steps: the preprocessed point cloud data and image data are subjected to spatial coordinate alignment and time synchronization to generate fused laser-vision multi-modal data; a Kalman filtering algorithm is used to fuse the laser-vision multi-modal data and the attitude data collected by the inertial measurement unit to estimate the three-dimensional position and attitude state of the weld seam center line in real time; the continuous weld seam spatial trajectory points are output through the state updating and prediction mechanism of the Kalman filtering.

4. The multi-modal sensor based non-destructive inspection robotic weld seam automatic tracking system of claim 1, wherein, The step of obtaining the target motion trajectory comprises the following steps: the continuous weld seam spatial trajectory points output by the Kalman filtering are sorted and parameterized according to arc length parameters; a three-dimensional B-spline curve fitting method is used to generate a smooth and continuous weld seam trajectory curve based on the parameterized trajectory points; local normal information of a weld seam surface is estimated by using point cloud data, and corresponding end pose sequences are generated in combination with the fitted curve.

5. The multi-modal sensor based non-destructive inspection robotic weld seam automatic tracking system of claim 4, wherein, When the trajectory generation module generates the end pose sequences, a quaternion interpolation-based attitude smoothing processing method is used to perform spherical linear interpolation on adjacent poses to avoid sudden changes in the attitude of the robot and ensure the continuity and stability of the trajectory attitude.

6. The multi-modal sensor based non-destructive inspection robotic weld seam automatic tracking system of claim 1, wherein, The step of the differential path keeping control comprises the following steps: the lateral deviation between the weld seam center line and the robot central axis is measured in real time by using a laser sensor; a speed difference value is generated by calculating the deviation through a PID control algorithm; the PWM duty cycle of the left and right wheels is adjusted according to the calculated speed difference value to dynamically adjust the speed ratio of the left and right wheels; the stability of the adjusted wheel speed ratio is verified to ensure that the robot accurately tracks the weld seam center line.

7. The multi-modal sensor based non-destructive inspection robotic weld seam automatic tracking system of claim 1, wherein, The step of the hybrid position-force control comprises the following steps: the spatial deviation of the weld seam center is provided by a laser vision module; The deviation is calculated by inverse kinematics of the robot arm, and each joint angle is calculated to compensate for the position space deviation and realize the position adjustment of the probe center to the weld center line. The probe contact force feedback value is collected in real time by a six-dimensional force sensor, and compared with the preset target force value. The Z-axis displacement of the robot arm is adjusted by a PID controller to realize constant force adjustment.

8. The multi-modal sensor based non-destructive inspection robotic weld seam automatic tracking system of claim 7, wherein, The hybrid position-force control adjusts the control strategy adaptively according to the weld type. When the weld is detected as a plane weld, the constant force control mode is adopted; when the weld is detected as a curved surface weld, the hybrid control mode combining position adjustment and force adjustment is adopted.

9. The multi-modal sensor based non-destructive inspection robotic weld seam automatic tracking system of claim 1, wherein, The system also includes a process error correction module, and the implementation logic of the process error correction is as follows: When the laser signal is lost, switch to the pure vision mode, and detect that the robot moving speed is reduced to 50% of the preset speed. When the laser signal is restored and lasts for 2 seconds, the initial state is restored. When the contact force exceeds 50% of the preset initial value, the robot arm immediately retreats 3 mm and recouples. When the force value is stable at the preset initial value for 1 second, the initial state is restored. When communication interruption occurs, the robot immediately stops moving, and is manually remotely controlled back to the original position by an external controller.

10. A method for automatic tracking of a weld seam by a robot for non-destructive testing based on multi-modal sensing, characterized in that The method comprises the following steps: Obtain point cloud data by a laser scanner, obtain RGB-D image data by an RGB-D camera, and obtain attitude data by an inertial measurement unit. Preprocess the point cloud data, the RGB-D image data and the attitude data to obtain preprocessed multi-modal data. Synchronize the preprocessed multi-modal data, and output continuous weld space trajectory points by a Kalman fusion algorithm. According to the weld space trajectory points, perform three-dimensional B-spline fitting according to arc length parameterization, and generate end pose sequences in combination with the surface normal estimated by the point cloud to obtain a target motion trajectory. Drive the robot to perform a detection task according to the target motion trajectory, including differential path keeping control based on lateral deviation and hybrid position-force control of the robot arm end based on feedforward-PID and force feedback, so as to realize automatic tracking and nondestructive testing of the weld.

Images