Slope truss robot installation system based on multi-algorithm fusion

Abstract

The present application relates to the field of slope engineering construction and robot control technology, and discloses a slope truss robot installation system based on multi-algorithm fusion, which comprises a slope adaptive truss guide rail, a slope leveling mechanical arm and a slope adaptive truss robot main body built by adjustable end effector through slope surface; a dustproof and anti-light RGB-D camera and a slope anti-vibration IMU are built in the slope outdoor anti-interference sensing hardware layer; a slope scene optimization perception processing unit is constructed through a slope anti-interference identification and segmentation module, a slope multi-target tracking module, a slope 6D attitude fine estimation module, a slope coordinate system conversion module and a vision-inertial fusion module; the slope surface collaborative decision planning unit is used in combination with the size of the prefabricated part CAD model and the slope terrain to generate the motion trajectory fitting the slope surface and optimize the trajectory parameters; finally, the trajectory instruction is received by the execution control unit, and the robot is controlled to complete the prefabricated part grabbing, slope carrying and flexible assembly.

Description

Technical Field

[0001] This invention relates to the field of slope engineering construction and robot control technology, specifically a slope truss robot installation system based on multi-algorithm fusion. Background Technology

[0002] Precast slope protection is a core process in slope reinforcement systems. It primarily uses factory-standardized precast concrete components, which are precisely assembled, interlocked, and integrally embedded on-site. These components directly bear the crucial roles of soil constraint, runoff erosion resistance, and shallow landslide prevention. In the overall slope reinforcement process, this process acts as a bridge between preliminary steps such as slope trimming and foundation treatment, and works in conjunction with drainage systems and anchoring structures to form a stable protection system. Through the strength of the components themselves and the integrity of the joints, it effectively inhibits soil weathering, spalling, and sliding, significantly improving the long-term stability of the slope. Compared to traditional protection methods, it offers higher construction efficiency, controllable quality, and convenient repair, enabling rapid and reliable reinforcement. It is a key link in ensuring the safety, durability, and compliant delivery of slope projects, holding an irreplaceable core position in the reinforcement of soil and weathered rock slopes.

[0003] Traditional precast slope protection installation mainly relies on manual slope work combined with simple hoisting. The overall construction mode has four major defects: First, manual installation on the slope is highly dangerous, with a high risk of falls from heights. Second, the slope site is in an open outdoor environment, subject to multiple interferences such as dust, strong sunlight, and vegetation obstruction, making it difficult for sensing equipment to stably and continuously identify precast components. Third, the slope lacks a unified flat benchmark, and traditional technical methods cannot achieve high-precision slope pose estimation, resulting in low assembly accuracy of precast components. Fourth, a sensing-control closed-loop system adapted to the slope conditions has not been formed, making it easy for collisions to occur during installation, causing damage to precast components or slope structure. Summary of the Invention

[0004] (a) Technical problems to be solved

[0005] To address the shortcomings of existing technologies, this invention provides a slope truss robot installation system based on multi-algorithm fusion, solving the challenges of continuous, high-precision installation of prefabricated components in complex outdoor slope environments and the high risks associated with manual slope work. The system integrates a slope-adaptive truss robot, outdoor anti-interference sensing hardware, a slope optimization sensing algorithm layer, and a slope collaborative decision-making and control layer, constructing a closed-loop sensing-control system of "segmentation-tracking-precision estimation-slope coordinate transformation." It employs a YOLOv8-seg+ByteTrack+FoundationPose algorithm stack, optimized for slope scenarios, and combines real-time IMU-acquired robot end-effector motion data with a motor controller to achieve high-precision, compliant assembly of slope protection prefabricated components. This invention balances robustness, accuracy, and real-time performance, replacing high-risk manual work, improving installation efficiency and safety, and possesses high engineering application value and patentability.

[0006] (II) Technical Solution

[0007] To achieve the above objectives, the present invention provides the following technical solution: a slope truss robot installation system based on multi-algorithm fusion, comprising a slope-adaptive truss robot body, a slope outdoor anti-interference perception hardware layer, a slope scene optimization perception processing unit, a slope collaborative decision planning unit, and an execution control unit;

[0008] The main body of the slope-adaptive truss robot includes: a slope-adjustable end effector, a slope-adaptive truss guide rail, and a slope-leveling robotic arm.

[0009] The outdoor anti-interference sensing hardware layer for slopes includes: a dustproof and light-resistant RGB-D camera and a slope vibration-resistant IMU;

[0010] The perception processing unit for slope scene optimization includes: a slope anti-interference identification and segmentation module, a slope multi-target tracking module, a slope 6D attitude fine estimation module, a slope coordinate system transformation module, and a vision-inertial navigation fusion module.

[0011] The slope anti-interference identification and segmentation module uses the YOLOv8-seg model to segment prefabricated component instances and suppress slope background interference.

[0012] The slope multi-target tracking module uses the ByteTrack tracker to track the 2D motion trajectory of the precast slope surface and optimizes the trajectory prediction parameters;

[0013] The slope 6D attitude estimation module uses the FoundationPose model to output the high-precision 6D attitude of the precast component relative to the camera coordinate system and the local coordinate system of the slope.

[0014] The slope coordinate system transformation module is used to achieve accurate pose transformation between multiple coordinate systems;

[0015] The vision-inertial fusion module fuses FoundationPose visual pose data with IMU motion data and outputs corrected high-precision pose data.

[0016] The slope collaborative decision-making and planning unit is used to receive sensing data, combine the dimensions of the prefabricated component CAD model with the slope topography, generate a motion trajectory that fits the slope, and optimize the trajectory parameters.

[0017] The execution control unit is used to receive trajectory instructions and control the robot to complete prefabricated component grasping, slope handling and compliant assembly;

[0018] Preferably, the RGB-D camera is used to acquire RGB video streams, depth images and point cloud data of the slope scene, and the IMU is installed on the end effector of the gantry robot to acquire end attitude, angular velocity and acceleration data;

[0019] Preferably, the slope multi-target tracking module uses ByteTrack to continuously track the prefabricated component, and triggers the slope 6D attitude fine estimation module only when grabbing or measuring the slope pose.

[0020] Preferably, the working steps of the slope anti-interference identification and segmentation module are as follows:

[0021] A1. Extract RGB video streams and depth images of slope scenes under different slope, lighting, dust, and vegetation obstruction conditions from data collected by RGB-D cameras. Simultaneously, collect pure slope background data to form the original dataset. , Represents a single frame image. Represents the total number of samples;

[0022] A2. Use LabelMe to annotate the prefabricated component instance to divide the outline and generate an annotation file. , Represents a pixel-level segmentation mask used to distinguish prefabs from the background; and is classified and statistically analyzed according to interference type;

[0023] ;

[0024] in, Represents the number of a certain type of interference samples; Represents light interference. Represents dust disturbance. Represents vegetation cover;

[0025] A3. Calculate the image signal-to-noise ratio to quantify the degree of dust noise interference. The calculation formula is:

[0026] ;

[0027] in, Represents the image signal-to-noise ratio. Represents the image mean; filtering The data are valid samples;

[0028] A4. Calculate the mean image brightness under different light intensities. Determine the range of brightness disturbance And the coefficient of image fogging caused by dust. ;

[0029] ;

[0030] in, represent ;

[0031] ;

[0032] in, The slope of the concentration-atomization coefficient fitting; This represents the measured dust concentration on the slope. Represents the fit intercept;

[0033] A5. Targeted data augmentation is performed to address different types of interference, specifically:

[0034] a. Light interference: ;

[0035] in, Represents the brightness offset;

[0036] b. Dust disturbance: ;

[0037] in, Represents atomized background image;

[0038] C. Background blending enhancement: ;

[0039] in, The fusion weight coefficients represent the images of the prefabricated components; Image representing a prefabricated component; Image representing a pure slope background;

[0040] A6. Divide the augmented dataset into training, validation, and test sets in a 7:2:1 ratio.

[0041] A7. Initialize the YOLOv8-seg model by setting the input size, batch size, and initial learning rate, then calculate the total model loss using the following formula:

[0042] ;

[0043] in, Represents the total loss; Represents classification loss; Represents the bounding box loss weights; Represents the bounding box loss; Represents the segmentation loss weight; Represents the segmentation loss;

[0044] when When, execute step A8; when When the time comes, stop training and save the current model parameters;

[0045] A8. Monitor the loss curves of the training and validation sets in real time. If the loss of either the training or validation set increases for five consecutive rounds, reduce the learning rate. , Representing the initial learning rate, and analyzing the background false positive rate, the formula is as follows:

[0046] ;

[0047] in, Represents the background false positive rate; The background information was mistakenly identified as the number of prefabricated components. Represents the number of correctly identified background elements, when Retrain using supplementary background samples until... ;

[0048] A9. Calculate the segmentation accuracy, background suppression rate, and segmentation success rate of the prefabricated instance to verify the effectiveness of the YOLOv8-seg model. The calculation formula is as follows:

[0049] ;

[0050] ;

[0051] ;

[0052] in, Represents the segmentation accuracy of precast component instances; Represents the number of pixels in the correctly segmented prefab; The number of pixels that represent the background that was mistakenly identified as a prefab. This represents the number of pixels that were missed in the prefab segmentation. This represents the number of background pixels that were correctly identified. The number of pixels that represent prefabricated components that were mistakenly identified as background.

[0053] Represents background suppression rate; Represents the total number of background pixels;

[0054] This represents the success rate of the segmentation. This represents the number of correctly segmented frames; Represents the total number of frames;

[0055] A10. Calculate the lighting, dust, and occlusion scenarios respectively. When different scenarios fluctuation At that time, supplement with corresponding low-precision interference data and retrain until the scene-to-scene communication is improved. fluctuation ;

[0056] when At that time, supplement with background samples for iterative training until... ;

[0057] when At that time, low-precision frames are extracted to analyze the interference type and corresponding data is added. The model is then fine-tuned until... ;

[0058] Preferably, in A7, The formula for segmentation loss is:

[0059] ;

[0060] in, This represents the total number of candidate regions predicted by the model; Representing the The true labels of each candidate region; The representative model predicts the first The probability that each candidate region is a prefabricated component;

[0061] ;

[0062] in, The prefabricated component boundary box represented by the model prediction; Represents the actual boundary box of the prefabricated component; represent The coordinates of the center point; represent The coordinates of the center point; represent and The square of the Euclidean distance; It can cover and The length of the diagonal of the smallest rectangle; This represents the aspect ratio penalty item;

[0063] ;

[0064] in, Represents image height; Represents the image width; Represents pixel-level loss weights; Representing pixels The probability of being predicted as a prefabricated component;

[0065] Preferably, the working steps of the slope multi-target tracking module are as follows:

[0066] B1. Based on the instance segmentation results of YOLOv8-seg, extract the 2D bounding box of the prefab in each frame image. Calculate the coordinates of the center point of the bounding box. ,in , , Represents the frame number, then... , bounding box width Bounding box height As the initial detection box input for ByteTrack tracking;

[0067] B2. Initialize ByteTrack tracker core parameters: Set the detection box matching threshold Trajectory smoothness threshold Trajectory interruption rate threshold Trajectory prediction error threshold The Kalman filter is initialized for trajectory prediction, and the Kalman filter state equation is:

[0068]

[0069] in, Represents the state vector. Represents the coordinates of the center of the frame; Represents the speed of the center pixel; Represents the width and height of the frame; Represents the state transition matrix; Represents the control matrix; Represents control quantity; Represents process noise;

[0070] B3. For the detection box in the current frame, first predict the position of the trajectory box using Kalman filtering, and then calculate the matching coefficient between the detection box and the predicted box. The calculation formula is:

[0071] ;

[0072] in, Represents the area of ​​the detection box in the current frame; Represents the area of ​​the Kalman filter prediction box;

[0073] when If a match is completed and the Kalman filter state of the trajectory is updated, step B4 is executed; otherwise, it is determined as a non-match, a new trajectory is created for the non-matched detection box, and then step B4 is executed.

[0074] B4. Calculate the 2D motion speed of the trajectory, trajectory smoothness, and trajectory interruption rate respectively. The calculation formulas are as follows:

[0075] ;

[0076] in, Representing the Frame trajectory 2D motion speed; Representing the The x-coordinate of the frame center; Representing the The vertical coordinate of the frame center; Represents the frame interval time;

[0077] ;

[0078] in, Represents trajectory smoothness; Represents the total number of frames in the trajectory; Representing the Frame trajectory 2D motion speed;

[0079] ;

[0080] in, Represents the trajectory interruption rate; Represents the number of interrupted tracks; Represents the total number of trajectories;

[0081] B5. Based on the calculation results in B4, optimize the trajectory prediction parameters, and then calculate the trajectory prediction error. ;

[0082] a. When At the same time, adjust the noise covariance of the Kalman filter process. This reduces the sensitivity of predicted values ​​to noise;

[0083] b. When At the same time, lower the detection box matching threshold. Improve the error tolerance of trajectory matching;

[0084] C. Trajectory prediction error The calculation formula is:

[0085] ;

[0086] in, Represents the number of verification frames; Representing the The x-coordinate of the center of the frame prediction box; Representing the The x-coordinate of the actual detection center of the frame; Representing the The center ordinate of the frame prediction box; Representing the The vertical coordinate of the center of the actual detection frame;

[0087] B6. Repeat steps B1 to B5, iteratively optimizing the parameters until... Then, save the parameters as optimal parameters. The current parameters are the optimal parameters. Based on the optimal parameters, output the 2D motion trajectory sequence of the precast component slope. ;

[0088] Preferably, the working steps of the slope 6D attitude fine estimation module are as follows:

[0089] C1. Based on the CAD model of the prefabricated component, extract the three-dimensional geometric features of the model and generate a model point cloud template. For the first point cloud in the model points The covariance matrix of the nearest neighbor points is decomposed by SVD and normalized, and the surface normal vector is directly calculated. Then, based on the coordinates and normal vectors of the point and its neighboring points, the angular feature histogram is calculated and normalized to obtain the feature descriptor. Simultaneously calculate the model center point , serving as the reference point for attitude estimation, is expressed as:

[0090] ;

[0091] in, , , The first in the representative model point cloud The coordinates of the points; The total number of points in the representative model's point cloud;

[0092] ;

[0093] in, , , The first in the representative model point cloud Coordinates of points

[0094] C2. Based on the output 2D motion trajectory sequence of the precast slope, extract the depth data within the trajectory bounding box in the corresponding frame's RGB-D depth image. ;

[0095] C3, via camera intrinsic parameter matrix Convert pixel coordinates to point clouds in the camera coordinate system. The expression is:

[0096] ;

[0097] in, , Represents the camera's focal length; , Represents the coordinates of the principal point;

[0098] ;

[0099] C4. Statistical filtering is used to remove outliers with a distance from the mean exceeding 3σ, resulting in the denoised real-time point cloud of the precast component. The expression is:

[0100] ;

[0101] in, Represents the first in real-time point cloud One effective point; , , Representing the The coordinates of a point in the camera coordinate system; Represents the number of valid points in the real-time point cloud;

[0102] C5. Set the feature matching threshold Number of ICP iterations Reprojection error threshold ;

[0103] C6. Calculate real-time point clouds Chinese characteristics With model template Chinese characteristics Feature matching The expression is:

[0104] ;

[0105] when At that time, the judgment , For matching point pairs, where, Represents real-time point cloud The Middle The three-dimensional coordinates of each valid point in the camera coordinate system; Representative model template The Middle The three-dimensional coordinates of each point;

[0106] C7. Based on the matching point pairs, calculate the initial rotation matrix of the preform relative to the camera coordinate system. Translation vector , and then with , ICP iterative optimization is performed with initial values, and the error of the corresponding point pair is calculated in each iteration. and minimize the error function Update rotation matrix Translation vector Specifically:

[0107] ;

[0108] in, This represents the number of valid matching pairs; Represents the predicted pixel coordinates after the model points are projected; Represents the actual pixel coordinates of the real-time point;

[0109] ;

[0110] ;

[0111] C8. Calculate reprojection error The calculation formula is:

[0112] ;

[0113] in, , Represents the predicted coordinates of the real-time point cloud projected onto the pixel plane; , Represents the actual pixel coordinates;

[0114] when When this happens, return to C6 to re-execute feature matching and ICP optimization; when Determine the final 6D pose in the camera coordinate system. ,in Represents a 3×3 rotation matrix; Represents the translation vector;

[0115] C9. Select three non-collinear reference points on the slope and obtain their coordinates in the camera coordinate system. , , and coordinates in the local coordinate system of the slope. , , ;

[0116] C10. Solve for the transformation matrix from the camera coordinate system to the local coordinate system of the slope. Then, the 6D pose in the camera coordinate system is converted into the pose in the local coordinate system of the slope. The expression is:

[0117] ;

[0118] in, Represents a 3×3 rotation matrix; Represents a 3×1 translation vector;

[0119] ;

[0120] ;

[0121] C11, will Decomposed into Euler angles The expression is:

[0122] ;

[0123] ;

[0124] ;

[0125] C13. Finally, the 6D attitude of the slope in the local coordinate system is obtained. It outputs the complete result including position and Euler angles of the pose, expressed as:

[0126] ;

[0127] in, Represents the position coordinates of the slope in the local coordinate system;

[0128] Preferably, the working steps of the slope coordinate system transformation module are as follows:

[0129] D1. Obtain the 4×4 homogeneous transformation matrix from the IMU coordinate system to the camera coordinate system. The expression is:

[0130] ;

[0131] in, Represents a 3×3 rotation matrix from the IMU to the camera; Represents the translation vector;

[0132] D2, based on And C10 Calculate the transformation matrix from IMU to slope. The pose of the IMU data in the slope system is obtained, expressed as:

[0133] ;

[0134] D3. Extract the three non-collinear reference points on the slope selected by C9 to verify the conversion error from the camera / IMU system to the slope system. The expression is:

[0135] ;

[0136] in, This represents the true coordinates of the slope reference point in the local coordinate system of the slope. The camera / IMU coordinates representing the reference point are obtained through / The calculated coordinates obtained from the transformation;

[0137] when At that time, Translation vector Adjustments and recalculations are required. until Save the adjusted version ,in, This represents the conversion error threshold;

[0138] Preferably, the working steps of the vision-inertial fusion module are as follows:

[0139] E1. Extract the visual pose and IMU pose, and annotate them respectively. The expression is:

[0140] Visual pose:

[0141] ;

[0142] ;

[0143] in, The coordinates of the precast component position calculated visually under the local coordinate system of the slope; These represent the positions on the X, Y, and Z axes, respectively. Euler angles representing the orientation of the precast component as visually calculated in the local coordinate system of the slope; These represent roll angle, pitch angle, and yaw angle, respectively.

[0144] IMU pose:

[0145] ;

[0146] ;

[0147] in, This represents the coordinates of the precast component position calculated by the IMU in the local coordinate system of the slope after transformation; These represent the positions on the X, Y, and Z axes, respectively. The Euler angles represent the attitude of the precast component calculated by the IMU in the local coordinate system of the slope after transformation. These represent roll angle, pitch angle, and yaw angle, respectively.

[0148] E2, Based on conversion error Determine visual weights respectively and IMU weights The expression is:

[0149] Visual weight:

[0150] ;

[0151] IMU weights:

[0152] ;

[0153] E3. After fusing the position and attitude by weighted averaging separately, the fused position and fused attitude are established. The formula expression is:

[0154] Fusion location:

[0155] ;

[0156] Fusion posture:

[0157] ;

[0158] E4. Calculate the error between the fused pose and the true value of the reference point. The formula expression is:

[0159] ;

[0160] when At that time, a linear correction is made to the fusion position. until ,in, This represents the corrected merged position, i.e., the current point; Represents the error threshold of the true value;

[0161] Preferably, the working steps of the slope collaborative decision-making and planning unit are as follows:

[0162] F1, Construct the current point To the target point Continuous motion trajectory points:

[0163] ;

[0164] in, Representing the The position of each trajectory point in the local coordinate system of the slope; Represents the trajectory interpolation coefficients. Represents the trajectory point number; Represents the total number of points on the trajectory;

[0165] F2. Calculate the trajectory parameters, specifically including the total trajectory length, the distance between trajectory points, the trajectory speed, the trajectory running time, and the trajectory smoothness. The expression is:

[0166] ;

[0167] in, Represents the total length of the trajectory;

[0168] ;

[0169] in, Represents the distance between trajectory points;

[0170] ;

[0171] in, Represents the speed of the trajectory movement; Represents the reference speed; Represents the slope-velocity correction factor;

[0172] ;

[0173] in, Represents the trajectory execution time; This represents the time required to complete the entire trajectory;

[0174] ;

[0175] in, This represents the smoothness of the trajectory.

[0176] Compared with existing technologies, this invention provides a slope truss robot installation system based on multi-algorithm fusion, which has the following advantages:

[0177] 1. Innovative Optimization and Application of Algorithm Stack in Slope Scenarios: Targeting complex working conditions such as sudden changes in outdoor lighting, dust, and vegetation obstruction on slopes, we have optimized the three core algorithms of YOLOv8-seg, ByteTrack, and FoundationPose. We have designed a slope coordinate system transformation module to achieve one-to-one mapping of multi-sensor coordinate systems. This effectively solves the industry pain points of weak anti-interference ability and insufficient pose transformation accuracy in outdoor slope sensing, providing reliable algorithmic support for precise system operation.

[0178] 2. The “tracking-triggering-precision calculation” mode for slope adaptation: The triggering logic of multi-target tracking and pose estimation is optimized. FoundationPose precision calculation is triggered only at key nodes such as precast component grabbing and slope pose measurement. At the same time, the core algorithm is lightweighted to adapt to the operation requirements of outdoor embedded industrial control computers. While ensuring the accuracy of operation, the hardware deployment cost is reduced and the outdoor adaptability of the system is improved.

[0179] 3. Slope vision-motor control closed loop: Integrating high-precision pose data after vision-inertial navigation fusion, combined with the slope terrain error compensation mechanism, the robot actuator motion parameters are adjusted in real time to achieve smooth assembly of prefabricated components on the slope, effectively avoiding assembly errors caused by sudden trajectory changes and positioning deviations, and improving the quality and stability of slope installation operations.

[0180] 4. System-level integration of slope scenarios: Integrating the main body of the slope-adaptive truss robot, outdoor anti-interference perception hardware and scenario-based optimization algorithms, a complete closed-loop system of "perception-decision-execution" is constructed to realize the fully automated operation of the entire process of grasping, handling and assembling prefabricated slope components. It successfully replaces the manual operation mode in high and steep slopes and high-risk environments, reduces personnel safety risks and improves operation efficiency and standardization. Attached Figure Description

[0181] Figure 1 This is a schematic diagram of the system of the present invention. Detailed Implementation

[0182] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. It is worth noting that this application also relates to prior art. Since prior art is well known to those skilled in the art, it will not be described in detail in this application.

[0183] Please see Figure 1 The slope truss robot installation system based on multi-algorithm fusion includes a slope-adaptive truss robot body, an outdoor anti-interference perception hardware layer for slopes, a perception and processing unit optimized for slope scenarios, a slope collaborative decision-making and planning unit, and an execution control unit.

[0184] The main body of the slope-adaptive truss robot includes: an adjustable end effector, a slope-adaptive truss guide rail, and a slope-leveling robotic arm.

[0185] The outdoor anti-interference sensing hardware layer for slopes includes: a dustproof and light-resistant RGB-D camera and a slope vibration-resistant IMU; the RGB-D camera is used to collect RGB video streams, depth images and point cloud data of the slope scene, and the IMU is installed on the end effector of the gantry robot to collect end attitude, angular velocity and acceleration data.

[0186] The perception processing unit for slope scene optimization includes: slope anti-interference identification and segmentation module, slope multi-target tracking module, slope 6D attitude fine estimation module, slope coordinate system transformation module, and vision-inertial navigation fusion module;

[0187] The slope anti-interference recognition and segmentation module is based on YOLOv8-seg. It first collects slope image data under multiple interference conditions, and constructs a robust dataset through targeted augmentation such as signal-to-noise ratio filtering, illumination, dust, and background fusion. Then, it trains the model with the joint loss of classification, bounding box regression, and segmentation, and iteratively optimizes it based on indicators such as background false detection rate, segmentation accuracy, and background suppression rate. It can achieve accurate instance segmentation of precast components and strong background interference suppression in complex slope environments. It can stably identify targets under harsh outdoor conditions such as sudden changes in illumination, dust, and vegetation obstruction, and significantly reduce the background false detection rate, providing reliable perception results for subsequent tracking and pose estimation.

[0188] The slope multi-target tracking module is based on ByteTrack to realize 2D trajectory tracking of precast slopes. It takes the detection box output by YOLOv8-seg as input, uses Kalman filtering for trajectory prediction and matching, calculates trajectory speed, smoothness, interruption rate and other indicators in real time, and iteratively optimizes filtering parameters and matching thresholds. Pose estimation is triggered only during measurement and grasping, forming a continuous and stable sequence of precast motion trajectories. This can ensure tracking continuity and low latency, reduce invalid pose calculations, improve the overall efficiency of the system and the accuracy of trajectory prediction, and adapt to the dynamic operation requirements of slopes.

[0189] The slope 6D attitude estimation module is based on the FoundationPose framework. First, a point cloud template with normal vectors and feature descriptors is generated from the prefabricated CAD model. Then, the real-time point cloud of the target is extracted from the RGB-D data and denoised. The 6D attitude in the camera coordinate system is obtained through feature matching and ICP iterative optimization. Then, the coordinate transformation is solved by the slope reference point, and the high-precision position and Euler angle in the local coordinate system of the slope are output. It can get rid of the dependence on a large amount of labeled data and use CAD prior to achieve high-precision and strong generalization of 6D pose calculation, which meets the alignment accuracy requirements of slope assembly.

[0190] The slope coordinate system transformation module first calibrates the extrinsic parameter matrix from the IMU to the camera, and then, combined with the transformation relationship from the camera to the local coordinate system of the slope, derives the unified transformation matrix from the IMU to the slope. It uses non-collinear reference points on the slope to verify the transformation error and iteratively corrects it, establishing a unified mapping between the camera, IMU, and local coordinate system of the slope, eliminating the problem of inconsistent poses of multiple sensors, and providing a unified reference for subsequent vision-inertial navigation fusion and robot control.

[0191] The vision-inertial navigation fusion module combines the visual pose output by FoundationPose with the attitude and motion data calculated by the IMU in a unified slope coordinate system, and performs weighted fusion according to the transformation error. It also performs linear error correction on the result, which can complement the advantages of vision and inertial navigation, suppress jumps and drifts caused by visual occlusion and vibration interference, and output a smoother, more stable and interference-resistant high-precision pose, thus improving the reliability of the system in complex slope environments.

[0192] The slope collaborative decision-making and planning unit receives the fused high-precision pose, prefabricated component dimensions, and slope terrain information. It generates a smooth trajectory from the current point to the target point through linear interpolation and calculates key parameters such as trajectory length, point spacing, speed, running time, and smoothness. It also performs adaptive speed correction based on the slope, which can generate a smooth and safe motion trajectory that conforms to the slope terrain, balancing work efficiency and motion stability, and avoiding impacts on the robot and prefabricated components caused by sudden trajectory changes.

[0193] The execution control unit receives trajectory instructions from the decision planning output and drives the leveling robotic arm and adjustable end effector of the slope-adaptive truss robot to complete the precise gripping of prefabricated components, slope handling and compliant assembly. It transforms high-precision perception and decision-making into reliable physical actions, and relies on truss guide rails and end-effector compliant control to ensure the positioning accuracy and assembly quality of slope installation, thereby realizing automated, standardized and high-efficiency installation of slope prefabricated components.

[0194] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A slope truss robot installation system based on multi-algorithm fusion, characterized in that, It includes a slope-adaptive truss robot body, an outdoor anti-interference perception hardware layer for slopes, a perception and processing unit for slope scene optimization, a slope collaborative decision-making and planning unit, and an execution control unit. The main body of the slope-adaptive truss robot includes: a slope-adjustable end effector, a slope-adaptive truss guide rail, and a slope-leveling robotic arm. The outdoor anti-interference sensing hardware layer for slopes includes: a dustproof and light-resistant RGB-D camera and a slope vibration-resistant IMU; The perception processing unit for slope scene optimization includes: a slope anti-interference identification and segmentation module, a slope multi-target tracking module, a slope 6D attitude fine estimation module, a slope coordinate system transformation module, and a vision-inertial navigation fusion module. The slope anti-interference identification and segmentation module uses the YOLOv8-seg model to segment prefabricated component instances and suppress slope background interference. The slope multi-target tracking module uses the ByteTrack tracker to track the 2D motion trajectory of the precast slope surface and optimizes the trajectory prediction parameters; The slope 6D attitude estimation module uses the FoundationPose model to output the high-precision 6D attitude of the precast component relative to the camera coordinate system and the local coordinate system of the slope. The slope coordinate system transformation module is used to achieve accurate pose transformation between multiple coordinate systems; The vision-inertial fusion module fuses FoundationPose visual pose data with IMU motion data and outputs corrected high-precision pose data. The slope collaborative decision-making and planning unit is used to receive sensing data, combine the dimensions of the prefabricated component CAD model with the slope topography, generate a motion trajectory that fits the slope, and optimize the trajectory parameters. The execution control unit is used to receive trajectory instructions and control the robot to complete prefabricated component grasping, slope handling and compliant assembly.

2. The slope truss robot installation system based on multi-algorithm fusion according to claim 1, characterized in that, The RGB-D camera is used to acquire RGB video streams, depth images, and point cloud data of the slope scene. The IMU is installed on the end effector of the gantry robot and is used to acquire end attitude, angular velocity, and acceleration data.

3. The slope truss robot installation system based on multi-algorithm fusion according to claim 1, characterized in that, The slope multi-target tracking module uses ByteTrack to continuously track prefabricated components, and triggers the slope 6D attitude fine estimation module only when grabbing or measuring slope pose.

4. The slope truss robot installation system based on multi-algorithm fusion according to claim 2, characterized in that, The working steps of the slope anti-interference identification and segmentation module are as follows: A1. Extract RGB video streams and depth images of slope scenes under different slope, lighting, dust, and vegetation obstruction conditions from data collected by RGB-D cameras. Simultaneously, collect pure slope background data to form the original dataset. , Represents a single frame image. Represents the total number of samples; A2. Use LabelMe to annotate the prefabricated component instance to divide the outline and generate an annotation file. , Represents a pixel-level segmentation mask used to distinguish prefabs from the background; and is classified and statistically analyzed according to interference type; ； in, Represents the number of a certain type of interference samples; Represents light interference. Represents dust disturbance. Represents vegetation cover; A3. Calculate the image signal-to-noise ratio to quantify the degree of dust noise interference. The calculation formula is: ； in, Represents the image signal-to-noise ratio. Represents the image mean; filtering The data are valid samples; A4. Calculate the mean image brightness under different light intensities. Determine the range of brightness disturbance And the coefficient of image fogging caused by dust. ; ； in, represent ; ； in, The slope of the concentration-atomization coefficient fitting; This represents the measured dust concentration on the slope. Represents the fit intercept; A5. Targeted data augmentation is performed to address different types of interference, specifically: a. Light interference: ; in, Represents the brightness offset; b. Dust disturbance: ; in, Represents atomized background image; C. Background blending enhancement: ; in, The fusion weight coefficients represent the images of the prefabricated components; Image representing a prefabricated component; Image representing a pure slope background; A6. Divide the augmented dataset into training, validation, and test sets in a 7:2:1 ratio. A7. Initialize the YOLOv8-seg model by setting the input size, batch size, and initial learning rate, then calculate the total model loss using the following formula: ； in, Represents the total loss; Represents classification loss; Represents the bounding box loss weights; Represents the bounding box loss; Represents the segmentation loss weight; Represents the segmentation loss; when When, execute step A8; when When the time comes, stop training and save the current model parameters; A8. Monitor the loss curves of the training and validation sets in real time. If the loss of either the training or validation set increases for five consecutive rounds, reduce the learning rate. , Representing the initial learning rate, and analyzing the background false positive rate, the formula is as follows: ； in, Represents the background false positive rate; The background information was mistakenly identified as the number of prefabricated components. Represents the number of correctly identified background elements, when Retrain using supplementary background samples until... ; A9. Calculate the segmentation accuracy, background suppression rate, and segmentation success rate of the prefabricated instance to verify the effectiveness of the YOLOv8-seg model. The calculation formula is as follows: ； ； ； in, Represents the segmentation accuracy of precast component instances; Represents the number of pixels in the correctly segmented prefab; The number of pixels that represent the background that was mistakenly identified as a prefab. This represents the number of pixels that were missed in the prefab segmentation. This represents the number of background pixels that were correctly identified. The number of pixels that represent prefabricated components that were mistakenly identified as background. Represents background suppression rate; Represents the total number of background pixels; This represents the success rate of the segmentation. This represents the number of correctly segmented frames; Represents the total number of frames; A10. Calculate the lighting, dust, and occlusion scenarios respectively. When different scenarios fluctuation At that time, supplement with corresponding low-precision interference data and retrain until the scene-to-scene communication is improved. fluctuation ; when At that time, supplement with background samples for iterative training until... ; when At that time, low-precision frames are extracted to analyze the interference type and corresponding data is added. The model is then fine-tuned until... .

5. The slope truss robot installation system based on multi-algorithm fusion according to claim 4, characterized in that, In A7, The formula for segmentation loss is: ； in, This represents the total number of candidate regions predicted by the model; Representing the The true labels of each candidate region; The representative model predicts the first The probability that each candidate region is a prefabricated component; ； in, The prefabricated component boundary box represented by the model prediction; Represents the actual boundary box of the prefabricated component; represent The coordinates of the center point; represent The coordinates of the center point; represent and The square of the Euclidean distance; It can cover and The length of the diagonal of the smallest rectangle; This represents the aspect ratio penalty item; ； in, Represents image height; Represents the image width; Represents pixel-level loss weights; Representing pixels The probability of being predicted as a prefabricated component.

6. The slope truss robot installation system based on multi-algorithm fusion according to claim 3, characterized in that, The working steps of the slope multi-target tracking module are as follows: B1. Based on the instance segmentation results of YOLOv8-seg, extract the 2D bounding box of the prefab in each frame image. Calculate the coordinates of the center point of the bounding box. ,in , , Represents the frame number, then... , bounding box width Bounding box height As the initial detection box input for ByteTrack tracking; B2. Initialize ByteTrack tracker core parameters: Set the detection box matching threshold Trajectory smoothness threshold Trajectory interruption rate threshold Trajectory prediction error threshold The Kalman filter is initialized for trajectory prediction, and the Kalman filter state equation is: ； in, Represents the state vector. Represents the coordinates of the center of the frame; Represents the speed of the center pixel; Represents the width and height of the frame; Represents the state transition matrix; Represents the control matrix; Represents control quantity; Represents process noise; B3. For the detection box in the current frame, first predict the position of the trajectory box using Kalman filtering, and then calculate the matching coefficient between the detection box and the predicted box. The calculation formula is: ； in, Represents the area of ​​the detection box in the current frame; Represents the area of ​​the Kalman filter prediction box; when If a match is completed and the Kalman filter state of the trajectory is updated, step B4 is executed; otherwise, it is determined as a non-match, a new trajectory is created for the non-matched detection box, and then step B4 is executed. B4. Calculate the 2D motion speed of the trajectory, trajectory smoothness, and trajectory interruption rate respectively. The calculation formulas are as follows: ； in, Representing the Frame trajectory 2D motion speed; Representing the The x-coordinate of the frame center; Representing the The vertical coordinate of the frame center; Represents the frame interval time; ； in, Represents trajectory smoothness; Represents the total number of frames in the trajectory; Representing the Frame trajectory 2D motion speed; ； in, Represents the trajectory interruption rate; Represents the number of interrupted tracks; Represents the total number of trajectories; B5. Based on the calculation results in B4, optimize the trajectory prediction parameters, and then calculate the trajectory prediction error. ; a. When At the same time, adjust the noise covariance of the Kalman filter process. This reduces the sensitivity of predicted values ​​to noise; b. When At the same time, lower the detection box matching threshold. Improve the error tolerance of trajectory matching; C. Trajectory prediction error The calculation formula is: ； in, Represents the number of verification frames; Representing the The x-coordinate of the center of the frame prediction box; Representing the The x-coordinate of the actual detection center of the frame; Representing the The center ordinate of the frame prediction box; Representing the The vertical coordinate of the center of the actual detection frame; B6. Repeat steps B1 to B5, iteratively optimizing the parameters until... Then, save the parameters as optimal parameters. The current parameters are the optimal parameters. Based on the optimal parameters, output the 2D motion trajectory sequence of the precast component slope. .

7. The slope truss robot installation system based on multi-algorithm fusion according to claim 3, characterized in that, The working steps of the slope 6D attitude precision estimation module are as follows: C1. Based on the CAD model of the prefabricated component, extract the three-dimensional geometric features of the model and generate a model point cloud template. For the first point cloud in the model points The covariance matrix of the nearest neighbor points is decomposed by SVD and normalized, and the surface normal vector is directly calculated. Then, based on the coordinates and normal vectors of the point and its neighboring points, the angular feature histogram is calculated and normalized to obtain the feature descriptor. Simultaneously calculate the model center point , serving as the reference point for attitude estimation, is expressed as: ； in, , , The first in the representative model point cloud The coordinates of the points; The total number of points in the representative model's point cloud; ； in, , , The first in the representative model point cloud Coordinates of points C2. Based on the output 2D motion trajectory sequence of the precast slope, extract the depth data within the trajectory bounding box in the corresponding frame's RGB-D depth image. ; C3, via camera intrinsic parameter matrix Convert pixel coordinates to point clouds in the camera coordinate system. The expression is: ； in, , Represents the camera's focal length; , Represents the coordinates of the principal point; ； C4. Statistical filtering is used to remove outliers with a distance from the mean exceeding 3σ, resulting in the denoised real-time point cloud of the precast component. The expression is: ； in, Represents the first in real-time point cloud One effective point; , , Representing the The coordinates of a point in the camera coordinate system; Represents the number of valid points in the real-time point cloud; C5. Set the feature matching threshold Number of ICP iterations Reprojection error threshold ; C6. Calculate real-time point clouds Chinese characteristics With model template Chinese characteristics Feature matching The expression is: ； when At that time, the judgment , For matching point pairs, where, Represents real-time point cloud The Middle The three-dimensional coordinates of each valid point in the camera coordinate system; Representative model template The Middle The three-dimensional coordinates of each point; C7. Based on the matching point pairs, calculate the initial rotation matrix of the preform relative to the camera coordinate system. Translation vector , and then with , ICP iterative optimization is performed with initial values, and the error of the corresponding point pair is calculated in each iteration. and minimize the error function Update rotation matrix Translation vector Specifically: ； in, This represents the number of valid matching pairs; Represents the predicted pixel coordinates after the model points are projected; Represents the actual pixel coordinates of the real-time point; ； ； C8. Calculate reprojection error The calculation formula is: ； in, , Represents the predicted coordinates of the real-time point cloud projected onto the pixel plane; , Represents the actual pixel coordinates; when When this happens, return to C6 to re-execute feature matching and ICP optimization; when Determine the final 6D pose in the camera coordinate system. ,in Represents a 3×3 rotation matrix; Represents the translation vector; C9. Select three non-collinear reference points on the slope and obtain their coordinates in the camera coordinate system. , , and coordinates in the local coordinate system of the slope. , , ; C10. Solve for the transformation matrix from the camera coordinate system to the local coordinate system of the slope. Then, the 6D pose in the camera coordinate system is converted into the pose in the local coordinate system of the slope. The expression is: ； in, Represents a 3×3 rotation matrix; Represents a 3×1 translation vector; ； ； C11, will Decomposed into Euler angles The expression is: ； ； ； C13. Finally, the 6D attitude of the slope in the local coordinate system is obtained. It outputs the complete result including position and Euler angles of the pose, expressed as: ； in, This represents the position coordinates of the slope in the local coordinate system.

8. The slope truss robot installation system based on multi-algorithm fusion according to claim 7, characterized in that, The working steps of the slope coordinate system transformation module are as follows: D1. Obtain the 4×4 homogeneous transformation matrix from the IMU coordinate system to the camera coordinate system. The expression is: ； in, Represents a 3×3 rotation matrix from the IMU to the camera; Represents the translation vector; D2, based on And C10 Calculate the transformation matrix from IMU to slope. The pose of the IMU data in the slope system is obtained, expressed as: ； D3. Extract the three non-collinear reference points on the slope selected by C9 to verify the conversion error from the camera / IMU system to the slope system. The expression is: ； in, This represents the true coordinates of the slope reference point in the local coordinate system of the slope. The camera / IMU coordinates representing the reference point are obtained through / The calculated coordinates obtained from the transformation; when At that time, Translation vector Adjustments and recalculations are required. until Save the adjusted version ,in, This represents the conversion error threshold.

9. The slope truss robot installation system based on multi-algorithm fusion according to claim 8, characterized in that, The working steps of the vision-inertial navigation fusion module are as follows: E1. Extract the visual pose and IMU pose, and annotate them respectively. The expression is: Visual pose: ； ； in, The coordinates of the precast component position calculated visually under the local coordinate system of the slope; These represent the positions on the X, Y, and Z axes, respectively. Euler angles representing the orientation of the precast component as visually calculated in the local coordinate system of the slope; These represent roll angle, pitch angle, and yaw angle, respectively. IMU pose: ； in, This represents the coordinates of the precast component position calculated by the IMU in the local coordinate system of the slope after transformation; These represent the positions on the X, Y, and Z axes, respectively. The Euler angles represent the attitude of the precast component calculated by the IMU in the local coordinate system of the slope after transformation. These represent roll angle, pitch angle, and yaw angle, respectively. E2, Based on conversion error Determine visual weights respectively and IMU weights The expression is: Visual weight: ； IMU weights: ； E3. After fusing the position and attitude by weighted averaging separately, the fused position and fused attitude are established. The formula expression is: Fusion location: ； Fusion posture: ； E4. Calculate the error between the fused pose and the true value of the reference point. The formula expression is: ； when At that time, a linear correction is made to the fusion position. until ,in, This represents the corrected merged position, i.e., the current point; This represents the error threshold of the true value.

10. The slope truss robot installation system based on multi-algorithm fusion according to claim 9, characterized in that, The working steps of the slope collaborative decision-making and planning unit are as follows: F1, Construct the current point To the target point Continuous motion trajectory points: ； in, Representing the The position of each trajectory point in the local coordinate system of the slope; Represents the trajectory interpolation coefficients. Represents the trajectory point number; Represents the total number of points on the trajectory; F2. Calculate the trajectory parameters, specifically including the total trajectory length, the distance between trajectory points, the trajectory speed, the trajectory running time, and the trajectory smoothness. The expression is: ； in, Represents the total length of the trajectory; ； in, Represents the distance between trajectory points; ； in, Represents the speed of the trajectory movement; Represents the reference speed; Represents the slope-velocity correction factor; ； in, Represents the trajectory execution time; This represents the time required to complete the entire trajectory; ； in, This represents the smoothness of the trajectory.

Images