An Adaptive Anti-Tilt Method and System for Wheeled Legged Robots Based on Vision-Inertial Fusion

Abstract

This invention relates to an adaptive anti-tilt method and system for wheeled and legged robots based on visual-inertial fusion, belonging to the field of robot motion control technology. The method includes: First, fusing visual road surface semantics, IMU acceleration, and angular velocity information to estimate the ground adhesion coefficient online in real time, and dynamically calibrating the theoretical tilt angle calculated from the visual path curvature accordingly. Second, the system dynamically adjusts the weight ratio of visual feedforward control and IMU feedback control based on the confidence level of visual path recognition, achieving an adaptive fusion of aggressive prediction and robust disturbance resistance. Finally, the fused final tilt command is precisely mapped to the angle control command of the drive joint servo motors through an analytical solution of the inverse kinematics of a five-bar linkage based on spatial geometry and the cosine theorem, achieving error-free conversion from macroscopic attitude to low-level execution. This system improves the robot's cornering stability and safety in dynamic unstructured environments.

Description

Technical Field

[0001] This invention relates to an adaptive anti-tilt method and system for wheeled legged robots based on visual-inertial fusion. Background Technology

[0002] In existing technologies, wheeled-legged robots typically employ two methods when cornering at high speeds: one is to rely solely on the underlying balance controller to maintain upright posture, using differential steering via the left and right drive wheels; the other is to introduce vision or forward-looking radar to acquire path curvature, calculate the theoretical roll angle, and then directly send the calculations to the actuators (feedforward control). Patents such as CN119659795A and CN119690059A propose methods for preventing rollover and speed planning in vehicles or robots, utilizing visual path prediction combined with vehicle dynamics for attitude pre-control.

[0003] Disadvantages of existing technology:

[0004] 1. Pure feedforward models lack environmental adaptability: Existing visual feedforward anti-roll control is mostly based on ideal rigid body dynamics and a fixed ground friction coefficient (assuming sufficient grip at all times). However, in practical applications, when water stains or sand appear on the ground causing a sudden change in the friction coefficient, or when the tires slip, the fixed theoretical roll angle will lead to understeering or direct sideslip and loss of control of the robot.

[0005] 2. Poor robustness of a single sensing source: Existing technologies blindly trust the confidence of visual sensors. When visual edges become blurred or sudden changes in illumination cause visual prediction to fail or occur, incorrect feedforward instructions can lead to sudden changes in the action of the actuator, which can then cause fatal accidents.

[0006] 3. The underlying execution mapping mechanism is vague: Existing solutions mostly stay at the macroscopic theoretical level of "outputting the desired roll angle". For parallel five-bar linkage wheel mechanism with highly nonlinear characteristics, there is a lack of specific, feasible and high-precision inverse kinematic mapping schemes to transform the macroscopic roll angle and virtual leg length into the underlying collaborative control command of the multi-joint servo motor without error. Summary of the Invention

[0007] The purpose of this invention is to solve the problem of sideslip and overturning caused by sudden environmental changes and nonlinearity of actuators when wheeled robots make high-speed turns on complex roads, and to provide an adaptive anti-rolling method and system for wheeled robots based on visual-inertial fusion.

[0008] To achieve the above objectives, the technical solution of the present invention is: an adaptive anti-tilt method for wheeled legged robots based on visual-inertial fusion, comprising:

[0009] Acquire image information of the road surface ahead, motion state information of the robot body, and actual angular velocity of the drive wheels;

[0010] Based on road image information, robot motion state information and actual angular velocity of drive wheels, the true adhesion coefficient between the current wheels and the ground is estimated online.

[0011] The curvature of the path ahead is calculated based on road surface image information, and the theoretical roll angle is calculated based on the motion state information of the robot body. The theoretical roll angle is dynamically calibrated based on the actual adhesion coefficient between the front wheels and the ground to obtain the visual feedforward roll angle.

[0012] The visual confidence level is assessed based on road image information. Based on the visual confidence level and the motion state information of the robot body, the weights of visual feedforward control and IMU feedback control are dynamically allocated. The visual feedforward tilt angle and IMU feedback tilt angle are fused to obtain the final tilt command.

[0013] Based on the final tilt command, the expected height of the virtual leg length on the left and right sides of the robot is calculated, and based on inverse kinematics calculation, the expected height is mapped into control commands for controlling the robot's posture adjustment mechanism to achieve active tilting of the robot body.

[0014] Furthermore, online estimation of the current true coefficient of adhesion between the wheel and the ground includes:

[0015] The tire slip ratio is calculated based on the actual angular velocity of the drive wheels and the longitudinal velocity of the robot. ;

[0016] A nonlinear friction observer based on slip ratio is constructed, and the true adhesion coefficient is estimated by combining the prior value of the basic friction coefficient provided by the road texture features extracted from road surface images. .

[0017] Furthermore, the theoretical roll angle is calculated based on the robot's motion state information; the theoretical roll angle is dynamically calibrated based on the actual adhesion coefficient between the front wheels and the ground to obtain the visual feedforward roll angle, including:

[0018] Based on the curvature of the path ahead , to obtain the turning radius ,in =1 / ;

[0019] Based on the robot's linear velocity v, gravitational acceleration g, and turning radius Calculate the theoretical roll angle The calculation formula is:

[0020]

[0021] Based on the true adhesion coefficient and / or tire slip ratio The theoretical roll angle is adjusted using a preset compensation function. Perform constraints and calibrations to output the visual feedforward tilt angle. .

[0022] Furthermore, the visual feedforward tilt angle and the IMU feedback tilt angle are fused, including:

[0023] Construct the formula for the fusion roll command:

[0024]

[0025] in, This is the final roll command. For visual feedforward tilt angle, The feedback roll angle is based on IMU data. For visual feedforward weights, For IMU feedback weights, and + = 1;

[0026] Based on visual confidence and environmental dynamics intensity Dynamically adjust visual feedforward weights With IMU feedback weights .

[0027] Furthermore, the expected height of the robot's virtual legs on the left and right sides is calculated, including:

[0028] According to the final roll command and robot wheelbase Calculate the height difference between the left and right virtual legs. The calculation formula is:

[0029]

[0030] Based on height difference The desired height of the left virtual leg is calculated based on the robot's current base height. And the expected height of the right virtual leg .

[0031] The present invention also provides an adaptive anti-tilt system for wheeled-legged robots based on visual-inertial fusion, for implementing the method described above, the system comprising:

[0032] The road surface estimation module is used to estimate the true adhesion coefficient between the current wheels and the ground online based on the acquired road surface image information, robot motion state information, and actual angular velocity of the drive wheels.

[0033] The feedforward calibration module is used to calculate the curvature of the path ahead based on road surface image information and calculate the theoretical roll angle, and to dynamically calibrate the theoretical roll angle based on the actual adhesion coefficient to generate a visual feedforward roll angle.

[0034] The fusion decision module is used to evaluate the visual confidence of road image information and dynamically fuse the visual feedforward roll angle and IMU feedback roll angle based on the visual confidence to generate the final roll command.

[0035] The kinematics mapping module is used to calculate the desired height of the virtual leg length based on the final tilt command, and to generate control commands for controlling the robot's posture adjustment mechanism through inverse kinematics calculation.

[0036] Furthermore, it also includes:

[0037] The perception module is used to acquire road surface image information, robot body motion state information, and actual angular velocity of the drive wheels.

[0038] Furthermore, the sensing module includes:

[0039] A visual perception unit is used to acquire road surface image information;

[0040] An inertial measurement unit (IMU) is used to acquire motion state information of the robot body.

[0041] An encoder is used to obtain the actual angular velocity of the drive wheels.

[0042] Furthermore, it also includes:

[0043] The actuator is communicatively connected to the kinematic mapping module and is used to respond to control commands to adjust the robot's posture in order to achieve active tilting of the robot.

[0044] The present invention also provides a computer-readable storage medium having stored thereon computer program instructions that can be executed by a processor, wherein when the processor executes the computer program instructions, it can implement the steps of the method described above.

[0045] Compared with the prior art, the present invention has the following beneficial effects:

[0046] 1. Extremely high robustness in extreme environments: Existing technologies are prone to loss of control and failure when vision is obstructed or lighting changes suddenly. This invention, through visual inertia dynamic weight allocation, instantly degrades to high-frequency IMU feedback control when visual confidence decreases, ensuring the absolute safety of the system under any harsh conditions.

[0047] 2. Possesses all-terrain dynamic adaptability: Existing technologies blindly rely on ideal curvature calculation feedforward. This invention introduces real-time estimation of tire slip ratio and ground adhesion coefficient, enabling the robot to autonomously limit the tilt amplitude when cornering on wet, slippery, sandy, or other low-adhesion surfaces. This achieves adaptive cornering that becomes more and more precise as it runs, completely solving the problem of steering sideslip.

[0048] 3. Solving the ambiguity of the underlying execution and strong system-level feasibility: This invention provides a rigorously derived set of analytical equations for the inverse kinematics of a five-bar linkage, which not only proves the physical feasibility of the technical solution, but also greatly improves the attitude response accuracy of the underlying actuator, and builds a complete closed-loop technical barrier from "top-level environmental perception" to "bottom-level motor servo". Attached Figure Description

[0049] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0050] The technical solution of the present invention will now be described in detail with reference to the accompanying drawings.

[0051] This invention provides an adaptive anti-tilt method for wheeled-legged robots based on visual-inertial fusion, comprising:

[0052] Acquire image information of the road surface ahead, motion state information of the robot body, and actual angular velocity of the drive wheels;

[0053] Based on road image information, robot motion state information and actual angular velocity of drive wheels, the true adhesion coefficient between the current wheels and the ground is estimated online.

[0054] The curvature of the path ahead is calculated based on road surface image information, and the theoretical roll angle is calculated based on the motion state information of the robot body. The theoretical roll angle is dynamically calibrated based on the actual adhesion coefficient between the front wheels and the ground to obtain the visual feedforward roll angle.

[0055] The visual confidence level is assessed based on road image information. Based on the visual confidence level and the motion state information of the robot body, the weights of visual feedforward control and IMU feedback control are dynamically allocated. The visual feedforward tilt angle and IMU feedback tilt angle are fused to obtain the final tilt command.

[0056] Based on the final tilt command, the expected height of the virtual leg length on the left and right sides of the robot is calculated, and based on inverse kinematics calculation, the expected height is mapped into control commands for controlling the robot's posture adjustment mechanism to achieve active tilting of the robot body.

[0057] The present invention also provides an adaptive anti-tilt system for wheeled-legged robots based on visual-inertial fusion, for implementing the method described above, the system comprising:

[0058] The perception module is used to acquire road surface image information, robot body motion state information, and actual angular velocity of the drive wheels;

[0059] The road surface estimation module is used to estimate the true adhesion coefficient between the current wheels and the ground online based on the acquired road surface image information, robot motion state information, and actual angular velocity of the drive wheels.

[0060] The feedforward calibration module is used to calculate the curvature of the path ahead based on road surface image information and calculate the theoretical roll angle, and to dynamically calibrate the theoretical roll angle based on the actual adhesion coefficient to generate a visual feedforward roll angle.

[0061] The fusion decision module is used to evaluate the visual confidence of road image information and dynamically fuse the visual feedforward roll angle and IMU feedback roll angle based on the visual confidence to generate the final roll command.

[0062] The kinematics mapping module is used to calculate the desired height of the virtual leg length based on the final tilt command, and to generate control commands for controlling the robot's posture adjustment mechanism through inverse kinematics calculation.

[0063] The actuator is communicatively connected to the kinematic mapping module and is used to respond to control commands to adjust the robot's posture in order to achieve active tilting of the robot.

[0064] The following is a detailed implementation process of the present invention.

[0065] In this embodiment, the visual perception unit may specifically employ an RGB-D camera with depth perception capability (such as a binocular depth camera) to ensure high-definition acquisition of road surface texture and edge features; the inertial measurement unit (IMU) employs a six-axis high-frequency sensor including a three-axis gyroscope and a three-axis accelerometer, with a data sampling rate of not less than 200Hz to ensure low-latency feedback of roll angular velocity and lateral acceleration; the actuator of the attitude adjustment mechanism is a high-frequency response joint servo motor, which receives angle control commands output by the kinematic mapping module and drives the five-bar linkage.

[0066] This invention provides an adaptive anti-tilt method and system for wheeled and legged robots based on visual-inertial fusion. The system includes: a visual perception module, a high-frequency six-axis IMU module, a body encoder, and joint servos driving a five-bar linkage.

[0067] like Figure 1 As shown, the implementation flow of the core control method of this invention is as follows:

[0068] Step 1: Multimodal sensing of road surface and real-time estimation of adhesion coefficient

[0069] The vision module extracts real-time road surface texture features (such as semantic information like water stains, sand, gravel, and grass) to provide prior values ​​for the basic friction coefficient. ;in The system uses semantic segmentation and empirical lookup table methods. The visual module inputs an RGB image from a forward-facing camera and utilizes a pre-trained lightweight semantic segmentation neural network (such as MobileNet-Seg or YOLOv8-Seg) to perform pixel-level classification of the road surface. The network outputs "semantic labels" for the road surface, such as: dry asphalt, wet asphalt, gravel, grass, waterlogged area, icy / snowy road, etc. Internally, the system uses a pre-defined friction coefficient empirical lookup table (LUT). If the road is identified as "dry asphalt," the table is consulted to output the prior value. If the road surface is identified as "slippery / waterlogged", the prior value is output by looking up the table. If identified as "gravel road surface", the prior value is output by looking up the table. If the road surface is identified as "icy or snowy", the prior value is output by looking up the table. .

[0070] Simultaneously, by combining the actual longitudinal acceleration obtained from the high-frequency IMU and the actual angular velocity obtained from the drive wheel encoder, the tire slip ratio is calculated in real time. Among them, the robot's longitudinal linear velocity Acquire longitudinal (X-axis) acceleration for IMU Then integrate over time: It came later.

[0071] The formula for calculating tire slip ratio is:

[0072]

[0073] In the formula, This is the actual angular velocity of the drive wheel. The radius of the drive wheel, Let this be the robot's longitudinal linear velocity. Then, a system based on the aforementioned slip ratio is constructed. A nonlinear sliding mode friction observer is used to estimate the true coefficient of adhesion between the wheel and the ground in real time online. .

[0074] Step 2: Online calibration of visual curvature feedforward and theoretical tilt angle

[0075] The vision module extracts the boundary and fits the centerline to calculate the curvature of the curve ahead. (turning radius) (The reciprocal of the original). The theoretically optimal roll angle is calculated using fundamental dynamics. An online real-time compensation mechanism is introduced: based on the true adhesion coefficient obtained in step 1... and the current actual slip ratio Establish correlation compensation function In the formula, For the adhesion coefficient mapping gain, This is the slip penalty coefficient. For the safe slip ratio threshold, For sign functions. When When the slip ratio decreases or becomes abnormal, the system dynamically lowers the maximum permissible roll angle threshold and outputs the calibrated feedforward desired roll angle. This is to prevent sideslip caused by excessive tilt angle exceeding the limit of the ground friction circle.

[0076] Step 3: Dynamic weight allocation for visual-inertial adaptive fusion.

[0077] Design a confidence-based vision-inertial fusion anti-tilt controller. To achieve dynamic and accurate weight allocation, this step specifically defines the parameters involved in the calculation and provides a mathematical model:

[0078] First, the motion state information of the robot body used here specifically includes: the lateral acceleration of the robot body collected in real time by the IMU. Actual roll angle and roll rate .

[0079] Secondly, the system evaluates the "visual confidence" of the vision module regarding the path curvature in real time. The specific evaluation method is as follows:

[0080] Extract the Laplacian variance of the road surface image as an image sharpness score. The calculation formula is as follows:

[0081]

[0082] In the formula, and These are the width and height (in pixels) of the image, respectively. The pixel grayscale values ​​are the result of the original image being processed by the Laplacian operator convolution. This represents the average pixel grayscale value of the entire image after convolution;

[0083] Edge detection algorithms (such as the Canny operator) are used to extract edge pixels of the preceding path. The ratio of the number of continuous edge pixels to the theoretical number of edge pixels is calculated to obtain a path continuity score. The calculation formula is as follows:

[0084]

[0085] In the formula, This represents the total number of consecutive edge pixels actually extracted by the vision module. This is the total number of ideal edge pixels calculated based on the current field of view and the preset path model.

[0086] Visual confidence The calculation formula is: In the formula, This is a normalized weighting constant. Simultaneously, the "environmental dynamic intensity" is calculated based on the robot's motion state information. This is used to characterize the severity of the current disturbance and attitude change of the organism: In the formula, This is the gain coefficient.

[0087] Based on the above evaluation results, the visual feedforward weights are dynamically adjusted. With IMU feedback weights The weight dynamic allocation function is defined as follows: , In the formula, Let be the environmental dynamic attenuation coefficient. Construct the fusion roll command formula: The control execution logic is as follows: when the road conditions are good and the edges are clear ( High) and the body is stable ( When the system calculates (low), the result is... Approaching the maximum value, it achieves aggressive high-speed predictive cornering; when the cornering is ambiguous, it leads to... Decrease, or slippery road surface causing sudden changes in lateral acceleration and angular velocity, leading to As the value increases, the exponential decay function will be forcibly weakened. Smoothly and rapidly increase the feedback weight of the high-frequency IMU. This causes the system to instantly degenerate into a robust and conservative control system that relies primarily on high-frequency anti-disturbance control of the underlying joint servo motors, thereby ensuring absolute stability like a roly-poly toy.

[0088] Step 4: Virtual leg length differential allocation based on five-bar nonlinear inverse kinematics.

[0089] According to the final roll command after fusion Calculate the virtual height difference that the left and right legs need to generate. ( (Based on wheelbase), the desired height coordinates of the left and right virtual legs are obtained. , , These are the desired height coordinates for the left and right virtual legs, respectively. To accurately translate the height command into the rotation angle of the hip joint servo motor driving the five-bar linkage, the controller incorporates a real-time floating-point inverse kinematics solver. Taking the left leg as an example, let the thigh length be... Lower leg length Hip joint distance First, calculate the spatial distance from the foot to the hip joint axis. , The desired horizontal coordinates of the foot are given; then, the target angle command of the active joint servo is precisely calculated using the law of cosines. , , These represent the squares of the thigh and lower leg lengths, respectively. This inverse kinematic equation eliminates the nonlinear execution error caused by the mechanical linkage mechanism, ensuring that the macroscopic anti-roll attitude can be executed accurately and without blind spots by the underlying servo motor.

[0090] The above are preferred embodiments of the present invention. Any changes made to the technical solution of the present invention that do not exceed the scope of the technical solution of the present invention shall fall within the protection scope of the present invention.

Claims

1. An adaptive anti-tilt method for wheeled legged robots based on visual-inertial fusion, characterized in that, include: Acquire image information of the road surface ahead, motion state information of the robot body, and actual angular velocity of the drive wheels; Based on road image information, robot motion state information and actual angular velocity of drive wheels, the true adhesion coefficient between the current wheels and the ground is estimated online. The curvature of the path ahead is calculated based on road surface image information, and the theoretical roll angle is calculated based on the motion state information of the robot body. The theoretical roll angle is dynamically calibrated based on the actual adhesion coefficient between the front wheel and the ground to obtain the visual feedforward roll angle. The visual confidence level is assessed based on road image information. Based on the visual confidence level and the motion state information of the robot body, the weights of visual feedforward control and IMU feedback control are dynamically allocated. The visual feedforward tilt angle and IMU feedback tilt angle are fused to obtain the final tilt command. The visual feedforward tilt angle and the IMU feedback tilt angle are fused, including: Construct the formula for the fusion roll command: in, For the final roll command, For visual feedforward tilt angle, The feedback roll angle is based on IMU data. For visual feedforward weights, For IMU feedback weights, and + = 1; Based on visual confidence and environmental dynamics intensity Dynamically adjust visual feedforward weights With IMU feedback weights ; Based on the final tilt command, the expected height of the virtual leg length on the left and right sides of the robot is calculated, and based on inverse kinematics calculation, the expected height is mapped into control commands for controlling the robot's posture adjustment mechanism to achieve active tilting of the robot body.

2. The adaptive anti-tilt method for wheeled legged robots based on visual-inertial fusion according to claim 1, characterized in that, Online estimation of the current true coefficient of adhesion between the wheel and the ground, including: The tire slip ratio is calculated based on the actual angular velocity of the drive wheels and the longitudinal velocity of the robot. ; A nonlinear friction observer based on slip ratio is constructed, and the true adhesion coefficient is estimated by combining the prior value of the basic friction coefficient provided by the road texture features extracted from road surface images. .

3. The adaptive anti-tilt method for wheeled legged robots based on visual-inertial fusion according to claim 1, characterized in that, The theoretical tilt angle is calculated based on the robot's motion state information. The theoretical roll angle is dynamically calibrated based on the actual adhesion coefficient between the front wheel and the ground to obtain the visual feedforward roll angle, including: Based on the curvature of the path ahead , to obtain the turning radius ,in =1 / ; Based on the robot's linear velocity v, gravitational acceleration g, and turning radius Calculate the theoretical roll angle The calculation formula is: Based on the true adhesion coefficient and / or tire slip ratio The theoretical roll angle is adjusted using a preset compensation function. Perform constraints and calibrations to output the visual feedforward tilt angle. .

4. The adaptive anti-tilt method for wheeled legged robots based on visual-inertial fusion according to claim 1, characterized in that, Calculate the expected height of the robot's virtual legs on both the left and right sides, including: According to the final roll command and robot wheelbase Calculate the height difference between the left and right virtual legs. The calculation formula is: Based on height difference The desired height of the left virtual leg is calculated based on the robot's current base height. And the expected height of the right virtual leg .

5. An adaptive anti-tilt system for wheeled legged robots based on visual-inertial fusion, characterized in that, The system for implementing the method as described in any one of claims 1 to 4 comprises: The road surface estimation module is used to estimate the true adhesion coefficient between the current wheels and the ground online based on the acquired road surface image information, robot motion state information, and actual angular velocity of the drive wheels. The feedforward calibration module is used to calculate the curvature of the path ahead based on road surface image information and calculate the theoretical roll angle, and to dynamically calibrate the theoretical roll angle based on the actual adhesion coefficient to generate a visual feedforward roll angle. The fusion decision module is used to evaluate the visual confidence of road image information and dynamically fuse the visual feedforward roll angle and IMU feedback roll angle based on the visual confidence to generate the final roll command. The kinematics mapping module is used to calculate the desired height of the virtual leg length based on the final tilt command, and to generate control commands for controlling the robot's posture adjustment mechanism through inverse kinematics calculation.

6. The adaptive anti-tilt system for wheeled legged robots based on visual-inertial fusion according to claim 5, characterized in that, Also includes: The perception module is used to acquire road surface image information, robot body motion state information, and actual angular velocity of the drive wheels.

7. The adaptive anti-tilt system for wheeled legged robots based on visual-inertial fusion according to claim 6, characterized in that, The sensing module includes: A visual perception unit is used to acquire road surface image information; An inertial measurement unit (IMU) is used to acquire motion state information of the robot body. An encoder is used to obtain the actual angular velocity of the drive wheels.

8. The adaptive anti-tilt system for wheeled legged robots based on visual-inertial fusion according to claim 7, characterized in that, Also includes: The actuator is communicatively connected to the kinematic mapping module and is used to respond to control commands to adjust the robot's posture in order to achieve active tilting of the robot.

9. A computer-readable storage medium, characterized in that, It stores computer program instructions that can be executed by a processor, and when the processor executes the computer program instructions, it can implement the steps of the method as described in any one of claims 1-4.

Images