A four-legged robot side-walking navigation method and system based on a depth camera and distance cost

Abstract

The application belongs to the technical field of robot navigation, and discloses a four-legged robot edge walking navigation method and system based on a depth camera and distance cost, which realizes edge walking navigation through a four-legged robot and a depth camera arranged on the four-legged robot and comprises the following steps: information collection, boundary identification, coordinate conversion, distance calculation, cost calculation and walking control. The application has the following advantages: environment semantic understanding capability: through deep learning semantic segmentation identification, drivable pavement and non-drivable area boundary can be identified, and the application is no longer limited to simple obstacle detection, precise perception of boundaries such as road edges, green belts, guardrails and handrails is realized, and semantic priori for autonomous edge walking is provided; asymmetric and unbalanced cost design: the traditional path planning symmetry constraint is broken, one-side close and the other-side repulsion are realized through a self-defined distance cost function, the robot is autonomously and stably maintained at a specified edge position, the trajectory is smooth and has no random deviation, and the application conforms to the human-machine coexistence traffic specification.

Description

Technical Field

[0001] This invention relates to the field of robot navigation technology, specifically a method and system for quadruped robot edge-walking navigation based on depth cameras and distance cost. Background Technology

[0002] Traditional wheeled or tracked robots typically rely on pre-planned paths to navigate structured roads, guiding them to travel in the middle of the road. However, quadruped robots, with their legged structure, have a unique advantage in walking in unstructured environments such as parks, wilderness, and open squares.

[0003] In open road environments, walking directly in the middle of the road may present the following problems: 1. The middle area is usually the main area for human activity or vehicle passage. A robot occupying the middle position will affect the passage of others and pose a collision risk; 2. There is a lack of space for emergency avoidance in the middle area. Therefore, allowing quadruped robots to "walk on the side" like humans can significantly improve their safety and social etiquette.

[0004] While existing technologies rely on lidar for obstacle avoidance and path planning, they often lack semantic understanding of road boundaries (such as curbs and green belt boundaries) and struggle to generate conscious and stable "edge-keeping" trajectories. This results in robots walking randomly on wide roads, leading to insufficient safety. Summary of the Invention

[0005] The purpose of this invention is to provide a method and system for quadruped robot sidewalk navigation based on depth camera and distance cost, so as to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A method for edge-keeping navigation of a quadruped robot based on depth camera and distance cost, which achieves edge-keeping navigation through a quadruped robot and a depth camera mounted on the quadruped robot, includes the following steps:

[0008] S1, Information Acquisition: Acquire road surface images and corresponding depth information in the direction the quadruped robot is moving using a depth camera;

[0009] S2, Boundary recognition: Based on the road surface image, identify the left and right boundary lines between the road surface and the non-driving areas on both sides;

[0010] S3, Coordinate Transformation: Combining the depth information, the left and right boundary lines are converted into 3D spatial coordinates in the robot coordinate system;

[0011] S4, Distance Calculation: Calculate the vertical distances from the current position of the quadruped robot to the left and right boundary lines, respectively, denoted as d_left and d_right;

[0012] S5, Cost Calculation: Based on the preset walking direction, construct and calculate an unbalanced distance cost function. This function assigns a lower cost to distance changes on the side closer to the preset direction and a higher cost to distance changes on the side farther from the preset direction.

[0013] S6, Walking Control: Based on the calculation results of the unbalanced distance cost, generate control commands to adjust the lateral position of the quadruped robot to achieve walking along the side.

[0014] As a further aspect of the present invention, the boundary recognition step specifically includes: processing the RGB image using a deep learning semantic segmentation model to segment the image into at least a drivable road surface area, a left non-driving area, and a right non-driving area, and extracting the boundary lines of adjacent areas as left and right boundary lines; and smoothly connecting discontinuous boundary lines using cubic spline interpolation.

[0015] As a further aspect of the present invention: the distance cost function in the cost calculation step is an asymmetric function, which ensures that when the system has the minimum cost, the position of the quadruped robot satisfies the following: the distance from one side of the preset edge direction converges to the preset expected value d_target, and the distance from the other side is greater than the safety threshold.

[0016] As a further aspect of the present invention: the distance cost function is specifically: Cost = |d_right - d_target| + Repulse(d_left), used to achieve walking on the right;

[0017] Repulse(d_left) is a monotonically decreasing function of d_left; Repulse(d_left) increases as d_left decreases.

[0018] As a further aspect of the present invention: the walking control step includes: taking the negative gradient direction of the distance cost as the desired lateral movement direction, combining it with the robot's forward speed to generate a resultant velocity vector, and driving the quadruped robot to move.

[0019] As a further aspect of the present invention, it also includes a dynamic obstacle avoidance step: detecting obstacles ahead, adding an obstacle cost term to the cost function, so that the robot can avoid dynamic obstacles while maintaining its sidewalking position, and automatically returning to the preset sidewalking state after the obstacle is passed.

[0020] As a further aspect of the present invention, it also includes a boundary validity assessment step: assessing the semantic confidence, continuity, and effective perception integrity of the left and right boundary lines; when a single or double boundary is detected to be missing or invalid, lateral control of walking along the edge is paused, and the current heading is maintained to move forward normally; Kalman filtering is used to perform short-term position prediction for occluded or missing boundaries.

[0021] As a further aspect of the present invention, it also includes a special terrain adaptation step: when the current terrain is identified as a staircase or a ramp, the boundary extraction method is redefined, and the distance calculation method and cost function form are adjusted; for staircase scenarios, the handrail and walls are used as boundaries and a vertical stability cost is introduced; for ramp scenarios, the coordinate system direction is adjusted and slope distance compensation is performed.

[0022] A quadruped robot edge-walking navigation system based on depth camera and distance cost, comprising:

[0023] Depth camera module, used to acquire RGB images and depth maps of the road surface;

[0024] A boundary recognition module is used to process the RGB image, identify and output the pixel coordinates of the left and right boundary lines;

[0025] A coordinate transformation module is used to convert the pixel coordinates into 3D spatial coordinates by combining them with the depth map;

[0026] The cost calculation module is used to calculate the distance between the robot and the boundary based on the 3D spatial coordinates, and to perform the unbalanced distance cost calculation as described in any one of claims 1-8;

[0027] The motion control module is used to generate underlying motion commands based on the output of the cost calculation module.

[0028] Compared with the prior art, the beneficial effects of the present invention are:

[0029] It possesses environmental semantic understanding capabilities: Through deep learning semantic segmentation, it identifies the boundaries between drivable road surfaces and non-drivable areas, no longer limited to simple obstacle detection, and achieves accurate perception of boundaries such as curbs, green belts, guardrails, and handrails, providing semantic priors for autonomous walking along the edge.

[0030] Asymmetric and unbalanced cost design: Breaking the symmetric constraints of traditional path planning, it achieves one-sided proximity and the other-sided repulsion through a custom distance cost function, enabling the robot to autonomously and stably maintain a designated edge position with a smooth trajectory and no random deviation, which conforms to the human-robot coexistence and passage standards.

[0031] No need to rely on high-precision maps: It completes visual perception, depth calculation and autonomous planning solely by airborne depth cameras, adapting to mapless scenarios such as unknown open roads and wilderness parks, and has a wide range of applications.

[0032] Strong dynamic obstacle avoidance and adaptive capability: It integrates obstacle cost terms to achieve temporary avoidance of dynamic obstacles such as pedestrians and vehicles. After the avoidance is completed, it automatically returns to the edge trajectory, simulating human traffic behavior and has good social adaptability.

[0033] High robustness to boundary loss: The addition of boundary validity assessment and Kalman filter prediction mechanisms addresses scenarios with boundary occlusion, interruption, and loss, preventing robot walking jitter and trajectory loss.

[0034] Universal adaptation to special terrains: It can adapt to complex terrains such as stairs and ramps. Through boundary redefinition, coordinate system adjustment, distance compensation and center of gravity collaborative control, it can achieve stable walking on the edge in all scenarios, greatly improving terrain adaptability.

[0035] Excellent real-time performance and stability: The limited effective sensing distance range reduces the amount of computation, the cost gradient closed loop corrects the lateral position in real time, the walking process is smooth and continuous, and it meets the underlying motion control requirements of quadruped robots. Attached Figure Description

[0036] Figure 1 This is a flowchart illustrating a method for quadruped robot edge-walking navigation based on depth camera and distance cost, provided in an embodiment of the present invention.

[0037] Figure 2 This is a flowchart of the boundary validity assessment and processing strategy in an embodiment of the present invention.

[0038] Figure 3 This is a schematic diagram of boundary redefinition under special terrain scenarios in an embodiment of the present invention. Detailed Implementation

[0039] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other.

[0040] The following combination Figures 1-3 The specific implementation of the present invention will be described in detail below with reference to specific embodiments.

[0041] Example 1: Basic edge walking implementation

[0042] This embodiment provides a method for edge-keeping navigation of a quadruped robot based on depth cameras and distance cost, such as... Figure 1 As shown, the method includes the following steps:

[0043] Step 101: Data collection.

[0044] The quadruped robot utilizes an Intel RealSense D435i depth camera mounted on its head to capture real-time RGB images and corresponding aligned depth maps of the surrounding environment. The camera is mounted at approximately 0.5 meters high with a 5-degree downward tilt to provide better visibility of the road surface. The robot's desired direction of travel is set along the current road, provided by the global path planning module.

[0045] Step 102: Road surface boundary line identification.

[0046] RGB images are input into a pre-trained lightweight semantic segmentation model (such as BiSeNet or Fast-SCNN). This model is trained on 5000 labeled images, with labeled categories including: road surface (drivable area), left-side obstacles / boundaries (such as green belts, curbs, walls, and guardrails), and right-side obstacles / boundaries. The model outputs a three-channel confidence map, and pixel-level classification results are obtained through argmax operations. Figure 2 As shown, the intersection of the road surface area and the non-driving area on the left is extracted as the left boundary line, and the right boundary line is extracted similarly. For cases where the boundary lines are discontinuous, cubic spline interpolation is used for smooth connection.

[0047] Step 103: Obtain the 3D coordinates of the boundary line.

[0048] For each pixel p(u, v) on the identified left and right boundary lines, based on the intrinsic parameter matrix K of the depth camera and the corresponding depth value Z, the 3D coordinates of the point in the camera coordinate system are calculated using the projection geometry relationship P = Z * K-1 * [u, v, 1]T. Then, using the robot's kinematic model (considering the robot's pitch and roll angles), these points are transformed into a robot coordinate system with the robot's center as the origin, resulting in the following... Figure 3 The left and right boundary point clouds, B_left and B_right, are shown. For pixels with missing depth values, neighborhood interpolation is used to complete the data.

[0049] Step 104: Calculate the distance between the robot and the boundary.

[0050] In the robot coordinate system, calculate the minimum vertical distance d_left from the current position (origin) to the left boundary point cloud and the minimum vertical distance d_right to the right boundary point cloud. Specifically, calculate: d_left = min(||P_left, x||), where P_left, x is the x-coordinate (lateral coordinate) of the left boundary point. d_right = min(||P_right, x||), where P_right, x is the x-coordinate of the right boundary point. In actual calculations, only boundary points within a 3-8 meter range in front of the robot are considered to reduce computational load, improve real-time performance, and avoid interference from distant noise points.

[0051] Step 105: Construct the unbalanced distance cost function.

[0052] The robot's behavior pattern is set to "walk to the right," with a desired distance of d_target = 0.5 meters from the right boundary. The cost function is constructed as follows:

[0053] Cost = w1 * |d_right - d_target| + w2 * exp(-d_left / δ)

[0054] Where w1 = 1.0, w2 = 2.0, and δ = 0.3. The first term is a linear cost that causes d_right to approach 0.5 meters; the second term is an exponential potential cost that increases sharply as the robot approaches the left boundary (d_left becomes smaller), generating a repulsive force.

[0055] Step 106: Path generation and motion control.

[0056] Based on the gradient direction of the cost function, a lateral control variable is generated. Specifically, the derivative of the cost (Cost) with respect to the robot's lateral displacement (x) is calculated to obtain the desired lateral velocity command v_x = -λ * (dCost / dx), where λ = 0.1 is the control gain. This lateral velocity command is then synthesized with the robot's original forward velocity command v_forward = 0.5 m / s to form the final low-level motion command, which is sent to the quadruped robot's joint motors. During the movement, steps 101-106 are updated in real time, forming a closed-loop control system. Ultimately, the robot will stabilize at a position approximately 0.5 meters from the right boundary and move forward, achieving safe sidewalking.

[0057] Example 2: Walking along the edge in a dynamic obstacle avoidance scenario

[0058] This embodiment adds the ability to handle dynamic obstacles (pedestrians and vehicles) based on embodiment 1.

[0059] Steps 201-204: Same as steps 101-104 in Example 1.

[0060] Step 205: Extended cost function (including obstacle cost).

[0061] Based on the original boundary cost function, obstacle detection and a cost term are added. Obstacles in front are detected simultaneously using a depth camera, and their 3D positions P_obs and radii r_obs are obtained. The extended cost function is then constructed:

[0062] Cost_total = Cost_boundary + Cost_obstacle

[0063] Where Cost_boundary is the same as in Example 1, and Cost_obstacle is defined as:

[0064] Cost_obstacle = Σ w_obs * exp(-d_obs / δ_obs),

[0065] When d_obs < d_safe, d_obs is the distance from the robot to the obstacle, d_safe = 1.0 meters is the safe distance threshold, w_obs = 3.0, and δ_obs = 0.5.

[0066] Step 206: Multi-objective optimization and local path adjustment.

[0067] When encountering oncoming pedestrians:

[0068] Under normal circumstances: When pedestrians are far away, Cost_obstacle contributes very little, and the robot keeps walking on the right.

[0069] Avoidance phase: When a pedestrian approaches to within d_obs < 1.5 meters, Cost_obstacle begins to increase significantly, generating a repulsive force. The robot will make a slight adjustment to the left (moving to a more central position on the road) to make room for the pedestrian on the right.

[0070] Recovery phase: After the pedestrian passes, Cost_obstacle disappears, boundary cost takes over again, and the robot automatically recovers to the right, returning to a right-hand walking state 0.5 meters away from the right boundary.

[0071] This mechanism simulates the human behavior pattern of "walking to the side and temporarily giving way," which is more in line with social etiquette.

[0072] Example 3: Handling Strategies for Scenarios with Missing Boundaries

[0073] This embodiment addresses situations where boundaries may be incomplete or missing in real-world environments. The processing flow is as follows.

[0074] Steps 301-303: Same as steps 101-103 in Example 1.

[0075] Step 304: Boundary validity assessment.

[0076] Before calculating the distance from the robot to the boundary, first evaluate the validity of the left and right boundaries:

[0077] Confidence assessment: Check the semantic segmentation confidence of boundary line recognition. If the average confidence is below a threshold (e.g., 0.7), the boundary is considered unreliable.

[0078] Continuity assessment: Check the continuity of the boundary line. If the missing length of the boundary line exceeds a threshold (e.g., 1.0 meter), the boundary is considered interrupted.

[0079] Distance range assessment: Check whether all boundary points are within the effective sensing range (e.g., 3-8 meters). If a side boundary is completely missing at the near end (<2 meters), then that side boundary is considered invalid.

[0080] Step 305: Adaptive cost calculation.

[0081] The cost function is dynamically adjusted based on the boundary validity assessment results.

[0082] Scenario A: Both sides of the boundary are valid. Using the standard cost function of Example 1, the robot maintains right-hand walking.

[0083] Scenario B: One-sided boundary missing (e.g., right boundary missing). In this case, the expected distance to the missing side cannot be calculated. Therefore, lateral control for walking along the edge is paused, and the robot continues to move forward normally on the current heading while continuing to attempt to identify the boundary of the missing side.

[0084] Scenario C: Both side boundaries are missing. In this situation, walking along the edge is impossible. The robot pauses lateral control, maintains its current heading and continues moving forward, while actively searching for the boundary by rotating its body.

[0085] Step 306: Memory and rollback mechanism.

[0086] When the right boundary is briefly missing (e.g., obscured by a pedestrian), a Kalman filter is used to make a short-term prediction of the boundary position, maintaining the estimated value of d_right and preventing the robot from shaking violently due to a brief loss of perception. If the boundary recovers within the prediction time, the robot seamlessly switches back to the normal edge-walking mode.

[0087] Example 4: Walking along the side of the road in special terrain – stairs and ramps

[0088] This embodiment extends the method of the present invention to staircase and ramp scenarios, and the boundary redefinition in the staircase scenario is shown below.

[0089] Step 401: Terrain recognition.

[0090] In addition to identifying boundary lines, a terrain classification module has been added to identify the current terrain type: flat road, uphill, downhill, and stairs.

[0091] Step 402: Staircase side-by-side strategy.

[0092] When identified as a staircase scene:

[0093] Boundary redefinition: The left and right boundaries of the staircase are no longer lines on the ground, but rather handrails or walls on both sides of the staircase.

[0094] 3D boundary extraction: Combine depth information to extract the 3D spatial position of the handrail as the new left and right boundaries.

[0095] Height-adaptive expected distance: The expected distance to the edge, d_target_stair, is no longer a fixed value, but is dynamically adjusted according to the width of the staircase to ensure that the robot's feet land within the safe area of ​​the stair treads.

[0096] Vertical cost introduction: Add a vertical stability cost to the cost function to prevent the robot from getting too close to the edge and falling off the edge.

[0097] Step 403: Ramp-side strategy.

[0098] When identified as a ramp scene:

[0099] Coordinate system adjustment: Redefine "lateral" in the robot coordinate system as a direction parallel to the ramp surface.

[0100] Distance calculation compensation: When calculating the distance from the robot to the two sides of the ramp, the slope factor is taken into account, and the shortest distance along the ramp surface is calculated instead of the simple horizontal projection distance.

[0101] Center of gravity control: The lateral adjustment command output by the cost function needs to work in conjunction with the center of gravity control algorithm on the ramp to ensure that the robot remains stable when adjusting its lateral position.

[0102] The preferred embodiments of the present invention have been described in detail above. However, the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of the present invention.

Claims

1. A method for edge-walking navigation of a quadruped robot based on depth camera and distance cost, characterized in that: Includes the following steps: S1, Information Acquisition: Acquire road surface images and corresponding depth information in the direction the quadruped robot is moving using a depth camera; S2, Boundary recognition: Based on the road surface image, identify the left and right boundary lines between the road surface and the non-driving areas on both sides; S3, Coordinate Transformation: Combining the depth information, the left and right boundary lines are converted into 3D spatial coordinates in the robot coordinate system; S4, Distance Calculation: Calculate the vertical distances from the current position of the quadruped robot to the left and right boundary lines, respectively, denoted as d_left and d_right; S5, Cost Calculation: Based on the preset walking direction, construct and calculate an unbalanced distance cost function. This function assigns a lower cost to distance changes on the side closer to the preset direction and a higher cost to distance changes on the side farther from the preset direction. S6, Walking Control: Based on the calculation results of the unbalanced distance cost, generate control commands to adjust the lateral position of the quadruped robot to achieve walking along the side.

2. The method for edge-walking navigation of a quadruped robot based on depth camera and distance cost according to claim 1, characterized in that, The boundary recognition step specifically includes: processing the RGB image using a deep learning semantic segmentation model to segment the image into at least a drivable road surface area, a left non-driving area, and a right non-driving area, and extracting the boundary lines of adjacent areas as left and right boundary lines; and using cubic spline interpolation to smoothly connect discontinuous boundary lines.

3. The method for edge-walking navigation of a quadruped robot based on depth camera and distance cost according to claim 1, characterized in that, The distance cost function in the cost calculation step is an asymmetric function that ensures that when the cost is minimized, the quadruped robot's position satisfies the following conditions: the distance from one side of the preset edge direction converges to the preset expected value d_target, and the distance from the other side is greater than a safety threshold.

4. The method for edge-walking navigation of a quadruped robot based on depth camera and distance cost according to claim 3, characterized in that, The distance cost function is specifically: Cost = |d_right - d_target| + Repulse(d_left), which is used to achieve right-hand walking; Repulse(d_left) is a monotonically decreasing function of d_left; Repulse(d_left) increases as d_left decreases.

5. The method for edge-walking navigation of a quadruped robot based on depth camera and distance cost according to claim 1, characterized in that, The walking control steps include: taking the negative gradient direction of the distance cost as the desired lateral movement direction, combining it with the robot's forward speed to generate a resultant velocity vector, and driving the quadruped robot to move.

6. The method for edge-walking navigation of a quadruped robot based on depth camera and distance cost according to claim 1, characterized in that, It also includes a dynamic obstacle avoidance step: detecting obstacles ahead, adding an obstacle cost term to the cost function, so that the robot can avoid dynamic obstacles while maintaining its sidewalking, and automatically returning to the preset sidewalking state after the obstacle is passed.

7. The method for edge-walking navigation of a quadruped robot based on depth camera and distance cost according to claim 1, characterized in that, It also includes a boundary validity assessment step: assessing the semantic confidence, continuity and effective perception integrity of the left and right boundary lines; when a single or double boundary is detected to be missing or invalid, lateral control of walking along the edge is suspended, and the current course is maintained to move forward normally; Kalman filtering is used to predict the short-term position of occluded or missing boundaries.

8. The method for edge-walking navigation of a quadruped robot based on depth camera and distance cost according to claim 1, characterized in that, It also includes special terrain adaptation steps: when the current terrain is identified as stairs or ramps, the boundary extraction method is redefined and the distance calculation method and cost function form are adjusted; for stair scenes, the handrails and walls are used as boundaries and vertical stability costs are introduced; for ramp scenes, the coordinate system direction is adjusted and slope distance compensation is performed.

9. A quadruped robot edge-walking navigation system based on depth camera and distance cost, characterized in that, include: Depth camera module, used to acquire RGB images and depth maps of the road surface; A boundary recognition module is used to process the RGB image, identify and output the pixel coordinates of the left and right boundary lines; A coordinate transformation module is used to convert the pixel coordinates into 3D spatial coordinates by combining them with the depth map. The cost calculation module is used to calculate the distance between the robot and the boundary based on the 3D spatial coordinates, and to perform the unbalanced distance cost calculation as described in any one of claims 1-8; The motion control module is used to generate underlying motion commands based on the output of the cost calculation module.

Images