Non-structured scene oriented foot type robot visual navigation system

By combining expansion information maps and gradient information maps in the foot robot visual navigation system, integrating ground feature parameters and optimizing trajectory planning, the problem of inaccurate navigation of foot robots in unstructured scenarios is solved, and the accuracy of navigation is improved .

CN118960741AActive Publication Date: 2024-11-15HARBIN INSTITUTE OF TECHNOLOGY SUZHOU RESEARCH INSTITUTE +1

Patent Information

Application Number
CN202411039741.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-31
Publication Date
2024-11-15
Estimated Expiration
2044-07-31

AI Technical Summary

Technical Problem

In unstructured scenarios, foot-type robots have inaccurate navigation problems due to insufficient utilization of ground feature information.

Method used

Adopt a foot-type robot vision navigation system for unstructured scenes, including a map module, planning module and state estimation module, estimate the robot position and posture through binocular visual inertial odometry, obtain RGB images and depth images, and combine multi-module local maps. Carry out trajectory planning, use expansion information maps and gradient information maps to integrate ground feature parameters, and optimize trajectory planning.

Benefits of technology

The accuracy of robot visual navigation is improved, and the ground features are described through multi-layer local perceptual maps, which solves the problem of insufficient utilization of ground feature information and improves navigation accuracy.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118960741A_ABST
    Figure CN118960741A_ABST
Patent Text Reader

Abstract

The invention discloses an unstructured scene-oriented foot type robot visual navigation system, relates to the technical field of robot navigation, and establishes a set of multi-layer local sensing map capable of describing ground features in order to solve the problem of inaccurate navigation caused by insufficient utilization of ground feature information in the prior art. Description of ground features is added on the basis of a traditional local map. And secondly, on the basis of the local perception map, the ground feature parameters are fused into the trajectory optimization process, so that the problem that the foot robot cannot fully utilize the ground feature information is solved. And the accuracy of visual navigation of the robot is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of robot navigation technology, and in particular to a legged robot visual navigation system for unstructured scenes. Background Art

[0002] Unstructured scenes, such as emergency and disaster relief sites, wilderness, and forests, have complex ground structures, such as slope, height, and inclination; complex ground features, such as stiffness, friction coefficient, and hidden dangers. Legged robots have better terrain adaptability in such scenes due to their advantages of non-continuous support, so many application scenarios of legged robots are oriented towards complex outdoor environments. Currently, the navigation and planning of legged robots are hot topics of research. The mainstream solutions include coupled navigation planning that combines center of mass planning (CoM planning) with foothold planning, and hierarchical navigation planning that decouples the two tasks. The implementation methods include optimization-based and reinforcement learning-based. Compared with the high amount of computation required for the coupled method, and the large amount of data and training time required for reinforcement learning, the hierarchical navigation solution based on optimization is more suitable for deployment on the legged robot platform. Legged robots face two main problems during movement: first, the friction between the foot and the ground is too small, which may cause sliding problems; second, due to the unknown environment stiffness information, the robot leg stiffness may not match the environment stiffness. Therefore, if only the center of mass movement is considered without considering the ground conditions, the two parts of the hierarchical navigation scheme will conflict. If the center of mass planning does not meet the kinematic constraints of the legged robot, it will also conflict. As a result, the legged robot does not fully utilize the ground feature information, which in turn leads to inaccurate robot visual navigation. Summary of the invention

[0003] The purpose of the present invention is to propose a legged robot visual navigation system for unstructured scenes to address the problem of insufficient utilization of ground feature information in the prior art, which leads to inaccurate navigation.

[0004] The technical solution adopted by the present invention to solve the above technical problems is:

[0005] A legged robot visual navigation system for unstructured scenes, the navigation system comprising a map module, a planning module, a state estimation module and a user instruction module;

[0006] The state estimation module is used to estimate the position and posture information of the robot in the world coordinate system using a binocular visual inertial odometer;

[0007] The map module is used to obtain the RGB image and depth image of the scene, and combine the position and posture information of the robot in the world coordinate system to obtain a global map and a multi-module local map;

[0008] The multi-module local map includes a dilation information map and a gradient information map;

[0009] The expansion information map is obtained by the following steps:

[0010] First, a fixed three-dimensional (x, y, z) bounded voxel space is set as the container of the local perception map. Then, the pixel position of the depth image is mapped to the voxel space according to the external parameter matrix of the camera to generate occupied voxels. Then, the sparsity of the point cloud of the robot size and the sparsity of the point cloud of the depth image are obtained, and the container of the local perception map is spatially filled according to the sparsity of the point cloud of the robot size and the sparsity of the point cloud of the depth image to obtain the expansion information map.

[0011] The gradient information layer map is obtained by the following steps:

[0012] S1: Use the real-time semantic segmentation network ENet to process the RGB image and obtain the ground type label map;

[0013] S2: Use the CSR method to sort the obtained ground type label map by cost level to obtain a ground cost level label map. The CSR method is specifically as follows:

[0014] The physical properties and safety of the ground types are empirically quantified by the Likert scale to obtain the stiffness property level, friction property level and safety level. The detectability level is established based on whether the depth camera is detectable. After that, all levels are multiplied to obtain the total level representing the reliability of the ground. Finally, the ground is ranked according to the total level representing the reliability of the ground.

[0015] S3: Convert the ground cost level label map into a gradient map through the Label2Grad method;

[0016] The user instruction module is used to obtain the user command instruction, that is, the target coordinate point, and send the user command instruction to the planning module;

[0017] The planning module is used to plan the robot's center of mass trajectory according to the global map, the multi-module local perception map and the user command instructions, and send the planning result to the robot.

[0018] Further, the planning module includes a global planning unit, a path smoothing unit and a local trajectory planning unit;

[0019] The global planning unit performs global path planning based on the target coordinate points and the global map using the A* algorithm;

[0020] The path smoothing unit is used to perform path differential flattening on the path planned by the global planning unit to obtain a local planning initial value;

[0021] The local trajectory planning unit is used to perform trajectory planning using B-spline and soft constraints, and the specific steps are as follows:

[0022] The local planning initial value is sampled to obtain the control points of the B-spline. Then the control points are used to construct a penalty function that characterizes the ground features, obstacles, and robot motion constraints. Based on the penalty function and the soft constraint method, an unconstrained optimization problem is constructed. The unconstrained optimization problem is solved using L-BFGS to obtain the optimal trajectory.

[0023] Further, the local trajectory planning unit includes a map information layer planning subunit and a motion information layer planning subunit;

[0024] The map information layer planning subunit is used to establish a penalty function using a multi-module local perception map, wherein the penalty function includes an obstacle avoidance penalty function, a centroid height penalty function, a pitch angle penalty function, and a gradient information layer penalty function;

[0025] The motion information layer planning subunit uses the relative relationship of the control points to establish a penalty function, and the motion information layer planning subunit includes a motion capability penalty function, a time allocation penalty function, and a smoothing penalty function.

[0026] Further, the motion information layer planning subunit is expressed as:

[0027] J locomotion =λ smooth f smooth +λ fitness f fitness +λ fessible f fessible

[0028] Among them, f smooth represents the smooth penalty function, λ smooth represents its penalty factor, f fitness represents the athletic ability penalty function, λ iitness represents its penalty factor, f fessible represents the time allocation penalty function, λ fessible represents its penalty factor.

[0029] Further, the map information layer planning subunit is represented as:

[0030] J map =λ collision fcollision +λ auto (f mch +f pitch )+λ auto f label

[0031] Among them, f collision represents the obstacle avoidance penalty function, f mch represents the centroid height penalty function, f pitch represents the pitch angle penalty function, f label represents the gradient information layer penalty function, λ auto represents the adaptive penalty factor, λ collision Represents the gradient information layer penalty function.

[0032] Furthermore, the process of establishing the centroid height penalty function is as follows:

[0033] Use the control point of the B-spline to find the difference between the projection point of the multi-module local perception map on the z-axis of the world coordinate system, obtain the distance between the restricted control point and the ground, and penalize the distance;

[0034] The centroid height penalty function is expressed as:

[0035]

[0036]

[0037] c m,i =H(Q i )-MCH

[0038] Among them, N C represents the total number of control points, i represents the index of the i-th control point, and f i represents the penalty function of the ith control point, c m,i Indicates the difference between the control point height H and MCH, c m,i If it is greater than the threshold, it will be punished, otherwise it will not be punished. i ) represents the height from the control point to the ground, MCH represents the height of the robot's center of mass;

[0039] Furthermore, the process of establishing the pitch angle penalty function is as follows:

[0040] First, the optimization dimension is limited to the z-axis of the world coordinate system. Then, two adjacent control points Qi and Qi+1 on the z-axis are obtained, and the difference between the control points Qi and Qi+1 on the z-axis is obtained to approximately represent the tan value of the pitch angle from Qi to Qi+1. Then, the threshold of the robot's pitch angle is set. Finally, the tan value of the pitch angle greater than the threshold is penalized, and the penalty is regularized.

[0041] Furthermore, the pitch angle penalty function is expressed as:

[0042]

[0043]

[0044]

[0045]

[0046] Among them, k represents the approximate value of the denominator of tan, the subscripts x, y, and z represent the directions of the x, y, and z axes respectively, and Pitch i represents the pitch angle at the i-th control point;

[0047] Furthermore, the establishment process of the gradient information layer penalty function is as follows:

[0048] The control points are projected onto the ground information points on the gradient information map unit, and then the cost and gradient information of the ground information points are used to establish the ground quality label constraint penalty function on the x-axis and y-axis of the world coordinate system respectively;

[0049] The ground quality label constraint penalty function is expressed as:

[0050]

[0051] Where r = x, y, f i (Q i,r ).

[0052] Furthermore, the global map is obtained by the following steps:

[0053] According to the position and posture information of the robot in the world coordinate system and the RGB image and depth image of the scene, the scene point cloud is obtained, and the point cloud is spliced ​​to obtain a global map.

[0054] The beneficial effects of the present invention are:

[0055] This application establishes a set of multi-layer local perception maps that can describe ground features, adding descriptions of ground features on the basis of traditional local maps. Secondly, on the basis of local perception maps, by incorporating ground feature parameters into the process of trajectory optimization, the problem of insufficient utilization of ground feature information by legged robots is solved. The accuracy of robot visual navigation is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 This is the overall framework diagram of this application;

[0057] Figure 2Establish a flow chart for the multi-layer local perception map of this application;

[0058] Figure 3 Schematic diagram of potential hazards considered for the multi-layer local perception map gradient information layer of this application Figure 1 ;

[0059] Figure 4 Schematic diagram of potential hazards considered for the multi-layer local perception map gradient information layer of this application Figure 2 ;

[0060] Figure 5 This is the experimental effect diagram of the multi-layer local perception map gradient information layer in the real scene of this application;

[0061] Figure 6 A schematic diagram of a penalty function for ground quality cost established based on a gradient information layer;

[0062] Figure 7 A schematic diagram of the tolerance of the trajectory to the obstacle crossing task based on the robot's passing ability is shown in Figure 1.

[0063] Figure 8 This is a schematic diagram of the pitch angle penalty;

[0064] Fig. 9 This is a schematic diagram of an unstructured scene;

[0065] Fig.10 This is a schematic diagram of the effect of the climbing task;

[0066] Fig.11 This is an illustration of the navigation task effect. Figure 1 ;

[0067] Fig.12 This is a schematic diagram of the navigation effect;

[0068] Fig.13 This is an illustration of the navigation task effect. Figure 2 ;

[0069] Fig.14 Transform schematics for conflict scenarios;

[0070] Fig.15 Schematic diagram of ground quality comparison for trajectory planning. DETAILED DESCRIPTION

[0071] It should be particularly noted that, in the absence of conflict, the various embodiments disclosed in this application can be combined with each other.

[0072] Specific implementation method 1: refer to Figure 1Specifically describing this embodiment, the visual navigation system of a legged robot for unstructured scenes described in this embodiment includes a map module, a planning module, a state estimation module and a user instruction module;

[0073] The state estimation module is used to estimate the position and posture information of the robot in the world coordinate system using a binocular visual inertial odometer;

[0074] The map module is used to obtain the RGB image and depth image of the scene, and combine the position and posture information of the robot in the world coordinate system to obtain a global map and a multi-module local map;

[0075] The multi-module local map includes a dilation information map and a gradient information map;

[0076] The expansion information map is obtained by the following steps:

[0077] First, a fixed three-dimensional (x, y, z) bounded voxel space is set as the container of the local perception map. Then, the pixel position of the depth image is mapped to the voxel space according to the camera's external parameter matrix to generate occupied voxels. Then, the sparsity of the point cloud of the robot size is obtained (in order to simplify the back-end trajectory planning problem) and the sparsity of the point cloud of the depth image are obtained. According to the sparsity of the point cloud of the robot size and the sparsity of the point cloud of the depth image, the container of the local perception map is spatially filled to obtain the expansion information map.

[0078] The gradient information layer map is obtained by the following steps:

[0079] S1: Use the real-time semantic segmentation network ENet to process the RGB image and obtain the ground type label map;

[0080] S2: Use the CSR method to sort the obtained ground type label map by cost level to obtain a ground cost level label map. The CSR method is specifically as follows:

[0081] The physical properties and safety of the ground types are empirically quantified by the Likert scale to obtain the stiffness property level, friction property level and safety level. The detectability level is established based on whether the sensor is detectable. After that, all levels are multiplied to obtain the total level representing the reliability of the ground. Finally, the ground is ranked according to the total level representing the reliability of the ground.

[0082] S3: Convert the ground cost level label map into a gradient map through the Label2Grad method.

[0083] The user instruction module is used to obtain the user command instruction, that is, the target coordinate point, and send the user command instruction to the planning module;

[0084] The planning module is used to plan the robot's center of mass trajectory according to the global map, the multi-module local perception map and the user command instructions, and send the planning result to the robot.

[0085] Legged robot hierarchical visual 3D navigation algorithm framework: Figure 1 As shown in the figure, the process framework is mainly divided into the map layer, planning layer and state planning layer. Among them, the map layer includes a global mapping module and a multi-layer local perception map module, which are responsible for updating and maintaining the global map and the local map respectively. The input is the RGB image and depth image from the depth camera and the position and posture information of the robot in the world coordinate system obtained from the state estimation module, and the output is the global map and the multi-layer local perception map; the planning layer includes a global planning module, a path softening module and a local trajectory planning module that can use the ground structure information and ground feature information. The global planning module is responsible for global path planning based on the target point and the known global map, and the path softening module is responsible for differential flattening the path to provide initial values ​​for local planning. The local trajectory planning module optimizes the trajectory based on the initial value and the multi-layer local perception map to obtain the local trajectory planning. The input is the user command instruction, the global map and the multi-layer local perception map, and the output is the trajectory planning of the center of mass of the legged robot.

[0086] A method for establishing a multi-layer local map that can describe ground features and structures: Figure 2 As shown in the figure, the perception scheme uses the RGB image and depth map of D435i as input. After the above process, a three-dimensional multi-layer local perception map is finally formed. This three-dimensional local perception map mainly includes two levels: the expansion map layer and the gradient information layer. The expansion map layer fills the space around the occupied voxels based on the occupied voxel space according to the sparsity of the point cloud information converted from the robot size and the depth map. This processing method is a common strategy for robot navigation when processing perception maps. This processing, on the one hand, makes up for the loss of local maps caused by sensors, and on the other hand, approximates the robot's planning to the center of mass planning as much as possible, reduces the complexity of the problem, and improves the safety of planning. The gradient information layer is to obtain information on whether the visual sensor in the bounded voxel space is visible, such as Figure 3 (a) shows the discrete cost information of the two-dimensional labels obtained based on real-time semantic segmentation reasoning after "cost sorting". Figure 3 As shown in (b), the state transition relationship between different voxels converted into a continuous three-dimensional state that can be used for trajectory optimization.

[0087] Gradient information layer map building method based on converting discrete states into continuous states (Label2Grad): In order to involve ground quality information in the construction of trajectory optimization problems, it is necessary to obtain the state transfer cost function between each voxel in the sampling space. To address this challenge, this application designs the Label2Grad method to convert discrete sampling maps into gradient maps with gradient directions, where the gradient direction indicates better ground quality. Figure 4 As shown, the present application transforms a single multi-state sampling space cost transfer problem into multiple dual-state sampling space transfer cost problems. Finally, based on the Euclidean distance transform, the present application can effectively model the state transfer cost between voxels in the state space, thereby providing gradient direction and cost value information for trajectory optimization.

[0088] A hierarchical trajectory optimization method that can utilize ground structure information and ground feature information: The trajectory planning problem of this application is based on B-spline and soft constraints. Since B-spline has a convex hull, that is, B-spline is within the range of its control points, and moving a control point will not affect other parts of the spline, the trajectory planning problem can be converted into a point planning problem, reducing the amount of calculation. Soft constraints combine all penalty functions into a cost function, and use penalty factors to weigh the effects of each penalty function. Therefore, this property can be used to introduce ground features, obstacles, and robot motion constraints in combination with control points to construct a penalty function, generate a cost function, and construct an unconstrained nonlinear optimization problem. In order to ensure the convergence of the solution, this application designs the trajectory optimization problem into two layers, namely the map information layer and the motion information layer. The map information layer is a cost function composed of penalty functions constructed by the joint constraints of obstacle avoidance constraints, ground feature constraints, and centroid height and crossing angle. This layer is responsible for providing the initial value of the safe trajectory. The motion information layer includes a cost function composed of motion ability constraints, smoothness constraints, and penalty functions constructed by trajectory adjustment constraints. This layer is responsible for providing adjusted trajectories and improving the differential flatness of trajectories.

[0089] A method for establishing trajectory optimization problems based on gradient information layer with available ground feature information: This method is a specific way to construct penalty functions using ground feature constraints in the ground information layer. Figure 4 As shown, the method aims to establish a penalty term for the ground quality cost based on the local perception map gradient information layer. According to the gradient direction of the state transition relationship, after optimization, the control point will converge to a position with higher ground quality, so that the trajectory is at a higher quality. Since the trajectory planning is based on the centroid trajectory, there is no ground feature information near the control point, so the control point is projected to the ground information point on the gradient layer, and the cost and gradient information of this point are used for optimization. In order to improve the convergence speed of the solution to the trajectory optimization problem, this application only optimizes on the x-axis and y-axis where the ground quality significantly affects the planning success rate.

[0090] A method for establishing a trajectory optimization problem based on the joint constraints of center of mass height and climbing angle that can determine obstacle avoidance behavior: This method is a specific way of constructing a penalty function using the joint constraints of center of mass height and climbing angle in the ground information layer. If the trajectory height is not restricted during trajectory optimization in three-dimensional space, the planned trajectory will be seriously inconsistent with the movement ability of the legged robot. In addition, although legged robots have excellent passability, they cannot cross obstacles with too large angles. Therefore, the planned trajectory must be reasonably planned for obstacle avoidance and obstacle crossing. In response to the above two problems, this application designs a center of mass height constraint to constrain the height of the trajectory relative to the ground. A pitch angle constraint is designed to constrain the trajectory's tolerance to obstacle crossing tasks based on the robot's passability. Figure 7 As shown in the figure, the combined effect of the two constraints ensures the rationality of trajectory planning and enables the trajectory to reasonably choose obstacle avoidance and obstacle crossing.

[0091] The overall framework of this application is as follows Figure 1 As shown, the framework process is mainly divided into a map layer, a planning layer, and a state planning layer. The hardware units are as follows: the camera model used by the sensor is D435i and the robot's onboard imu, the execution robot is the UnitreeGO1 footed robot, the computing unit is a 1.5kg laptop with low computing power, the graphics card unit is GTX-1050 and the computing unit is i7-8750H. This application is universal and has low requirements for hardware computing power and model reasoning capabilities, so the above hardware can be replaced. This solution is a layered navigation system, which is divided into a global planning layer and a local planning layer. The global planning layer mainly includes the global map in the map layer and the global planning of the planning layer. As the global map will continue to grow during the exploration process, it involves a large space and is responsible for the global planning of the task, so the update frequency is low, usually 5hz. The global map is obtained by combining the point cloud converted from the depth map with the current robot state estimated posture. The global planning layer uses the A* algorithm for path search. The state estimation layer uses the binocular visual inertial odometer vins-fusion to estimate the robot's posture, velocity, and acceleration information in the world coordinate system. As Figure 2As shown, the perception scheme uses the RGB image and depth map of D435i as input. After the above process, a three-dimensional multi-layer local perception map is finally formed. This three-dimensional local perception map mainly includes two levels: the expansion map layer and the gradient information layer. The depth map will be converted into an occupied voxel space that can describe three-dimensional information. First, a fixed three-dimensional (x, y, z) bounded voxel space is set as the container of the local perception map. Then, according to the camera's external parameter matrix K, the depth map pixel position (u, v, z) is mapped to the corresponding (X, Y, Z) in the voxel space, as shown in formula (1). Occupied voxels are generated, thereby obtaining the three-dimensional information within the perception range of the local perception map in the voxel space. By querying voxels, the present application can determine whether there is an object in the space, and can attach color, semantics and other information to the corresponding object.

[0092]

[0093] The RGB image will be converted into a two-dimensional ground cost label segmentation map with ground cost information. This application first inputs the RGB image into a lightweight real-time semantic segmentation network to obtain a ground type label segmentation map. The semantic segmentation network selected in this application is Enet. Finally, according to the cost sorting level judgment method, the ground type label is mapped to the corresponding Cost label to obtain a ground cost label segmentation map. This cost sorting method, referred to as CostSortRule (CSR) in this application, is a rule for subjectively evaluating the ground quality of the corresponding ground type. The current visual method can directly obtain the ground type through semantic segmentation, but cannot obtain its physical properties. And the ground problems that affect the passage of robots are not only physical information, but also potential dangers. For example, there may be shrubs and hidden traps on the grass, and there may be different depths on the water surface, the thickness of the ice surface, etc. Therefore, it is a reasonable choice to scientifically quantify people's subjective experience and integrate it into the evaluation system. In, Flamming et al. evaluated the material of the corresponding material by Likert scale, verifying the scientific nature of this method to quantify subjective experience. The next researchers used the same method to achieve semantic segmentation of ground physical property features, so the establishment of CSR will introduce subjective experience through this method. In addition, due to the limitations of the visual sensor's field of view and the presence of negative obstacles (such as potholes), some voxel spaces cannot be detected by the sensor. In this case, the detected voxels are obviously more reliable than the undetected areas. Therefore, this rule is based on the classification of ground types and covers three main dimensions: physical properties (friction properties and stiffness properties), safety, and sensor perception range. The comprehensive ranking result of these three dimensions is the final ground quality cost ranking.

[0094] Taking the laboratory's self-built ground segmentation dataset as an example, cost sorting is performed through CSR. This dataset has the following label forms. This dataset has six ground classifications: water, grass, land, ice, stone road, and cement road. Ten professionals in the laboratory were invited to conduct subjective evaluations on the impact of the friction and stiffness properties of each terrain in different states on the robot's motion. The level of friction properties is divided into 1 to 4, where 1 means the most slippery and 4 means the least slippery. The level of stiffness properties is divided into 1 to 5, where 1 means the most deformable and 5 means the least deformable. Professionals have assigned friction and stiffness level coefficients and hidden danger levels to each terrain based on subjective judgment and experience, as shown in Table 1. The higher the level, the better.

[0095] Table 1 Friction, stiffness and hidden danger level coefficient

[0096]

[0097] To distinguish the cost value of the detectable area from the undetectable area, such as Figure 3 As shown in (a), this application introduces detectability levels, as shown in Table 2.

[0098] Table 2 Detection level coefficient

[0099]

[0100] This application characterizes the ground quality by multiplying the above grade coefficients to obtain a total reliability grade coefficient. The lower the total coefficient, the lower the reliability of the voxel. Figure 3 (b). This application maps it to the cost level, and the lower the reliability, the higher the cost level, as shown in Table 3. Through the design of CSR, this application can more accurately and comprehensively evaluate the quality of different ground surfaces, which lays the foundation for 3) establishing a gradient direction gradient map that can better characterize the ground quality.

[0101] Table 3 Reliability level coefficient and cost mapping relationship

[0102]

[0103] The expansion map layer fills the space around the occupied voxels based on the robot size and the sparsity of the point cloud information converted from the depth map. This processing method is a common strategy for robot navigation when processing perception maps. This processing, on the one hand, makes up for the loss of local maps caused by sensors, and on the other hand, approximates the robot's planning to centroid planning as much as possible, reducing the complexity of the problem and improving the safety of planning.

[0104] Generally, the gradient-based trajectory planning problem cannot use feature information based on visual reasoning. The reason is that the discrete label state cannot provide a continuous state transition relationship between different voxels. To address this challenge, this method designs a gradient information layer map building method (Label2Grad) based on converting discrete states into continuous states, converting discrete sampling maps into gradient maps with gradient directions, and using gradient directions to represent better ground quality. Figure 4 As shown in (a), this method transforms a single multi-state sampling space cost transfer problem into multiple dual-state sampling space cost transfer problems. Figure 4 As shown in (c), finally, based on the Euclidean distance transform, this method can effectively model the state transfer cost between voxels in the state space, thereby providing information on the gradient direction and cost value for trajectory optimization.

[0105] Euclidean distance is the basis for establishing the state transfer cost function between different voxels in this paper. Reference [9] makes it possible to calculate the Euclidean distance in real time. This application refers to the same calculation method, which usually calculates the transfer cost of two states in the sampling space. Calculated in a certain dimension, the state transfer relationship between voxels with only two states can be expressed by the following formula. Assume that p is the voxel position of state 1, Dd(p) represents its distance from the nearest voxel position q of another state in the d dimension, and f(q) is the offset at the corresponding position, which is the calculation result of the previous dimension. Therefore, Dd(p) is the smallest parabola about (q, f(q)), as shown in formula (2).

[0106]

[0107] Therefore, for voxel p, the state transition cost of voxels in other states is the lower envelope of these parabolas. Any two parabolas have only one intersection point. The projection position s of this intersection point on the one-dimensional coordinate axis is calculated as shown in formula (3).

[0108]

[0109] According to the intersection position, determine whether the new parabola is the lowest parabola, and finally update the lower envelope of the entire dimension from left to right. The lower envelope can represent the cost function of the state transfer between voxels in different states. In this way, the state transfer relationship between different voxels in the dual-state space can be established.

[0110] According to CSR in 2), we first construct a single two-state problem. Figure 4As shown in (a), the voxels whose current voxel q is greater than their own cost value are regarded as another state, and those whose current voxel q is less than their own cost value are regarded as the same state. In this way, the voxels at the cost threshold are represented to represent the state transfer cost of the voxels with higher ground cost, that is, to establish a layer of gradient information graph that can represent the state transfer relationship of the dual-state voxel space with the purpose of "staying away from the ground with worse quality than itself". According to the threshold, multiple dual-state sampling space problems are calculated, and finally they are fused to characterize the state transfer cost between voxels in the multi-state sampling space, that is, to obtain the gradient relationship from good to bad quality between different grounds. The present application uses formula (4) for fusion to characterize the gradient direction of ground quality from good to bad. Taking into account the degree of adaptability of different robots to the ground, the present application introduces the adaptability coefficient λ. The present application can flexibly adjust the algorithm to adapt to the needs of different robots and environmental changes.

[0111] d label =λ1d1+λ2d2+…+λ n d n (4)

[0112] dlabel represents the total gradient information map, di represents the gradient information map under a certain threshold i, and n represents the total number of threshold i.

[0113] Generally, there are not too many differences in ground types in the same local perception map. If the transfer relationship between all label category voxels must be considered for each map update, it will not optimize performance and will occupy computing resources. Therefore, in the algorithm of this application, the top three cost labels are usually selected as significant cost labels by counting the number of pixels of each ground cost label in the current frame and using them as thresholds. The effect is as follows: Figure 4 As shown in (b), it can be concluded from the figure that no matter where the threshold is selected, it can be guaranteed that the voxel with the worst ground quality is regarded as the ground type that should be kept away.

[0114] The Label2Grad method uses three costs as reference gradient maps to create the effect as follows Figure 4 (c) is shown. Among them, λ1 controls the significance of the gradient map formed by the lower cost threshold, and λ2 controls the significance of the gradient map formed by the higher cost threshold. The ratio of the two coefficients is K = λ1 / λ2. It can be seen from the figure that the larger the K, the lower the tolerance of the established gradient value to the poor ground quality, and the smaller the K, the higher the tolerance, mainly highlighting the gradient with the worst ground quality. The mapping effect of this method in real scenes is as follows Figure 5 As shown, the gradient information map is a three-dimensional map, but for a more intuitive display effect, a two-dimensional slice at the ground height is displayed.

[0115] The trajectory of the present application is constructed by uniform B-spline curve. The establishment of the local planner of the present application is based on the current more common mobile robot EGO-planner, and the construction method of B-spline is similar to it. Since B-spline has the convex hull property, that is, the part of the spline curve located at (ti, ti+1) must be in the convex hull composed of control points {Qi-pb, Qi-pb+1,..., Qi}, where pb is the order. Using this property, the present application transforms the various constraint problems that need to be considered in the trajectory optimization of this article into problems constructed with control points.

[0116] Different from other local planners built based on optimization methods, which focus the planning task on obstacle avoidance, the local planner of this application needs to consider the constraints from ground features and ground structures in trajectory optimization. There are also constraints from the robot's motion ability and the time adjustment constraints of B-spline. Therefore, it is a huge trajectory optimization problem. Therefore, this application chooses a soft constraint method to establish the trajectory optimization problem. However, it is still a challenge to ensure the speed and convergence of the solution to this problem. Therefore, this application adopts a step-by-step optimization method to construct a planner, such as Figure 1 shown.

[0117] The target cost function optimized in the first step is shown in formula (5), which is called the perception layer trajectory planning cost function in this application. This part mainly uses map information and has the following three goals: obstacle avoidance, determining obstacle crossing or obstacle avoidance based on the map and its own capabilities, and selecting a trajectory with higher ground reliability as much as possible on the basis of the above.

[0118] J map =λ collision f collision +λ a (f mch +f pitch )+λ auto f label (5)

[0119] where f collision represents the penalty function established based on the distance from the robot to the obstacle, with the goal of achieving obstacle avoidance. mch and f pitch They represent the penalty functions based on the center of mass height constraint and the climbing ability constraint, respectively, to realize the robot’s choice of obstacle climbing or obstacle avoidance. The specific construction method is shown in (5). labelIt represents a penalty function established according to the ground quality, which enables the robot to choose a more reliable passing trajectory according to the ground conditions. The specific construction method is shown in (6). Each penalty function has a corresponding penalty factor λ to control the importance of this factor to the trajectory decision. However, in reality, there may be scenarios where the structure and features conflict, resulting in the non-convexity of the problem. In the experimental part of (7), this application designs such a scenario. In order to ensure that the solution meets the real-time solution, the penalty factor of the ground feature penalty function is designed in this application as an adaptive penalty factor λ auto The specific construction method is shown in (6).

[0120] The target cost function of the second step optimization is shown in formula (6), which is called motion layer planning in this application. This part mainly considers the robot's motion capabilities such as speed and acceleration, the time adjustment of the uniform B-spline, and the improved differential flatness of the planned trajectory.

[0121] J locomotion =λ smooth f smooth +λ fitness f fitness +λ fessible f fessible (6)

[0122] This application establishes a penalty function for ground quality cost based on the gradient information layer. Since the trajectory planning is the center of mass trajectory, there is no ground information near the control point. Figure 6 As shown, the control point is projected to the ground information point on the gradient layer, and the cost and gradient information of this point are used for optimization. The ground quality label constraint penalty function of a single control point is shown in formula (7).

[0123]

[0124] In order to improve the convergence speed of the optimization problem, this application only optimizes on the x-axis and y-axis where the ground quality significantly affects the planning success rate. Therefore, the ground quality label constraint penalty function of trajectory planning is shown in formula (8).

[0125]

[0126] In fact, in scenarios where there is a conflict between ground structure and ground features, such as Figure 4 There are scenes with obstacles, which may lead to non-convergence of the solution. Therefore, this application designs an adaptive relaxation penalty factor to automatically adjust the degree of influence of the penalty function according to the planning situation. Since the obstacle avoidance task has a greater impact on the success rate of planning than the impact of ground quality, if the algorithm has too many solutions in the planning and there are other costs such as obstacle avoidance, the penalty function coefficient is reduced to ensure the real-time planning.

[0127] The present application implements the robot's task planning of climbing over or avoiding obstacles by jointly constraining the center of mass and the climbing angle of the planned trajectory. The two-dimensional navigation system can only avoid obstacles, while the three-dimensional navigation system needs to distinguish between avoidance and climbing in unstructured environments. Although legged robots have excellent passability, they cannot cross obstacles with angles that are too high. Therefore, the planned trajectory must be reasonably planned for obstacle avoidance and obstacle crossing. In addition, if the trajectory height is not restricted during trajectory optimization in three-dimensional space, the planned trajectory will be seriously inconsistent with the movement ability of the legged robot. In response to the above two problems, the present application designs a center of mass height constraint to constrain the height of the trajectory relative to the ground. Since the pitch angle can best represent the climbing task, a pitch angle constraint is designed to constrain the trajectory's tolerance for obstacle crossing tasks based on the robot's passability. For example Figure 7 As shown in the figure, the combined effect of the two constraints ensures the rationality of trajectory planning and enables the trajectory to reasonably choose obstacle avoidance and obstacle crossing.

[0128] The trajectory planned by the local planner in this application is the trajectory of the center of mass of the legged robot. Therefore, the penalty is set by limiting the distance of the control point to the ground. This distance is obtained by subtracting the projection point of the control point on the map on the z axis. Then, this distance is subtracted from the center of mass height MCH to obtain c m,i , as shown in formula (10), and then quadratized to obtain a single control point penalty function, as shown in formula (9).

[0129]

[0130] c m,i =H(Q i )-MCH(10)

[0131] In this way, the control point is constrained near the centroid height. The rationality of the trajectory height is guaranteed by the convexity of the B-spline. Therefore, the trajectory planning centroid height penalty function is shown in formula (11).

[0132]

[0133] Based on the planning of the overrun capability, this application focuses on the penalty of the pitch angle. If the pitch angle in trajectory planning is directly calculated, the complexity of the equation of the problem will increase, so this application simplifies this problem. Figure 8 As shown. This application first limits the optimization dimension to the z-axis which has the greatest impact on the pitch angle. As shown in formula (12), the difference between the z-axis of two adjacent control points Qi and Qi+1 is used to approximate the tan value of the pitch angle from Qi to Qi+1. Although the positions of Qi and Qi+1 change during the optimization process, since the distance between the control points of the uniform B-spline is relatively stable, k is generally a relatively stable value. Figure 7 In the experiment, it will not affect the convergence of the solution.

[0134]

[0135]

[0136] Through the above method, the present application obtains an approximate expression of the pitch angle of Qi. According to the robot's motion capability, a threshold of the pitch angle is set. It is regularized by squaring to ensure that the pitch or elevation in planning will not exceed the robot's motion capability. In view of the convex hull of the B-spline, it can be ensured that the trajectory planning between the control points meets the constraints on the pitch angle. Therefore, the penalty function of the pitch angle of the control point is shown in formula (14).

[0137]

[0138] Therefore, the penalty function of the trajectory pitch angle of the present application is shown in formula (15):

[0139]

[0140] Experimental results:

[0141] The experiments mainly include mapping effect experiments and navigation effect ablation experiments under different ground structures and ground features. This application has designed four scene verification results, namely, scene 1: unstructured scene, such as Fig. 9 As shown in the figure, the scenario includes a ramp that meets the needs of the robot and a large angle drop that can create a sensor blind spot; Scenario 2: Obstacle scenario, such as Fig.12 As shown in the figure, the obstacle in this scene forms a large angle with the ground, and the height of the obstacle is too high for the robot to cross; Scene 3: Multiple ground feature scenes, such as Figure 5 As shown in the figure, there are grass and wooden pavement in this scene, and the sensor blind area can verify the effect of establishing a multi-layer local perception map; Scenario 4: Conflict between ground features and ground structures, that is, there are obstacles in areas with high ground quality. These four scenes are representative and are extreme scenes of conventional optimization-based hierarchical navigation. This verifies the superiority of this method.

[0142] The mapping effect experiment was carried out in scenes 1 and 2. Scene 1 is an unstructured scene, which can test the effect of local mapping on structural features. Scenes 1 and 2 are mainly composed of wooden boards and grass. Combined with the sensor-invisible area, there are at least three advantageous terrain features, which are sufficient to verify this application.

[0143] Table 4 shows the attribute information of each layer of the multi-layer local map and the effective range of the visual sensor. Each layer of the multi-layer local map of the present application is a three-dimensional map, and the size of the corresponding world coordinate is consistent, which is determined by the effective range of the sensor. The resolution is consistent. The indexing method of storing voxel information is consistent.

[0144] Table 4 Attribute information of each layer of multi-layer local map

[0145]

[0146] Table 5 shows the average time consumption in each part of the map construction process. The speed of building the entire multi-layer local map can ensure that the update speed is consistent with the camera frequency. During the construction process, the main time consumption is the reasoning time of the real-time semantic segmentation network for ground types, which accounts for 80% of the total time, while the Label2Grad of this method is 8%. From the time analysis, it can be seen that the local planning of this application has real-time requirements.

[0147] Table 5 Average time consumption in each part of the mapping process

[0148]

[0149] Figure 5 and Fig. 9 The mapping effect of multi-layer local perception maps can be demonstrated. In the expansion map layer, the expansion is performed according to the size of the robot. In addition, a planning lower limit is set. This design effectively ensures the safety of centroid planning. In the gradient information layer, through the CSR rule, this application sets the ground reliability ranking as wooden boards>grass=ground>invisible scenes. By comparing the three-dimensional semantic segmentation maps of scenes 1 and 2 with the trapezoidal information map slices, it can be seen that the Label2Grad of this method can effectively characterize the voxel state transfer relationship between different ground features and the voxel state transfer relationship between the same ground.

[0150] Navigation effect experiment The core goal of the navigation framework of this method is to design a set of visual navigation methods for legged robots for unstructured scenes, so that the legged robot can fully perceive the ground structure and feature information and use it for trajectory optimization. Therefore, the verification is mainly aimed at the penalty function of the ground structure information that can be used for the joint constraints of the centroid height and the crossing angle, the effectiveness of the trajectory optimization penalty function that can be used for the ground feature information based on the gradient information layer, and the conflict between the ground features and the ground structure. The verification method mainly adopts the method of ablation experiment and comparative experiment. In the verification experiment, the centroid height penalty function is defined as "M", the crossing angle penalty function is defined as "P", the obstacle distance penalty function is defined as "C", and the ground feature penalty function is defined as "L".

[0151] First, we verify the trajectory optimization effect of the combined constraints of the center of mass height and the crossing angle that can utilize the ground structure information. The verified scenarios are scenario 1 and scenario 2. Fig. 9 As shown, in scenario 1, this application sets two target points. Target point 1 corresponds to the climbing task, and target point 2 is in the sensor blind spot, so it corresponds to the avoidance task. The map layer cost functions are "M+P+C", "M+C", "P+C" and "C" to perform the above tasks. In fact, the effect of "C" is similar to the current common three-dimensional navigation planning method. Fig.10 From the robot operation results, when the map layer cost function is "M+P+C", the robot can correctly climb over or avoid according to the target points 1 and 2. For target point 1, the trajectory planning results of "M+C", "P+C" and "C" can make the robot reach the target successfully. However, for target point 2, the trajectory planning results of "M+C", "P+C" and "C" still choose to climb over. Fig.12 As shown in Figure 2, scenario 2 is a typical obstacle avoidance scenario. “M+P+C” can successfully complete the navigation task, while “M+C”, “P+C” and “C” completely fail. Fig.11 , Fig.13 As shown in the figure, it can be analyzed from the data that the planning result of "M+P+C" fully meets the robot's center of mass height and climbing ability threshold, while "M+C", "P+C" and "C" cannot meet them. This will lead to conflicts between the planning layer and the control layer, causing navigation failure in some scenarios. Analysis of the planning results of "M+C" and "P+C" shows that the penalty function designs of "M" and "P" do ensure the center of mass height and climbing angle respectively, but both will lead to a one-sided understanding of the environment structure. Only joint constraints can ensure the rationality of the solution.

[0152] Next is to verify the trajectory planning effect based on the available ground feature information of the gradient information layer. The verified scene is scene 3. Scene 3 is a scene that is as flat as possible but has different ground features. The design purpose is to not allow structural information to affect navigation planning. This application sets 1 target point, so that the map layer cost function is "M+P+C+L" and "M+P+C" to complete the above tasks. Fig.14 It can be seen from the robot operation results that when the cost function of the "M+P+C" map layer is "M+P+C+L", the robot chooses a solution with higher ground quality when completing the navigation task. Fig.15 As shown in the figure, it can be analyzed from the data that the ground quality of the planning result of "M+P+C+L" is higher than that of "M+P+C". This proves the effectiveness of the planning of "L".

[0153] Finally, the scene of the conflict between ground features and ground structures is shown in the following figure. The scene used is scene 4, which is modified based on scene 3. Fig.14As shown in Figure 1, obstacles are added at the best position of the ground feature gradient information map to create a conflict scenario. In a conflict scenario, the local planning solver will not converge, but under the effect of the adaptive penalty factor, the influence weight of the ground feature will decrease, allowing the robot to give priority to obstacle information. Fig.12 As shown, let the robot perform the same target point task in scene 3 and scene 4. If there is no conflict, this application will plan the trajectory with the best ground quality. If there is a conflict, the ground structure will be given priority.

[0154] A hierarchical visual three-dimensional navigation method for legged robots in unstructured scenes is proposed. By establishing a multi-layer local map that can describe ground features and ground structures, this method not only has the expression of structural information, but also supplements the use of visual information that is lacking in conventional methods, and converts discrete label information into continuous state transition information required for trajectory planning methods based on gradient optimization. This method uses a hierarchical trajectory optimization method that can utilize ground structure information and ground feature information. Through multi-layer local perception maps, the structure information and feature information are combined with the robot's own properties to construct a penalty function, and the factors that affect the navigation success rate are organically constructed as a trajectory planning problem through soft constraints. In order to ensure the convergence of the solution, step-by-step optimization is performed to ensure the planning safety first and then the planning smoothness, so that the robot can choose a trajectory with better ground quality as much as possible while ensuring that it meets its own motion capabilities. Finally, based on the above two parts, a full-stack hierarchical legged robot visual navigation method that can achieve global map establishment and global trajectory planning is constructed. The results of ablation experiments and qualitative experiments show that the results generated by this method have better task completion effects.

[0155] It should be noted that the specific implementation is only an explanation and description of the technical solution of the present invention, and cannot be used to limit the scope of protection of the rights. Any partial changes made according to the claims and description of the present invention should still fall within the scope of protection of the present invention.

Claims

1. A visual navigation system for legged robots in unstructured scenes, characterized by The navigation system includes a map module, a planning module, a state estimation module and a user instruction module; The state estimation module is used to estimate the position and posture information of the robot in the world coordinate system using a binocular visual inertial odometer; The map module is used to obtain the RGB image and depth image of the scene, and combine the position and posture information of the robot in the world coordinate system to obtain a global map and a multi-module local map; The multi-module local map includes a dilation information map and a gradient information map; The expansion information map is obtained by the following steps: First, a fixed three-dimensional (x, y, z) bounded voxel space is set as the container of the local perception map. Then, the pixel position of the depth image is mapped to the voxel space according to the external parameter matrix of the camera to generate occupied voxels. Then, the sparsity of the point cloud of the robot size and the sparsity of the point cloud of the depth image are obtained, and the container of the local perception map is spatially filled according to the sparsity of the point cloud of the robot size and the sparsity of the point cloud of the depth image to obtain the expansion information map. The gradient information layer map is obtained by the following steps: S1: Use the real-time semantic segmentation network ENet to process the RGB image and obtain the ground type label map; S2: Use the CSR method to sort the obtained ground type label map by cost level to obtain a ground cost level label map. The CSR method is specifically as follows: The physical properties and safety of the ground types are empirically quantified by the Likert scale to obtain the stiffness property level, friction property level and safety level. The detectability level is established based on whether the depth camera is detectable. After that, all levels are multiplied to obtain the total level representing the reliability of the ground. Finally, the ground is ranked according to the total level representing the reliability of the ground. S3: Convert the ground cost level label map into a gradient map through the Label2Grad method; The user instruction module is used to obtain the user command instruction, that is, the target coordinate point, and send the user command instruction to the planning module; The planning module is used to plan the robot's center of mass trajectory according to the global map, the multi-module local perception map and the user command instructions, and send the planning result to the robot.

2. The visual navigation method for a legged robot in an unstructured scene according to claim 1 is characterized in that The planning module includes a global planning unit, a path smoothing unit and a local trajectory planning unit; The global planning unit performs global path planning based on the target coordinate points and the global map using the A* algorithm; The path smoothing unit is used to perform path differential flattening on the path planned by the global planning unit to obtain a local planning initial value; The local trajectory planning unit is used to perform trajectory planning using B-spline and soft constraints, and the specific steps are as follows: The local planning initial value is sampled to obtain the control points of the B-spline. Then the control points are used to construct a penalty function that characterizes the ground features, obstacles, and robot motion constraints. Based on the penalty function and the soft constraint method, an unconstrained optimization problem is constructed. The unconstrained optimization problem is solved using L-BFGS to obtain the optimal trajectory.

3. The legged robot visual navigation method for unstructured scenes according to claim 2 is characterized in that The local trajectory planning unit includes a map information layer planning subunit and a motion information layer planning subunit; The map information layer planning subunit is used to establish a penalty function using a multi-module local perception map, wherein the penalty function includes an obstacle avoidance penalty function, a centroid height penalty function, a pitch angle penalty function, and a gradient information layer penalty function; The motion information layer planning subunit uses the relative relationship of the control points to establish a penalty function, and the motion information layer planning subunit includes a motion capability penalty function, a time allocation penalty function, and a smoothing penalty function.

4. The visual navigation method for a legged robot in an unstructured scene according to claim 3 is characterized in that The motion information layer planning subunit is represented as: J locomotion =λ smooth f smooth +λ fitness f fitness +λ fessible f fessible Among them, f smooth represents the smooth penalty function, λ smooth represents its penalty factor, f fitness represents the athletic ability penalty function, λ fitness represents its penalty factor, f fessible represents the time allocation penalty function, λ fessible represents its penalty factor.

5. The visual navigation method for a legged robot in an unstructured scene according to claim 4 is characterized in that The map information layer planning subunit is represented as: J map =λ collision f collision +λ auto (f mch +f pitch )+λ auto f label Among them, f collision represents the obstacle avoidance penalty function, f mch represents the centroid height penalty function, f pitch represents the pitch angle penalty function, f label represents the gradient information layer penalty function, λ auto represents the adaptive penalty factor, λ collision Represents the gradient information layer penalty function.

6. The legged robot visual navigation method for unstructured scenes according to claim 3 is characterized in that The establishment process of the centroid height penalty function is: Use the control point of the B-spline to find the difference between the projection point of the multi-module local perception map on the z-axis of the world coordinate system, obtain the distance between the restricted control point and the ground, and penalize the distance; The centroid height penalty function is expressed as: c m,i =H(Q i )-MCH Among them, N C represents the total number of control points, i represents the index of the i-th control point, and f i represents the penalty function of the ith control point, c m,i Indicates the difference between the control point height H and MCH, c m,i If it is greater than the threshold, it will be punished, otherwise it will not be punished. i ) represents the height from the control point to the ground, and MCH represents the height of the robot's center of mass.

7. The legged robot visual navigation method for unstructured scenes according to claim 6 is characterized in that The process of establishing the pitch angle penalty function is as follows: First, the optimization dimension is limited to the z-axis of the world coordinate system. Then, two adjacent control points Qi and Qi+1 on the z-axis are obtained, and the difference between the control points Qi and Qi+1 on the z-axis is obtained to approximately represent the tan value of the pitch angle from Qi to Qi+1. Then, the threshold of the robot's pitch angle is set. Finally, the tan value of the pitch angle greater than the threshold is penalized, and the penalty is regularized.

8. The legged robot visual navigation method for unstructured scenes according to claim 7 is characterized in that The pitch angle penalty function is expressed as: Among them, k represents the approximate value of the denominator of tan, and the subscript x, V , z represent x, V , z-axis direction, Pitch i represents the pitch angle at the i-th control point.

9. The visual navigation method for a legged robot in an unstructured scene according to claim 8, characterized in that The establishment process of the gradient information layer penalty function is: The control points are projected onto the ground information points on the gradient information map unit, and then the cost and gradient information of the ground information points are used to establish the ground quality label constraint penalty function on the x-axis and y-axis of the world coordinate system respectively; The ground quality label constraint penalty function is expressed as: Where r = x, y, f i (Q i,r ).

10. The legged robot visual navigation method for unstructured scenes according to claim 1, characterized in that The global map is obtained by the following steps: According to the position and posture information of the robot in the world coordinate system and the RGB image and depth image of the scene, the scene point cloud is obtained, and the point cloud is spliced ​​to obtain a global map.

Citation Information

Patent Citations

  • Construction method for multi-level hexagonal grid map

    CN108387240A

  • Path planning method of foot type robot, electronic equipment and readable storage medium

    CN114564027A

  • Mobile machine and unmanned driving perception mapping system for unstructured scene of mobile machine

    CN116168075A

  • Pedestrian indoor positioning method based on neural network assistance

    CN116772865A

  • Mine vehicle autonomous navigation method based on passable map

    CN116929363A

Cited By

  • A three-dimensional gaussian spatter scene model machine dog data acquisition method and system

    CN122780292A

  • A method and system for acquiring data from a 3D Gaussian splash scene model for a robotic dog

    CN122780292B