Three-dimensional real-time navigation method, device and electronic equipment for robot

Abstract

The application is suitable for the field of artificial intelligence technology, and provides a three-dimensional real-time navigation method and device of a robot and an electronic device, the method comprising: determining, according to real-time running data of the robot, a local update range of Octree-based Occupancy Grid Mapping (Octomap) information corresponding to a running scene of the robot, and an observation occupancy probability and a decay coefficient corresponding to each three-dimensional grid to be updated in the local update range; updating the occupancy probability of each three-dimensional grid to be updated according to the observation occupancy probability and the decay coefficient corresponding to each three-dimensional grid to be updated; and determining a real-time navigation path of the robot in the local update range according to the updated running scene Octomap information and an A* search algorithm. Thus, the reliability, safety and calculation efficiency of three-dimensional space navigation of the robot are improved.

Description

Technical Field

[0001] This application belongs to the field of artificial intelligence technology, and in particular relates to a three-dimensional real-time navigation method, device and electronic device for robots. Background Technology

[0002] In the field of robot three-dimensional spatial navigation, rapid point-to-point path planning in three-dimensional space is a key navigation technology for flying robots or off-road robots in complex terrain. Real-time path planning needs to avoid static and dynamic obstacles in space, and also needs to consider real-time performance, shortest path and other characteristics.

[0003] In related technologies, most robots rely on 2D maps or 2.5D cost maps (with height information) for path planning. However, because 2D maps lack height information, and 2.5D cost maps require compressing 3D point clouds onto a 2D plane (maximum height projection), 3D spatial information such as suspended obstacles (e.g., skybridges) and multi-layered structures (shelves) cannot be fully represented, resulting in the loss of more than 30% of spatial information. Therefore, due to the lack of 3D spatial information in 2D maps or 2.5D cost maps, they cannot be adapted for 3D spatial navigation, leading to lower path rationality and safety in path planning. Summary of the Invention

[0004] This application provides a method, device, electronic device, storage medium, and computer program product for real-time three-dimensional navigation of robots. It can solve the problem that due to the lack of three-dimensional spatial information in two-dimensional maps or 2.5-dimensional cost maps, they cannot be adapted to three-dimensional spatial navigation, and the path planning of robots based on them has low rationality and safety.

[0005] In a first aspect, embodiments of this application provide a method for real-time 3D navigation of a robot, comprising: acquiring current 3D point cloud data collected by a lidar in the robot, current depth images collected by a depth camera, and the current linear velocity of the robot; determining the local update range of octree-based occupancy grid mapping (Octomap) information of the robot's operating scene based on the current linear velocity and the current depth image, wherein the operating scene Octomap information contains information of multiple 3D grids; determining the observation occupancy probability and the attenuation coefficient of each 3D grid to be updated based on the local update range, the current 3D point cloud data, and the current depth image, wherein each 3D grid to be updated is located within the local update range; updating the occupancy probability of each 3D grid to be updated based on the observation occupancy probability and the attenuation coefficient of each 3D grid to be updated to generate updated operating scene Octomap information; and determining the real-time navigation path of the robot within the local update range based on the updated operating scene Octomap information and the A* search algorithm.

[0006] In one possible implementation of the first aspect, the determination of the local update range of the Octomap information corresponding to the robot's operating scene based on the current linear velocity and the current depth image includes:

[0007] Obtain the robot's minimum update radius, dynamic obstacle prediction time window, and collision avoidance safety buffer radius;

[0008] The base radius corresponding to the local update range is determined based on the robot's minimum update radius, dynamic obstacle prediction time window, collision avoidance safety buffer radius, and current linear velocity.

[0009] Based on the base radius and the current depth image, determine the types of obstacles included in the current depth image within the base radius;

[0010] The semantic radius corresponding to the local update range is determined based on the types of obstacles contained within the base radius of the current depth image;

[0011] The local update range is determined based on the base radius and semantic radius.

[0012] In some embodiments, in another possible implementation of the first aspect, determining the observation occupancy probability and the attenuation coefficient corresponding to each 3D raster to be updated based on the local update range, the current 3D point cloud data, and the current depth image includes:

[0013] Voxelize the current 3D point cloud data within the local update range to determine each 3D raster to be updated;

[0014] Determine the initial occupancy probability and observation occupancy probability for each 3D grid to be updated;

[0015] Target detection is performed on the current depth image within the local update range to determine the obstacle type and confidence level corresponding to each 3D raster to be updated;

[0016] Based on the obstacle type and confidence level of each 3D grid to be updated, the attenuation coefficient of each 3D grid to be updated is determined, and the initial occupancy probability of each 3D grid to be updated is corrected.

[0017] In some embodiments, in another possible implementation of the first aspect, determining the attenuation coefficient corresponding to each 3D grid to be updated based on the obstacle type and confidence level of each 3D grid to be updated, and correcting the initial occupancy probability corresponding to each 3D grid to be updated, includes:

[0018] If the obstacle type corresponding to the first three-dimensional grid to be updated is a dynamic obstacle and the confidence level is greater than the confidence level threshold, the initial occupancy probability corresponding to the first three-dimensional grid to be updated is corrected to the first occupancy probability value, and the attenuation coefficient corresponding to the first three-dimensional grid to be updated is determined as the first attenuation value, wherein the first three-dimensional grid to be updated is any three-dimensional grid to be updated.

[0019] When the obstacle type corresponding to the first 3D grid to be updated is a transparent obstacle, the initial occupancy probability corresponding to the first 3D grid to be updated is corrected to the second occupancy probability value, and the attenuation coefficient corresponding to the first 3D grid to be updated is determined as the second attenuation value, wherein the second occupancy probability value is less than the first occupancy probability value, and the second attenuation value is less than the first attenuation value.

[0020] In some embodiments, in another possible implementation of the first aspect, the updated octomap information of the operating scene includes the current occupancy probability of each 3D grid after the update; correspondingly, determining the real-time navigation path of the robot within the local update range based on the updated octomap information of the operating scene and the A* search algorithm includes:

[0021] Based on the current occupied probability and idle probability threshold of each 3D grid, determine each obstacle node and each non-obstacle node within the local update range;

[0022] Based on the robot's historical navigation path, obstacle nodes, and non-obstacle nodes, determine the robot's starting node and target node within the local update range;

[0023] Based on the A* search algorithm and a preset cost function, each non-obstacle node is searched to determine the real-time navigation path between the starting node and the target node. The preset cost function is determined based on the actual cumulative cost function, heuristic function, slope cost function, and rolling cost function.

[0024] In some embodiments, in yet another possible implementation of the first aspect, the starting node is the last unblocked node of the historical navigation path within the scope of the local update.

[0025] In some embodiments, in another possible implementation of the first aspect, when searching for the starting node using the A* search algorithm, the actual cumulative cost of the starting node is the actual cumulative cost of the last unblocked node.

[0026] In some embodiments, in another possible implementation of the first aspect, when searching for the effective neighboring nodes corresponding to the current node in the closed list according to the A* search algorithm and a preset cost function, the slope cost of the effective neighboring nodes is determined in the following manner:

[0027] Determine the height difference and horizontal distance between the current node and its effective neighboring nodes. The effective neighboring nodes are non-obstacle nodes that are adjacent to the current node and are not in the closed list.

[0028] The slope cost of effective neighboring nodes is determined based on the height difference, horizontal distance, and slope cost function.

[0029] In some embodiments, in yet another possible implementation of the first aspect, when searching for the effective neighboring nodes corresponding to the current node in the closed list according to the A* search algorithm and a preset cost function, the rolling cost of the effective neighboring nodes is determined in the following manner:

[0030] Determine the change in heading angle of the robot from the current node to the effective neighboring nodes, where the effective neighboring nodes are non-obstacle nodes that are adjacent to the current node and are not in the closed list;

[0031] The rolling cost of effective neighboring nodes is determined based on the change in heading angle and the rolling cost function.

[0032] Secondly, embodiments of this application provide a three-dimensional real-time navigation device for a robot, comprising: a first acquisition module, configured to acquire current three-dimensional point cloud data collected by a lidar in the robot, current depth image collected by a depth camera, and current linear velocity of the robot; a first determination module, configured to determine the local update range of the octomap information of the robot's operating scene based on the current linear velocity and the current depth image, wherein the octomap information of the operating scene includes information of multiple three-dimensional grids; a second determination module, configured to determine the observation occupancy probability and the attenuation coefficient of each three-dimensional grid to be updated based on the local update range, the current three-dimensional point cloud data, and the current depth image, wherein each three-dimensional grid to be updated is located within the local update range; a first update module, configured to update the occupancy probability of each three-dimensional grid to be updated based on the observation occupancy probability and the attenuation coefficient of each three-dimensional grid to be updated, to generate updated operating scene octomap information; and a third determination module, configured to determine the real-time navigation path of the robot within the local update range based on the updated operating scene octomap information and the A* search algorithm.

[0033] In one possible implementation of the second aspect, the first determining module includes:

[0034] The first acquisition unit is used to acquire the robot's minimum update radius, dynamic obstacle prediction time window, and anti-collision safety buffer radius.

[0035] The first determining unit is used to determine the base radius corresponding to the local update range based on the robot's minimum update radius, dynamic obstacle prediction time window, anti-collision safety buffer radius, and current linear velocity.

[0036] The second determining unit is used to determine the type of obstacle contained in the current depth image within the base radius based on the base radius and the current depth image;

[0037] The third determining unit is used to determine the semantic radius corresponding to the local update range based on the types of obstacles contained in the current depth image within the base radius.

[0038] The fourth determining unit is used to determine the local update range based on the base radius and semantic radius.

[0039] In some embodiments, in another possible implementation of the second aspect, the second determining module includes:

[0040] The fifth determining unit is used to voxelize the current 3D point cloud data within the local update range in order to determine each 3D raster to be updated.

[0041] The sixth determining unit is used to determine the initial occupancy probability and the observation occupancy probability for each 3D grid to be updated;

[0042] The seventh determining unit is used to perform target detection on the current depth image within the local update range to determine the obstacle type and confidence level corresponding to each 3D raster to be updated;

[0043] The eighth determining unit is used to determine the attenuation coefficient corresponding to each three-dimensional grid to be updated based on the obstacle type and confidence level of each grid to be updated, and to correct the initial occupancy probability of each grid to be updated.

[0044] In some embodiments, in yet another possible implementation of the second aspect, the eighth determining unit is specifically used for:

[0045] If the obstacle type corresponding to the first three-dimensional grid to be updated is a dynamic obstacle and the confidence level is greater than the confidence level threshold, the initial occupancy probability corresponding to the first three-dimensional grid to be updated is corrected to the first occupancy probability value, and the attenuation coefficient corresponding to the first three-dimensional grid to be updated is determined as the first attenuation value, wherein the first three-dimensional grid to be updated is any three-dimensional grid to be updated.

[0046] When the obstacle type corresponding to the first 3D grid to be updated is a transparent obstacle, the initial occupancy probability corresponding to the first 3D grid to be updated is corrected to the second occupancy probability value, and the attenuation coefficient corresponding to the first 3D grid to be updated is determined as the second attenuation value, wherein the second occupancy probability value is less than the first occupancy probability value, and the second attenuation value is less than the first attenuation value.

[0047] In some embodiments, in yet another possible implementation of the second aspect, the updated runtime scene Octomap information includes the updated current occupancy probability of each 3D grid; correspondingly, the third determining module includes:

[0048] The ninth determining unit is used to determine each obstacle node and each non-obstacle node within the local update range based on the current occupied probability and idle probability threshold of each three-dimensional grid.

[0049] The tenth determining unit is used to determine the starting node and target node of the robot within the local update range based on the robot's corresponding historical navigation path, each obstacle node and each non-obstacle node;

[0050] The eleventh determining unit is used to search for each non-obstacle node according to the A* search algorithm and a preset cost function in order to determine the real-time navigation path between the starting node and the target node. The preset cost function is determined based on the actual cumulative cost function, heuristic function, slope cost function and rolling cost function.

[0051] In some embodiments, in yet another possible implementation of the second aspect, the starting node is the last unblocked node of the historical navigation path within the scope of the local update.

[0052] In some embodiments, in another possible implementation of the second aspect, when searching for the starting node using the A* search algorithm, the actual cumulative cost of the starting node is the actual cumulative cost of the last unblocked node.

[0053] In some embodiments, in another possible implementation of the second aspect, when searching for the effective neighboring nodes corresponding to the current node in the closed list according to the A* search algorithm and a preset cost function, the slope cost of the effective neighboring nodes is determined in the following manner:

[0054] Determine the height difference and horizontal distance between the current node and its effective neighboring nodes. The effective neighboring nodes are non-obstacle nodes that are adjacent to the current node and are not in the closed list.

[0055] The slope cost of effective neighboring nodes is determined based on the height difference, horizontal distance, and slope cost function.

[0056] In some embodiments, in yet another possible implementation of the second aspect, when searching for the effective neighboring nodes corresponding to the current node in the closed list according to the A* search algorithm and a preset cost function, the rolling cost of the effective neighboring nodes is determined in the following manner:

[0057] Determine the change in heading angle of the robot from the current node to the effective neighboring nodes, where the effective neighboring nodes are non-obstacle nodes that are adjacent to the current node and are not in the closed list;

[0058] The rolling cost of effective neighboring nodes is determined based on the change in heading angle and the rolling cost function.

[0059] Thirdly, embodiments of this application provide an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the three-dimensional real-time navigation method for a robot as described above.

[0060] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program thereon, characterized in that the computer program, when executed by a processor, implements the three-dimensional real-time navigation method for a robot as described above.

[0061] Fifthly, embodiments of this application provide a computer program product that, when run on an electronic device, causes the electronic device to execute the three-dimensional real-time navigation method for a robot as described above.

[0062] The beneficial effects of this application embodiment compared to the prior art are as follows: By performing real-time path planning for the robot based on the Octomap map, and dynamically adjusting the local update range of the Octomap map according to the robot's real-time operation, the Octomap map is locally updated. Furthermore, the attenuation coefficient of each 3D grid within the locally updated range is dynamically adjusted according to the obstacle type, thereby enabling real-time local path planning within the updated Octomap map. Therefore, by providing 3D spatial information through the Octomap map and dynamically adjusting the update range of the Octomap map and the attenuation coefficient of the 3D grid based on the robot's actual speed and obstacle information, not only is the reliability and safety of the robot's 3D spatial navigation improved, but the computational efficiency of real-time navigation is also enhanced through locally updating the Octomap map. Attached Figure Description

[0063] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0064] Figure 1 This is a flowchart illustrating a three-dimensional real-time navigation method for a robot provided in an embodiment of this application;

[0065] Figure 2 This is a schematic flowchart illustrating the process of determining the scope of a local update according to an embodiment of this application;

[0066] Figure 3 This is a schematic diagram illustrating the dynamic updating of the initial occupancy probability and attenuation coefficient of a three-dimensional raster to be updated according to an embodiment of this application;

[0067] Figure 4 This is a flowchart illustrating a three-dimensional real-time navigation method for a robot provided in another embodiment of this application;

[0068] Figure 5 This is a schematic diagram of the structure of the three-dimensional real-time navigation device for a robot provided in the embodiments of this application;

[0069] Figure 6 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0070] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0071] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0072] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0073] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0074] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0075] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0076] The following description, with reference to the accompanying drawings, details the three-dimensional real-time navigation method, apparatus, electronic device, storage medium, and computer program for the robot provided in this application.

[0077] Figure 1 A flowchart illustrating a three-dimensional real-time navigation method for a robot provided in an embodiment of this application is shown.

[0078] like Figure 1 As shown, the robot's three-dimensional real-time navigation method includes the following steps:

[0079] Step 101: Obtain the current 3D point cloud data collected by the lidar in the robot, the current depth image collected by the depth camera, and the current linear velocity of the robot.

[0080] The robot in this application embodiment can refer to an intelligent agent equipped with autonomous mobility and a multi-dimensional perception system. Its types may include, but are not limited to: aerial robots (such as multi-rotor drones, vertical take-off and landing fixed-wing drones, etc.), ground mobile robots (such as wheeled or tracked inspection robots, logistics handling robots, wilderness exploration robots, sweeping robots, etc.), surface / underwater robots (such as unmanned surface vessels, underwater vehicles, etc.), and composite mobile robots (such as wheeled-legged robots, flying cockpits, etc.), but are not limited to these.

[0081] It should be noted that, regardless of the type of robot in the embodiments of this application, they all share the common feature of being equipped with LiDAR and depth cameras to support the three-dimensional real-time navigation method of the robot in this application.

[0082] The current 3D point cloud data can refer to the point cloud data of the robot's operating environment collected by the lidar onboard the robot at the current moment. The lidar obtains the 3D coordinates (X, Y, Z) of a large number of discrete points on the surface of objects in the environment by emitting a laser beam and measuring its flight time, and may include reflection intensity information.

[0083] It should be noted that point cloud data can provide a high-precision, high-confidence description of the geometry of the environment, and is particularly good at depicting the outline, edges and surface shapes of objects. It can be used to build accurate occupancy raster maps (such as OctoMap).

[0084] The current depth image refers to the depth image of the robot's operating environment captured by a depth camera mounted on the robot at the current moment. Depth cameras (such as structured light cameras, stereo cameras, and time-of-flight (ToF) depth cameras) can simultaneously output RGB color images and depth images. The value of each pixel in the depth image represents the distance from the corresponding scene point to the camera.

[0085] It should be noted that depth images can provide dense spatial information aligned with visual pixels. Compared to laser point cloud data, it can be more easily combined with computer vision algorithms for semantic understanding (such as object detection and segmentation).

[0086] The current linear velocity can refer to the vector value of the robot's instantaneous movement velocity in its local or global coordinate system. The current linear velocity reflects the robot's current real-time motion state. The robot's current linear velocity can be acquired by the state estimation module within the robot, but is not limited to this.

[0087] It should be noted that the current linear velocity can be one of the variables used to dynamically determine the local map update range in subsequent steps. Adaptively adjusting the range of the environment to be processed and planned based on the robot's current movement speed is the foundation for ensuring the algorithm's real-time performance and forward-looking safety.

[0088] In this embodiment of the application, the data acquisition frequency can be pre-configured, and the current three-dimensional point cloud data collected by the lidar and the current depth image collected by the depth camera can be acquired according to the data acquisition frequency, and the current linear velocity of the robot can be acquired accordingly.

[0089] It should be noted that the timestamps of sensors such as LiDAR and depth cameras in the robot can be synchronized in advance to ensure that the current 3D point cloud data, current depth image and current linear velocity collected each time can represent the robot's real-time operating status at the same moment.

[0090] As one possible approach, after acquiring a frame of current 3D point cloud data, current depth image, and current linear velocity, the known sensor extrinsic parameters (calibration results) on the robot can be used to uniformly transform the points in the current 3D point cloud data and current depth image into the robot's body coordinate system or a fixed global coordinate system, laying the foundation for subsequent fusion processing.

[0091] It should be noted that in actual use, the specific value of the data acquisition frequency can be configured according to actual needs and specific application scenarios, and this application embodiment does not limit this. For example, the data acquisition frequency can be 10Hz, 20Hz, etc.

[0092] Step 102: Based on the current linear velocity and the current depth image, determine the local update range of the Octomap information of the robot's operating scene, where the Octomap information of the operating scene contains information of multiple three-dimensional grids.

[0093] The octomap information refers to a dynamic probabilistic map describing the robot's 3D environment, constructed and maintained using octree-based occupancy grid mapping technology, based on the sensor data acquired in the aforementioned steps. It can be a hierarchical, probabilistic 3D voxel grid. The data structure of the octomap information can be as follows: the environmental space corresponding to the robot's operating environment is recursively divided into multiple 3D grids (voxels), and efficiently managed using an octree data structure. Each 3D grid node can store an occupancy probability value, representing the likelihood of an obstacle existing in the space corresponding to that 3D grid. The octomap information not only characterizes the geometric information of static structures (such as walls and the ground) in the operating environment, but more importantly, through a probabilistic update model, it continuously integrates the latest sensor observation data, dynamically reflecting the appearance, persistence, and disappearance of temporary obstacles (such as pedestrians or moved objects). Furthermore, the state of each 3D grid can be categorized into three states: occupied, vacant, and unknown. The runtime scenario Octomap information serves as a globally unique and unified environment representation, directly serving subsequent real-time path planning and replacing the separate global static map and local cost map in the traditional architecture.

[0094] It should be noted that the Octomap information of the robot's operating scene can be generated by fusing the global map of the robot's operating scene and the sensor data collected by the robot in real time. For example, the Octomap information of the operating scene of an indoor sweeping robot can be generated by fusing the overall indoor map of the robot's operating scene and the sensor data (such as 3D point cloud data and depth images) collected by the robot in real time during this operation; or it can be generated by fusing the sensor data collected by the robot in real time during this operation (such as when the robot is running in an unknown environment). This application embodiment does not limit this.

[0095] The local update range can refer to the spatial area where real-time path search is performed within each path planning cycle.

[0096] It should be noted that existing technologies typically update maps and plan paths within a fixed area. For example, each path planning operation updates the map and plans the path within a 5×5 meter area. However, in this embodiment, the size and shape of the local update area for map updates and path planning are not fixed but dynamically adjusted. For instance, the local update area can be a sphere or cube centered on the robot's position, but it is not limited to these. Taking a sphere centered on the robot's position as an example, the radius of the local update area can be determined based on the robot's real-time operating information to achieve dynamic updates. It is worth noting that the purpose of dynamically updating the local update area based on the robot's real-time operating status is to achieve a balance between safety and real-time performance (efficiency). Therefore, the local update area can include the area that the robot may reach in the future at its current linear velocity, as well as all dynamic and static obstacles that need to be avoided within that area. Furthermore, the local update area should not be too large to avoid unnecessary processing and searching of irrelevant, distant, or unchanging environmental areas, thereby concentrating limited computing resources on the local environment crucial to the current navigation task.

[0097] As one possible implementation, the size of the local update range can be positively correlated with the robot's current linear velocity. For example, when the local update range is a sphere, a mapping relationship can be established between the radius of the local update range and the robot's linear velocity. This mapping relationship can be continuous or segmented, and this embodiment does not limit this. After obtaining the robot's current linear velocity, Zhang can determine the radius matching the current linear velocity as the base radius of the local update range based on this mapping relationship. For example, when the robot is stationary or moving slowly at low speed, a smaller base radius (e.g., 3 meters) can be used, focusing only on the immediate surroundings. When the robot accelerates or moves at high speed, the base radius can expand linearly or non-linearly with increasing speed (e.g., the base radius increases by 1 meter for every 0.5 m / s increase in linear velocity) to ensure sufficient distance and time to perceive and respond to obstacles ahead. Subsequently, semantic recognition can be performed on the current depth image only to determine the type and number of obstacles contained in the current depth image, and then the base radius can be corrected based on the type and / or number of obstacles contained within the base radius in the current depth image. For example, when the area corresponding to the base radius contains dynamic obstacles, the base radius can be appropriately reduced, i.e., the local update range can be reduced (for example, the more dynamic obstacles are included, the greater the reduction in the base radius). When the area corresponding to the base radius does not contain dynamic obstacles, the base radius can be kept unchanged or appropriately increased. Finally, a sphere centered on the robot's current position and with a radius equal to the corrected base radius can be defined as the local update range.

[0098] As one possible implementation, a decision model for dynamically updating the local update range can be pre-trained. The current linear velocity and current depth image (or obstacle recognition results (such as obstacle type, number, location, etc.)) can be input into the decision model so that the decision model processes the input current linear velocity and current depth image to output the radius of the local update range. The local update range can then be determined based on the radius, or an optimized three-dimensional spatial boundary (such as an axially aligned bounding box, sphere, or more complex geometry) can be output.

[0099] As one possible implementation, the calculation of the local update range can be decomposed into two parts: a basic radius and a semantic radius. First, the basic radius is determined based on the current linear velocity, and then the semantic radius is determined based on the depth image. The final range is determined by both, thereby improving the reliability of dynamically updating the local update range and further enhancing the safety and efficiency of real-time path planning. Specifically, in one possible implementation of this application embodiment, step 102 may include:

[0100] Obtain the robot's minimum update radius, dynamic obstacle prediction time window, and collision avoidance safety buffer radius;

[0101] The base radius corresponding to the local update range is determined based on the robot's minimum update radius, dynamic obstacle prediction time window, collision avoidance safety buffer radius, and current linear velocity.

[0102] Based on the base radius and the current depth image, determine the types of obstacles included in the current depth image within the base radius;

[0103] The semantic radius corresponding to the local update range is determined based on the types of obstacles contained within the base radius of the current depth image;

[0104] The local update range is determined based on the base radius and semantic radius.

[0105] The minimum update radius refers to the minimum radius at which the robot performs map updates and path planning each time, and can be used to define a safety baseline value. As an example, the minimum update radius can be determined based on the dimensions of the robot body; for instance, the radius of the robot body's circumcircle can be used as the minimum update radius (if the robot body is circular, then the radius of the robot body can be used as the minimum update radius), but it is not limited to this.

[0106] It should be noted that, regardless of the robot's state, its local update range should at least cover this minimum update radius to ensure basic responsiveness to immediate hazards (such as objects that suddenly appear from the side).

[0107] The dynamic obstacle prediction time window can refer to the length of time required for path planning in advance to deal with dynamic obstacles.

[0108] The collision avoidance safety buffer radius can refer to an additional static safety margin (e.g., 0.5 meters) that can be used to compensate for robot body size, positioning errors, and control errors.

[0109] As one possible implementation, parameters such as minimum update radius, dynamic obstacle prediction time window, and collision avoidance safety buffer radius can be pre-configured to a fixed value according to the actual application scenario (such as the robot's running speed range, system latency, etc.).

[0110] As one possible implementation, parameters such as the minimum update radius, dynamic obstacle prediction time window, and collision avoidance safety buffer radius can be dynamically updated based on the robot's real-time operating data. For example, these parameters can be dynamically updated based on the robot's linear velocity, the type of obstacle within its perception range, and the number of obstacles. For instance, when the robot's linear velocity is high, the minimum update radius, dynamic obstacle prediction time window, and collision avoidance safety buffer radius can be set to larger values; conversely, when the robot's linear velocity is low, these parameters can be set to smaller values. Alternatively, when the obstacle types within the robot's perception range include dynamic obstacles or the number of obstacles is large, the minimum update radius, dynamic obstacle prediction time window, and collision avoidance safety buffer radius can be set to smaller values; conversely, when the obstacle types within the robot's perception range do not include dynamic obstacles or the number of obstacles is small, the minimum update radius, dynamic obstacle prediction time window, and collision avoidance safety buffer radius can be set to larger values. Alternatively, based on the above principles, parameters such as the robot's linear velocity and the type and number of obstacles within its perception range can be comprehensively considered to jointly update the minimum update radius, the dynamic obstacle prediction time window, and the collision avoidance safety buffer radius.

[0111] It should be noted that the specific methods for determining the minimum update radius, the dynamic obstacle prediction time window, and the collision avoidance safety buffer radius listed above are merely exemplary and should not be considered as limitations on this application. In actual use, the need to update these parameters in real time and the specific update method can be determined according to actual needs and specific application scenarios. This application embodiment does not limit this.

[0112] As one possible implementation, parameters such as the robot's minimum update radius, dynamic obstacle prediction time window, and collision avoidance safety buffer radius can be stored in the robot's storage medium. Therefore, when it is necessary to determine the local update range, the pre-stored minimum update radius, dynamic obstacle prediction time window, and collision avoidance safety buffer radius can be directly obtained, and then the base radius corresponding to the local update range can be determined by the following formula:

[0113]

[0114] in, The base radius corresponding to the local update range. To minimize the update radius, The current linear velocity, For the dynamic obstacle prediction time window, The safety buffer radius is designed to prevent collisions.

[0115] It is worth noting that, Reflects the current linear velocity Below, the robot predicts dynamic obstacles within the time window. The theoretical maximum distance the robot can move. This is the physical basis for ensuring that the robot can detect and handle dynamic obstacles ahead in advance. Plus... This is to add a fixed safety buffer on top of the aforementioned distance. Ultimately, the `max` function ensures that the calculated radius is not less than the minimum safe radius. .

[0116] As one possible approach, after determining the base radius corresponding to the local update range, the visual information provided by the current depth image acquired by the depth camera can be used to identify the types of obstacles contained within the base radius using a target detection model (such as the YOLO model), and the local update range can be finely adjusted based on the behavioral characteristics or perceptual uncertainties of different types of obstacles.

[0117] As an example, the radius increments corresponding to each obstacle type can be pre-configured, and then the semantic radius corresponding to the local update range can be determined based on the radius increments corresponding to the obstacle types included within the base radius. If the base radius contains only one obstacle type, the radius increment corresponding to that obstacle type can be determined as the semantic radius; if the base radius contains multiple obstacle types, the maximum radius increment corresponding to each obstacle type can be determined as the semantic radius.

[0118] For example, such as Figure 2As shown, the pre-configured obstacle types include dynamic obstacles, transparent obstacles (or low-reflectivity obstacles), and fixed-structure obstacles. The radius increments for dynamic obstacles, transparent obstacles, and fixed-structure obstacles are 2m, 1m, and -1m, respectively. If it is determined that the base radius contains dynamic obstacles (regardless of whether it also contains transparent or fixed-structure obstacles), the semantic radius can be determined to be 2m; if it is determined that the base radius does not contain dynamic obstacles but contains transparent obstacles (regardless of whether it contains fixed-structure obstacles), the semantic radius can be determined to be 1m; if it is determined that the base radius does not contain either dynamic or transparent obstacles but contains fixed-structure obstacles, the semantic radius can be determined to be -1m; if the base radius does not contain any of the above types of obstacles, the semantic radius can be determined to be 0, i.e., no correction is made to the base radius.

[0119] It should be noted that dynamic obstacles (such as pedestrians and vehicles) may approach the robot quickly, requiring a longer braking or avoidance distance. Therefore, a positive increment (e.g., a radius increment of 2m) can be assigned to them. Transparent obstacles (such as glass windows and doors) are not good for active sensors like LiDAR, easily leading to missed detection or inaccurate ranging, but visual information can indicate their presence. To compensate for this perceptual uncertainty, a positive increment (e.g., a radius increment of 1m) can also be assigned, expanding the local update range to reserve reaction space in advance. Fixed structural obstacles (such as static, stable objects like walls and pillars) remain in place, and no additional dynamic response space needs to be reserved for them. Therefore, a negative or zero increment (e.g., a radius increment of -0.5m, -1m, or 0m) can be assigned to them, appropriately shrinking the local update range to improve computational efficiency.

[0120] Furthermore, the obstacle types listed in the embodiments of this application, such as dynamic obstacles, transparent obstacles (or low reflectivity obstacles), and fixed structure obstacles, are merely exemplary and should not be considered as limitations on this application. In actual use, the types of obstacles to be monitored and their corresponding radius increments can be configured according to the specific usage scenario, and the embodiments of this application do not limit this.

[0121] As one possible implementation, after determining the base radius and semantic radius corresponding to the local update range, the sum of the base radius and semantic radius can be used to determine the radius of the local update range, and then the local update range can be determined based on this radius.

[0122] Step 103: Based on the local update range, the current 3D point cloud data, and the current depth image, determine the observation occupancy probability and the attenuation coefficient corresponding to each 3D grid to be updated, wherein each 3D grid to be updated is located within the local update range.

[0123] The observation occupancy probability of the 3D raster to be updated can be a parameter determined based on the current 3D point cloud data, directly reflecting the likelihood that the 3D raster to be updated is occupied at the current moment. The higher the observation occupancy probability of the 3D raster to be updated, the greater the likelihood that the 3D raster to be updated is occupied at the current moment.

[0124] The decay coefficient corresponding to the 3D raster to be updated can be a parameter in Octomap that controls the rate at which historical information is forgotten. The decay coefficient of the 3D raster to be updated defines the rate at which the original occupancy probability of the 3D raster to be updated converges to its corresponding current observation probability over time.

[0125] It should be noted that during the Octomap map update process, for each new frame of data acquired, the occupancy probability of a 3D grid is updated based on its corresponding attenuation coefficient. This gradually brings the occupancy probability closer to the current observation probability determined at the current moment, rather than directly modifying the occupancy probability to the current observation probability. Existing technologies typically update the occupancy probability of 3D grids based on a default fixed attenuation coefficient. This application, however, dynamically determines the attenuation coefficient of each 3D grid to be updated based on its characteristics (such as the type of obstacle occupying the grid), thereby further improving the reliability and safety of the robot's real-time path planning.

[0126] As one possible implementation, the attenuation coefficient corresponding to the 3D grid can be dynamically determined based on the obstacle type, thereby further improving the reliability and safety of the robot's real-time path planning. Specifically, in one possible implementation of this application embodiment, step 103 may include:

[0127] Voxelize the current 3D point cloud data within the local update range to determine each 3D raster to be updated;

[0128] Determine the initial occupancy probability and observation occupancy probability for each 3D grid to be updated;

[0129] Target detection is performed on the current depth image within the local update range to determine the obstacle type and confidence level corresponding to each 3D raster to be updated;

[0130] Based on the obstacle type and confidence level of each 3D grid to be updated, the attenuation coefficient of each 3D grid to be updated is determined, and the initial occupancy probability of each 3D grid to be updated is corrected.

[0131] The initial occupancy probability of the 3D grid to be updated can refer to the original occupancy probability of the 3D grid to be updated before the current time; that is, the initial occupancy probability of the 3D grid to be updated refers to the occupancy probability of the 3D grid to be updated after the last update.

[0132] It should be noted that in the Octomap map, the state of a 3D grid can be classified into any one of occupied, idle, or unknown based on the occupancy probability corresponding to the 3D grid and a pre-configured probability threshold. For example, when the occupancy probability of a 3D grid is greater than or equal to the occupancy probability threshold, the state of the 3D grid can be determined as occupied; when the occupancy probability of a 3D grid is less than the occupancy probability threshold but greater than the idle probability threshold, the state of the 3D grid can be determined as unknown; and when the occupancy probability of a 3D grid is less than or equal to the idle probability threshold, the state of the 3D grid can be determined as idle. The occupancy probability threshold is greater than the idle probability threshold; for example, the occupancy probability threshold is 0.65, and the idle probability threshold is 0.35. This application embodiment does not limit this. Based on this, if a 3D grid to be updated is detected for the first time at the current moment, that is, it has no historical occupancy probability before, and its state before this is unknown, then the initial occupancy probability of the 3D grid to be updated can be determined as a value between the occupancy probability threshold and the idle probability threshold, such as 0.5.

[0133] As one possible implementation, after determining the local update range through the aforementioned steps, the input current 3D point cloud data can be spatially filtered according to the local update range to remove all point clouds outside the local update range, thus ensuring that subsequent calculations focus on the local environment relevant to the current navigation task. The truncated point cloud space is then voxelized at a preset fixed resolution (e.g., 0.1m × 0.1m × 0.1m) and divided into uniform 3D grids. Each smallest cubic unit is called a voxel, i.e., a 3D grid to be updated. Then, all point clouds can be traversed, and their 3D coordinates can be assigned to the corresponding 3D grid to be updated. Each 3D grid to be updated, V_i, will be associated with the set of all point clouds falling within it. Thus, the environmental space corresponding to the local update range is discretized into a series of 3D grids {V_1, V_2, ..., V_n} to be updated, each containing point cloud observation data.

[0134] After identifying each 3D raster to be updated within the local update range, the initial occupancy probability of each raster can be determined according to the method described in the explanation of the initial occupancy probability. That is, for a previously observed 3D raster to be updated, its original occupancy probability can be determined as its corresponding initial occupancy probability; for a raster to be updated observed for the first time, its initial occupancy probability can be determined as a value between the idle probability threshold and the occupancy probability threshold. For example, if the idle probability threshold is 0.35 and the occupancy probability threshold is 0.65, then the initial occupancy probability of the first observed 3D raster to be updated can be determined as 0.5.

[0135] After identifying the individual 3D grid cells to be updated within the local update range, the observation occupancy probability of each cell can be determined based on the current 3D point cloud data. As an example, the observation occupancy probability of each cell can be determined using the following formula:

[0136]

[0137] in, Let be the observation occupancy probability corresponding to a 3D raster to be updated. This indicates the number of points detected within the 3D raster to be updated in the current 3D point cloud data. This indicates the total number of points emitted by the lidar when measuring the 3D grid to be updated. The sensor noise figure (e.g., a value of 0.05) is used to characterize the inherent ranging random error and noise level of the lidar.

[0138] As a possible implementation, to further improve the safety and reliability of real-time path planning, visual information (i.e., the current depth image) can be processed in parallel to inject semantic understanding into the environmental data, thereby associating the various 3D grids to be updated determined in the aforementioned steps with obstacle categories. Therefore, based on the calibration relationship between the camera and the robot body, the image region corresponding to the local update range in the current depth image can be calculated. This image region is then input into a target detection model (such as the YOLO model) to determine the detection boxes, obstacle categories (such as "person", "car", "glass door"), positions (coordinates), sizes, and confidence levels of each obstacle contained within the image region, but this is not limited to these. Then, using the intrinsic parameters and depth information of the depth camera, each detection box is back-projected into 3D space to form a corresponding 3D bounding volume. Next, this 3D bounding volume is spatially intersected with the 3D grids to be updated generated in the aforementioned steps. All 3D grids to be updated that intersect with or are contained within this bounding volume can be marked as associated with this detection result and "inherit" the obstacle type and confidence level of the detection box. It should be noted that if a 3D grid to be updated may be associated with multiple detection results, the obstacle category with the highest confidence can be retained.

[0139] It is worth noting that since different types of obstacles may have different disappearance rates and perceptibility characteristics within the robot's detection range, the attenuation coefficient of the 3D grid to be updated can be dynamically updated based on the obstacle category and confidence level corresponding to the 3D grid to be updated, and its corresponding initial occupancy probability can be appropriately corrected. This ensures that when updating the occupancy probability of the 3D grid to be updated, the update rate of its occupancy probability matches the obstacle type occupying the 3D grid to be updated, thereby further improving the safety and reliability of real-time path planning.

[0140] One possible implementation is to pre-configure the mapping relationships between obstacle types and attenuation coefficients, as well as between obstacle types and occupancy probabilities. Thus, when the obstacle type corresponding to the 3D grid to be updated has a confidence level greater than or equal to a confidence threshold, the attenuation coefficient corresponding to that obstacle type is determined as the attenuation coefficient for the 3D grid to be updated, and the occupancy probability corresponding to that obstacle type is determined as the corrected initial occupancy probability for the 3D grid to be updated. If the confidence level of the 3D grid to be updated is less than the confidence threshold, or if the obstacle type corresponding to the 3D grid to be updated does not exist in the aforementioned mapping relationships, then the attenuation coefficient and initial occupancy probability for the 3D grid to be updated do not need to be corrected, and the default attenuation coefficient can be used.

[0141] It should be noted that the attenuation coefficient and initial occupancy probability of the 3D grid to be updated are only corrected when the confidence level of the obstacle type corresponding to the 3D grid to be updated is greater than or equal to the confidence threshold. This is because the reliability of the determined obstacle type is higher when the confidence level is higher. Correcting the attenuation coefficient and initial occupancy probability of the 3D grid to be updated only at this time can avoid erroneous corrections caused by incorrect obstacle type identification, and further ensure the reliability and security of real-time path planning. In addition, in this embodiment, only the attenuation coefficient of the 3D grid to be updated may be corrected, and this embodiment does not limit this.

[0142] As one possible implementation, focus can be placed only on obstacle types that significantly impact the safety of the robot's real-time navigation, thereby further reducing the computational load of map updates while ensuring real-time navigation safety. Specifically, in one possible implementation of this application, determining the attenuation coefficient for each of the three-dimensional grids to be updated based on the obstacle type and confidence level, and correcting the initial occupancy probability for each of the three-dimensional grids to be updated, may include:

[0143] If the obstacle type corresponding to the first three-dimensional grid to be updated is a dynamic obstacle and the confidence level is greater than the confidence level threshold, the initial occupancy probability corresponding to the first three-dimensional grid to be updated is corrected to the first occupancy probability value, and the attenuation coefficient corresponding to the first three-dimensional grid to be updated is determined as the first attenuation value, wherein the first three-dimensional grid to be updated is any three-dimensional grid to be updated.

[0144] When the obstacle type corresponding to the first 3D grid to be updated is a transparent obstacle, the initial occupancy probability corresponding to the first 3D grid to be updated is corrected to a second occupancy probability value, and the attenuation coefficient corresponding to the first 3D grid to be updated is determined as a second attenuation value, wherein the second occupancy probability value is less than the first occupancy probability value, and the second attenuation value is less than the first attenuation value.

[0145] In one possible implementation of the embodiments of this application, such as Figure 3 As shown, since dynamic obstacles move, they typically disappear quickly from the corresponding 3D grid. Therefore, the occupancy probability of the 3D grid to be updated corresponding to a dynamic obstacle can be rapidly decayed. Furthermore, to prevent misidentification due to rapid decay, the initial occupancy probability of the 3D grid to be updated corresponding to a dynamic obstacle can be forcibly corrected to a larger value. Therefore, for a 3D grid to be updated, if the obstacle type corresponding to the 3D grid is a dynamic obstacle and the confidence level is greater than a confidence threshold (e.g., 0.8), the initial occupancy probability of the 3D grid to be updated can be corrected to a larger first occupancy probability value (e.g., 0.9), and the decay coefficient corresponding to the 3D grid to be updated can be determined to a larger first decay value (e.g., 0.5).

[0146] Furthermore, because the laser emitted by LiDAR can penetrate transparent obstacles (such as glass or water), it cannot accurately determine the position of transparent obstacles such as glass doors, potentially leading to missed detections of transparent obstacles and causing robot collisions. However, depth images captured by depth cameras can effectively identify transparent obstacles. Therefore, the occupancy probability of the corresponding 3D grid can be forcibly corrected to the occupancy probability corresponding to the presence of an obstacle, and its attenuation coefficient can be set to a small value to allow it to decay slowly, compensating for missed detections caused by laser penetration. Since it is a static object, setting extremely slow attenuation ensures its persistent presence in the map, guaranteeing safety. Therefore, if the obstacle type corresponding to a 3D grid to be updated is a transparent obstacle, the initial occupancy probability of the 3D grid to be updated can be corrected to a higher second occupancy probability value (e.g., 0.8), and the attenuation coefficient corresponding to the 3D grid to be updated can be set to a second attenuation value (e.g., 0.001). The second occupancy probability is less than the first occupancy probability, and both the first and second occupancy probabilities indicate that the 3D grid is occupied (i.e., an obstacle exists within the 3D grid).

[0147] If the obstacle type corresponding to a 3D grid to be updated is neither a dynamic obstacle nor a transparent obstacle, or if no obstacle is detected in the 3D grid to be updated, then the attenuation coefficient and initial occupancy probability of the 3D grid to be updated do not need to be corrected, and the default attenuation coefficient and the initial occupancy probability determined by the aforementioned steps can be used.

[0148] It should be noted that, in order to compensate for the possible missed detection of transparent obstacles by the LiDAR, the attenuation coefficient of the corresponding 3D grid to be updated is determined to be a very small value, so that the occupancy probability of the corresponding 3D grid to be updated decays very slowly, ensuring the safe operation of the robot. Therefore, when the obstacle type is a transparent obstacle, the confidence level corresponding to the 3D grid to be updated can be disregarded, which can compensate for the missed detection of transparent obstacles as much as possible.

[0149] It should be noted that the above examples are merely illustrative and should not be considered as limitations on this application. In actual use, the types of obstacles to be monitored, as well as the attenuation coefficients and occupancy probabilities corresponding to each obstacle type, can be set according to actual needs and specific application scenarios. This application does not limit these aspects.

[0150] Step 104: Update the occupancy probability of each 3D grid to be updated based on the observation occupancy probability and the attenuation coefficient of each 3D grid to be updated, so as to generate the updated Octomap information of the running scene.

[0151] In this embodiment of the application, after determining the observation occupancy probability and attenuation coefficient of each three-dimensional grid to be updated within the local update range, the occupancy probability of each three-dimensional grid to be updated can be updated to generate updated Octomap information of the running scene.

[0152] As an example, the occupancy probability of each 3D raster to be updated can be calculated using the following formula:

[0153]

[0154] in, The updated current occupancy probability corresponds to the 3D raster to be updated. The original occupancy probability of the 3D raster to be updated (that is, the initial occupancy probability determined in the previous steps). The observation occupancy probability corresponding to the 3D raster to be updated. The attenuation coefficient corresponding to the 3D raster to be updated. This is the map update interval.

[0155] It should be noted that the time interval It can be the reciprocal of the data acquisition frequency mentioned in the preceding steps, and in the embodiments of this application, the map update frequency, data acquisition frequency, and the frame rate of the LiDAR and depth camera can be uniform.

[0156] Step 105: Based on the updated Octomap information of the running scene and the A* search algorithm, determine the real-time navigation path of the robot within the local update range.

[0157] In this embodiment of the application, after updating the Octomap information of the robot's operating scene, the robot's real-time navigation path within the local update range can be planned based on the updated Octomap information and the A* search algorithm. That is, a safe, smooth, and feasible path that conforms to the robot's kinematic constraints can be quickly planned from the robot's current position to the local target point within the local update range.

[0158] As one possible implementation, based on the updated Octomap information of the running scenario, non-obstacle nodes with an occupation probability less than or equal to an idle occupation probability within the local update range can be identified. Then, the A* search algorithm can be used to select nodes that the robot can travel with from among the non-obstacle nodes to determine the robot's real-time navigation path within the local update range.

[0159] It should be noted that, in determining the real-time navigation path within the local update range using the A* search algorithm, the specific implementation process and principle of the A* search algorithm in the prior art can be referred to, and the embodiments of this application will not elaborate on this.

[0160] The 3D real-time navigation method for robots provided in this application performs real-time path planning based on an Octomap map, dynamically adjusts the local update range of the Octomap map according to the robot's real-time operation, and dynamically adjusts the attenuation coefficient of each 3D grid within the local update range according to the obstacle type. This allows for real-time local path planning within the updated Octomap map. Therefore, by using the Octomap map to provide 3D spatial information and dynamically adjusting the update range of the Octomap map and the attenuation coefficient of the 3D grid based on the robot's actual speed and obstacle information, the reliability and safety of the robot's 3D spatial navigation are improved, and the computational efficiency of real-time navigation is enhanced through local updates of the Octomap map.

[0161] The following is combined Figure 4 The three-dimensional real-time navigation method for robots provided in the embodiments of this application will be further described.

[0162] Figure 4 A flowchart illustrating another three-dimensional real-time navigation method for a robot provided in an embodiment of this application is shown.

[0163] like Figure 4 The robot's three-dimensional real-time navigation method includes the following steps:

[0164] Step 401: Obtain the current 3D point cloud data collected by the lidar in the robot, the current depth image collected by the depth camera, and the current linear velocity of the robot.

[0165] Step 402: Based on the current linear velocity and the current depth image, determine the local update range of the Octomap information of the robot's operating scene, where the Octomap information of the operating scene contains information of multiple three-dimensional grids.

[0166] Step 403: Based on the local update range, the current 3D point cloud data, and the current depth image, determine the observation occupancy probability and the attenuation coefficient corresponding to each 3D grid to be updated, wherein each 3D grid to be updated is located within the local update range.

[0167] Step 404: Based on the observation occupancy probability and the attenuation coefficient of each 3D grid to be updated, update the occupancy probability of each 3D grid to be updated to generate updated running scene Octomap information, wherein the updated running scene Octomap information contains the updated current occupancy probability of each 3D grid.

[0168] The specific implementation process and principle of steps 401-404 above can be found in the detailed description of the above embodiments, and will not be repeated here.

[0169] Step 405: Based on the current occupied probability and idle probability threshold of each 3D grid, determine each obstacle node and each non-obstacle node within the local update range.

[0170] One possible implementation is to iterate through all 3D grids within the local update range and their updated current occupancy probability values ​​after the map is updated. The current occupancy probability value of each 3D grid is then compared with an idle probability threshold. If the current occupancy probability of a 3D grid is less than or equal to the idle probability threshold, the spatial unit represented by that 3D grid is determined to be in an "idle" state, thus identifying the 3D grid as a non-obstacle node. Conversely, if the current occupancy probability of a 3D grid is greater than the idle probability threshold, the spatial unit represented by that 3D grid is determined to be in an "occupied" or "unknown" state, indicating a potential obstacle risk; therefore, the 3D grid can be identified as an obstacle node.

[0171] It should be noted that the setting of the idle probability threshold is essentially a reflection of the navigation system's safety strategy: a higher idle probability threshold results in stricter obstacle detection, a more conservative planned path, and a greater safety margin; a lower idle probability threshold results in greater tolerance for environmental passability, and the planning may be more aggressive. In actual use, the specific value of the idle probability threshold can be set according to actual needs and specific application scenarios, and this application embodiment does not limit this.

[0172] Step 406: Based on the robot's historical navigation path, obstacle nodes, and non-obstacle nodes, determine the robot's starting node and target node within the local update range.

[0173] Historical navigation paths can refer to navigation paths that were planned before the current moment.

[0174] The starting node can be the last unblocked node in the historical navigation path within the scope of the local update; the target node can be the endpoint of the current path planning within the scope of the local update.

[0175] One possible approach is to acquire the robot's current or previously generated historical navigation path. This historical path can be spatially compared and analyzed with the set of obstacle nodes within the local update range to determine whether the historical path is blocked by newly appearing obstacles in front of the robot's current position. Based on this determination, the last unblocked node in the historical navigation path can be selected as the starting node for real-time navigation path planning. This means that the last node on the historical path not blocked by any new obstacle nodes is considered safe and effective in the current environment. Using this node as the starting point for a new navigation path naturally inherits previous planning results, ensuring a smooth path transition. Furthermore, only the obstructed or unknown parts ahead are replanned, significantly reducing the search space, avoiding the computational overhead and path jitter caused by full path replanning, and ensuring the continuity and stability of navigation path planning.

[0176] It should be noted that if the historical navigation path is not blocked by any new obstacles, the starting node is the last node in the historical navigation path, so the robot's real-time navigation path can be planned from the end of the historical navigation path.

[0177] One possible implementation is to determine the target node on the boundary of the local update range based on the robot's direction of travel or the global goal. For example, the target node could be the next waypoint on the global path within that local area, or a point located on the boundary of the local update range along the direction the robot is moving toward the global goal.

[0178] Step 407: Search each non-obstacle node according to the A* search algorithm and the preset cost function to determine the real-time navigation path between the starting node and the target node. The preset cost function is determined based on the actual cumulative cost function, heuristic function, slope cost function and rolling cost function.

[0179] The preset cost function can be used to determine the cost value of each non-obstacle node when searching for each non-obstacle node using the A* search algorithm, so as to determine the nodes that can be added to the real-time navigation path based on the cost value of each non-obstacle node.

[0180] The actual cumulative cost function g(n) measures the actual cumulative cost (typically distance, time, or energy) from the starting node to the current node n being searched, accumulating as the path extends. This function guarantees the basic optimality of the path, ensuring that the search finds the true minimum-cost path from the starting node to the current node n. As an example, the actual cumulative cost of the current node can be calculated based on the distance traveled in three-dimensional space (such as Euclidean distance).

[0181] As an example, the actual cumulative cost of the starting node can be 0.

[0182] As an example, the actual cumulative cost of the starting node can be the actual cumulative cost of the last unblocked node in the historical navigation path. Thus, by making the starting node of this path planning inherit the actual cumulative cost of the last unblocked node in the historical navigation path, cost continuity is ensured, enabling a smooth transition between the historical navigation path and the newly planned path, avoiding sharp turns, and ensuring accurate energy consumption estimation. Only the blocked portion of the historical navigation path is adjusted, maintaining the trajectory stability of the unaffected segments.

[0183] The heuristic function h(n) measures the estimated cost from the current node n to the target node; it is a prediction. As an example, in three-dimensional space, it can typically be calculated using the three-dimensional Euclidean distance between nodes. The heuristic function h(n) provides directional guidance for path searching and is the source of the "heuristic" capability of the A* search algorithm. A well-designed h(n) can effectively narrow the search range and significantly improve the algorithm's efficiency. It must satisfy acceptability (i.e., the heuristic function h(n) never overestimates the actual cost of reaching the target) to ensure that the algorithm can find the optimal solution. As an example, the heuristic function h(n) can be calculated using any of the three-dimensional Euclidean distance, Manhattan distance, or diagonal distance between the current node n and the target node (but not limited to these), because these distances are always less than or equal to the length of any practically feasible path.

[0184] The slope cost function measures the steepness of a path in the vertical direction. By introducing terrain constraints, it penalizes positive slopes (uphill), negative slopes (downhill), or the absolute value of the slope, tending to select flatter, more stable paths. This benefits ground robots by preventing tipping over and reducing energy consumption, and helps aerial drones maintain flight efficiency and safety.

[0185] As one possible implementation, when searching for the effective neighboring nodes corresponding to the current node being searched that is in the closed list, based on the A* search algorithm and the preset cost function, the slope cost of the effective neighboring nodes is determined in the following way:

[0186] Determine the height difference and horizontal distance between the current node and its effective neighboring nodes. The effective neighboring nodes are non-obstacle nodes that are adjacent to the current node and are not in the closed list.

[0187] It should be noted that when performing path planning using the A* search algorithm, a priority queue can be created first as an open list to store nodes to be searched. The sorting criterion can be the cost value of the nodes determined by a preset cost function. A set can also be created as a closed list to store explored nodes. Next, the algorithm is initialized by adding the starting node S to the open list. Its actual cumulative cost g(S) is set: if S is a completely new starting point, then g(S) = 0; if S inherits from the historical navigation path and is the last unblocked node in the historical navigation path, then g(S) can inherit the g value of that last unblocked node. Its heuristic value h(S) (the three-dimensional Euclidean distance to the target node) is calculated. Based on the relationship between the starting node S and its parent node (which can be empty or itself during initialization), the slope cost slope(S) and roll cost roll(S) are calculated. Finally, based on the preset cost function, and the actual cumulative cost g(S), heuristic value h(S), slope cost slope(S), and roll cost roll(S) of the starting node S, the cost value f(S) of the starting node S is calculated. The main loop then begins, retrieving the node with the smallest cost value f(n) from the open list, denoted as the current node C, and moving it from the open list to the closed list. Next, neighborhood expansion is performed, traversing all valid neighboring nodes N of the current node C in 3D space and calculating the cost value of each valid neighboring node. These valid neighboring nodes are then added to the open list for further searching. Therefore, in this process, for a valid neighboring node N corresponding to the current node C, the slope cost of that valid neighboring node N can be determined based on the height difference and horizontal distance between the current node C and the valid neighboring node N.

[0188] As an example, the ratio of the height difference between the current node C and its effective neighboring nodes N to the horizontal distance can be determined as the slope cost of the effective neighboring nodes. That is: Slope(N) = (Δz / Δd)^2.

[0189] Where Slope(N) is the slope cost of the effective neighbor node N, Δz is the height difference (i.e., the distance difference in the vertical direction (Z axis)) between the current node C and the effective neighbor node N, and Δd is the horizontal distance between the current node C and the effective neighbor node N.

[0190] in, ,or, .in, Let C be the distance difference between the current node C and its effective neighboring nodes N in the first horizontal direction (X-axis). The distance difference between the current node C and the effective neighbor node N in the second horizontal direction (Y-axis); the X, Y, and Z axes are all perpendicular to each other, and the Z-axis is perpendicular to the plane containing the X and Y axes.

[0191] It should be noted that, in Using the square term can strengthen the penalty for steep slopes, making the real-time navigation path smoother.

[0192] As one possible implementation, based on the A* search algorithm and a preset cost function, when searching for the effective neighboring nodes corresponding to the current node being searched in the closed list, the rolling cost of the effective neighboring nodes can be determined in the following way:

[0193] Determine the change in heading angle of the robot from the current node to the effective neighboring nodes, where the effective neighboring nodes are non-obstacle nodes that are adjacent to the current node and are not in the closed list;

[0194] The rolling cost of effective neighboring nodes is determined based on the change in heading angle and the rolling cost function.

[0195] As an example, the absolute value of the change in the robot's heading angle from the current node C to the effective neighbor node N can be determined as the rolling cost of the effective neighbor node N, i.e.: .

[0196] in, The rolling cost of the effective neighboring node N, This represents the change in the robot's heading angle from the current node C to the effective neighboring node N.

[0197] As an example, the absolute square of the change in heading angle of the robot from the current node C to the effective neighbor node N can be determined as the rolling cost of the effective neighbor node N, i.e.: .

[0198] in, The rolling cost of the effective neighboring node N, This represents the change in the robot's heading angle from the current node C to the effective neighboring node N.

[0199] It should be noted that the specific forms of the slope cost function and rolling cost function listed above are merely exemplary and should not be considered as limitations on this application. In actual use, the slope cost function and rolling cost function can be determined according to actual needs and specific application scenarios, and the embodiments of this application do not limit this.

[0200] As one possible implementation, the preset cost function can be determined based on the weighted sum of the actual cumulative cost function, heuristic function, slope cost function, and rolling cost function.

[0201] As an example, the preset cost function can be determined by the following formula:

[0202]

[0203] in, Let n be the cost of node n. This represents the actual cumulative cost of node n. Let n be the heuristic value for node n. The slope cost for node n. Let n be the rolling cost. , is the weight, and n is the node number.

[0204] It should be noted that in actual use, the specific requirements and application scenarios can be determined accordingly. , The value of is not limited in the embodiments of this application.

[0205] The following uses a preset cost function Taking an example, the specific process of path planning using the A* search algorithm in this application embodiment will be explained:

[0206] When performing path planning using the A* search algorithm, a priority queue can be created as an open list to store nodes to be searched. The sorting criterion can be the cost value of the nodes determined by a preset cost function. A set can also be created as a closed list to store explored nodes. Next, the algorithm is initialized by adding the starting node S to the open list. Its actual cumulative cost g(S) is set: if S is a completely new starting point, then g(S) = 0; if S inherits from the historical navigation path and is the last unblocked node in the historical navigation path, then g(S) can inherit the g value of that last unblocked node. Its heuristic value h(S) (such as the three-dimensional Euclidean distance to the target node) is calculated. Based on the relationship between the starting node S and its parent node (which can be empty or itself during initialization), the slope cost slope(S) and roll cost roll(S) are calculated. Finally, the cost value of the starting node S is calculated. After that, it enters the main loop, takes out the node with the minimum cost value f(n) from the open list (the first node taken out is the starting node S), denoted as the current node C, and moves the current node C from the open list to the closed list; then performs neighborhood expansion, traverses all valid neighborhood nodes N of the current node C in the three-dimensional space, and calculates the cost values of each valid neighborhood node to add each valid neighborhood node to the open list for search.

[0207] For a valid neighborhood node N, calculate the comprehensive cost of movement: cost(C, N) = distance(C, N)+α·slope(C->N) +β·roll(C->N). Among them, distance(C,N) represents the actual cumulative cost from the current node C to the valid neighborhood node N, slope(C->N) represents the slope cost from the current node C to the valid neighborhood node N, and roll(C->N) represents the roll cost from the current node C to the valid neighborhood node N; then calculate the tentative cumulative cost: g_tentative = g(C) + cost(C, N). If the valid neighborhood node N is not in the open list, it can be added to the open list, and set the actual cumulative cost g(N) of the valid neighborhood node N = g_tentative, and calculate the heuristic value h(N) of the valid neighborhood node N, and then calculate the cost value f(N) = g(N) + h(N) + α·slope(N) + β·roll(N) of the valid neighborhood node N, and record the parent node of the valid neighborhood node N as the current node C. If the valid neighborhood node N is already in the open list and g_tentative<g(N), it means that a better path to the valid neighborhood node N is found, then the actual cumulative cost g(N) of the valid neighborhood node N can be updated = g_tentative, and the cost value f(N) of the valid neighborhood node N is recalculated according to the new actual cumulative cost g(N), and the parent node of the valid neighborhood node N is reset to the current node C. If the valid neighborhood node N is already in the open list and g_tentative≥g(N), no operation is performed.

[0208] After traversing all valid neighboring nodes corresponding to the current node C according to the aforementioned process, the process can continue by selecting the node with the smallest cost f(n) from the open list and adding it to the closed list. This process of traversing valid neighboring nodes is repeated until the target node G is removed from the open list. From the target node G, the pointer of its parent node is continuously tracked until it backtracks to the starting node S. The sequence of nodes traversed constitutes the optimal or suboptimal path from the starting node S to the target node G. Converting this node sequence into a series of continuous three-dimensional coordinates yields the robot's real-time navigation path within the current local update range. This real-time navigation path is not only geometrically short but also fully considers terrain accessibility and trajectory smoothness, representing a planning result that highly unifies safety, efficiency, and feasibility in a dynamic three-dimensional environment.

[0209] The 3D real-time navigation method for robots provided in this application performs real-time path planning based on an Octomap map. It dynamically adjusts the local update range of the Octomap map according to the robot's real-time operation, and dynamically adjusts the attenuation coefficient of each 3D grid within the local update range based on obstacle types. Then, within the updated Octomap map, it performs real-time local path planning by introducing an incremental 3D A* algorithm that integrates slope and rolling costs. Therefore, it not only provides complete 3D spatial information through the Octomap map and dynamically adjusts the map update strategy according to robot speed and obstacle semantics, but also ensures the smoothness, motion feasibility, and planning efficiency of the generated path by explicitly incorporating kinematic constraints and historical path inheritance mechanisms in path search. This further improves the reliability and safety of robot 3D spatial navigation, enhances the computational efficiency of real-time navigation, and improves the path rationality of real-time path navigation.

[0210] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0211] Corresponding to the three-dimensional real-time navigation method for robots described in the above embodiments, Figure 5 A schematic diagram of the structure of a three-dimensional real-time navigation device for a robot provided in an embodiment of this application is shown. For ease of explanation, only the parts related to the embodiment of this application are shown.

[0212] Reference Figure 5 The robot's three-dimensional real-time navigation device 50 includes:

[0213] The first acquisition module 51 is used to acquire the current three-dimensional point cloud data collected by the lidar in the robot, the current depth image collected by the depth camera, and the current linear velocity of the robot.

[0214] The first determining module 52 is used to determine the local update range of the Octomap information of the robot's running scene based on the current linear velocity and the current depth image. The Octomap information of the running scene contains information of multiple three-dimensional grids.

[0215] The second determining module 53 is used to determine the observation occupancy probability and the attenuation coefficient of each three-dimensional grid to be updated based on the local update range, the current three-dimensional point cloud data and the current depth image, wherein each three-dimensional grid to be updated is located within the local update range.

[0216] The first update module 54 is used to update the occupancy probability of each three-dimensional grid to be updated according to the observation occupancy probability and the attenuation coefficient of each three-dimensional grid to be updated, so as to generate updated running scene Octomap information.

[0217] The third determining module 55 is used to determine the robot's real-time navigation path within the local update range based on the updated Octomap information of the running scene and the A* search algorithm.

[0218] The 3D real-time navigation device for robots provided in this application performs real-time path planning based on an Octomap map. It dynamically adjusts the local update range of the Octomap map according to the robot's real-time operation, and dynamically adjusts the attenuation coefficient of each 3D grid within the local update range based on obstacle types. This allows for real-time local path planning within the updated Octomap map. Therefore, by providing 3D spatial information through the Octomap map and dynamically adjusting the update range and attenuation coefficients of the 3D grids based on the robot's speed and obstacle information during actual operation, the reliability and safety of the robot's 3D spatial navigation are improved. Furthermore, by locally updating the Octomap map, the computational efficiency of real-time navigation is enhanced.

[0219] In one possible implementation of this application, the first determining module 52 includes:

[0220] The first acquisition unit is used to acquire the robot's minimum update radius, dynamic obstacle prediction time window, and anti-collision safety buffer radius.

[0221] The first determining unit is used to determine the base radius corresponding to the local update range based on the robot's minimum update radius, dynamic obstacle prediction time window, anti-collision safety buffer radius, and current linear velocity.

[0222] The second determining unit is used to determine the type of obstacle contained in the current depth image within the base radius based on the base radius and the current depth image;

[0223] The third determining unit is used to determine the semantic radius corresponding to the local update range based on the types of obstacles contained in the current depth image within the base radius.

[0224] The fourth determining unit is used to determine the local update range based on the base radius and semantic radius.

[0225] Furthermore, in another possible implementation of this application, the second determining module 53 mentioned above includes:

[0226] The fifth determining unit is used to voxelize the current 3D point cloud data within the local update range in order to determine each 3D raster to be updated.

[0227] The sixth determining unit is used to determine the initial occupancy probability and the observation occupancy probability for each 3D grid to be updated;

[0228] The seventh determining unit is used to perform target detection on the current depth image within the local update range to determine the obstacle type and confidence level corresponding to each 3D raster to be updated;

[0229] The eighth determining unit is used to determine the attenuation coefficient corresponding to each three-dimensional grid to be updated based on the obstacle type and confidence level of each grid to be updated, and to correct the initial occupancy probability of each grid to be updated.

[0230] Furthermore, in another possible implementation of this application, the aforementioned eighth determining unit is specifically used for:

[0231] If the obstacle type corresponding to the first three-dimensional grid to be updated is a dynamic obstacle and the confidence level is greater than the confidence level threshold, the initial occupancy probability corresponding to the first three-dimensional grid to be updated is corrected to the first occupancy probability value, and the attenuation coefficient corresponding to the first three-dimensional grid to be updated is determined as the first attenuation value, wherein the first three-dimensional grid to be updated is any three-dimensional grid to be updated.

[0232] When the obstacle type corresponding to the first 3D grid to be updated is a transparent obstacle, the initial occupancy probability corresponding to the first 3D grid to be updated is corrected to the second occupancy probability value, and the attenuation coefficient corresponding to the first 3D grid to be updated is determined as the second attenuation value, wherein the second occupancy probability value is less than the first occupancy probability value, and the second attenuation value is less than the first attenuation value.

[0233] Furthermore, in another possible implementation of this application, the updated runtime scene Octomap information includes the updated current occupancy probability of each 3D grid; correspondingly, the third determining module 55 includes:

[0234] The ninth determining unit is used to determine each obstacle node and each non-obstacle node within the local update range based on the current occupied probability and idle probability threshold of each three-dimensional grid.

[0235] The tenth determining unit is used to determine the starting node and target node of the robot within the local update range based on the robot's corresponding historical navigation path, each obstacle node and each non-obstacle node;

[0236] The eleventh determining unit is used to search for each non-obstacle node according to the A* search algorithm and a preset cost function in order to determine the real-time navigation path between the starting node and the target node. The preset cost function is determined based on the actual cumulative cost function, heuristic function, slope cost function and rolling cost function.

[0237] Furthermore, in yet another possible implementation of this application, the starting node is the last unblocked node in the historical navigation path within the scope of the local update.

[0238] Furthermore, in another possible implementation of this application, when searching for the starting node using the A* search algorithm, the actual cumulative cost of the starting node is the actual cumulative cost of the last unblocked node.

[0239] Furthermore, in another possible implementation of this application, when searching for the effective neighboring nodes corresponding to the current node in the closed list according to the A* search algorithm and the preset cost function, the slope cost of the effective neighboring nodes is determined in the following way:

[0240] Determine the height difference and horizontal distance between the current node and its effective neighboring nodes. The effective neighboring nodes are non-obstacle nodes that are adjacent to the current node and are not in the closed list.

[0241] The slope cost of effective neighboring nodes is determined based on the height difference, horizontal distance, and slope cost function.

[0242] Furthermore, in another possible implementation of this application, when searching for the effective neighboring nodes corresponding to the current node in the closed list according to the A* search algorithm and the preset cost function, the rolling cost of the effective neighboring nodes is determined in the following way:

[0243] Determine the change in heading angle of the robot from the current node to the effective neighboring nodes, where the effective neighboring nodes are non-obstacle nodes that are adjacent to the current node and are not in the closed list;

[0244] The rolling cost of effective neighboring nodes is determined based on the change in heading angle and the rolling cost function.

[0245] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0246] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0247] To implement the above embodiments, this application also proposes an electronic device.

[0248] Figure 6 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application.

[0249] like Figure 6 As shown, the above-mentioned electronic device 200 includes:

[0250] The system includes a memory 210 and at least one processor 220, and a bus 230 connecting the different components (including the memory 210 and the processor 220). The memory 210 stores a computer program, which, when executed by the processor 220, implements the three-dimensional real-time navigation method for the robot described in this application embodiment.

[0251] Bus 230 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. For example, these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0252] Electronic device 200 typically includes a variety of electronic device readable media. These media can be any available media that can be accessed by electronic device 200, including volatile and non-volatile media, removable and non-removable media.

[0253] Memory 210 may also include computer system readable media in the form of volatile memory, such as random access memory (RAM) 240 and / or cache memory 250. Electronic device 200 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 260 may be used to read and write non-removable, non-volatile magnetic media (… Figure 6 Not shown; usually referred to as a "hard drive"). Although Figure 6 As not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 230 via one or more data media interfaces. Memory 210 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of this application.

[0254] A program / utility 280 having a set (at least one) of program modules 270 may be stored in, for example, memory 210. Such program modules 270 include—but are not limited to—an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 270 typically perform the functions and / or methods described in the embodiments of this application.

[0255] Electronic device 200 can also communicate with one or more external devices 290 (e.g., keyboard, pointing device, display 291, etc.), and with one or more devices that enable a user to interact with electronic device 200, and / or with any device that enables electronic device 200 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed via input / output (I / O) interface 292. Furthermore, electronic device 200 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 293. As shown, network adapter 293 communicates with other modules of electronic device 200 via bus 230. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 200, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0256] The processor 220 performs various functional applications and data processing by running programs stored in the memory 210.

[0257] It should be noted that the implementation process and technical principles of the electronic device in this embodiment are explained in the foregoing description of the three-dimensional real-time navigation method of the robot in the embodiments of this application, and will not be repeated here.

[0258] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps described in the various method embodiments above.

[0259] This application provides a computer program product that, when run on an electronic device, enables the electronic device to perform the steps described in the various method embodiments above.

[0260] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a device / electronic device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks.

[0261] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0262] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0263] In the embodiments provided in this application, it should be understood that the disclosed devices / electronic devices and methods can be implemented in other ways. For example, the device / electronic device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings or direct couplings or communication connections may be through some interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0264] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0265] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A three-dimensional real-time navigation method for a robot, characterized in that, include: The robot acquires the current 3D point cloud data collected by the lidar, the current depth image collected by the depth camera, and the current linear velocity of the robot. Based on the current linear velocity and the current depth image, the local update range of the octomap information of the running scene corresponding to the robot is determined, wherein the octomap information of the running scene contains information of multiple three-dimensional grids. Based on the local update range, the current 3D point cloud data, and the current depth image, determine the observation occupancy probability and the attenuation coefficient corresponding to each 3D grid to be updated, wherein each 3D grid to be updated is located within the local update range. Based on the observation occupancy probability and the attenuation coefficient of each of the three-dimensional grids to be updated, the occupancy probability of each of the three-dimensional grids to be updated is updated to generate updated Octomap information of the running scene. Based on the updated Octomap information of the operating scenario and the A* search algorithm, the real-time navigation path of the robot within the local update range is determined; The step of determining the observation occupancy probability and the attenuation coefficient corresponding to each 3D raster to be updated based on the local update range, the current 3D point cloud data, and the current depth image includes: The current 3D point cloud data within the local update range is voxelized to determine each of the 3D raster cells to be updated. Determine the initial occupancy probability and the observation occupancy probability for each of the three-dimensional grids to be updated; Target detection is performed on the current depth image within the local update range to determine the obstacle type and confidence level corresponding to each of the three-dimensional grids to be updated; Based on the obstacle type and confidence level of each of the three-dimensional grids to be updated, the attenuation coefficient of each of the three-dimensional grids to be updated is determined, and the initial occupancy probability of each of the three-dimensional grids to be updated is corrected. The step of determining the attenuation coefficient for each of the three-dimensional grids to be updated based on the obstacle type and confidence level, and correcting the initial occupancy probability for each of the three-dimensional grids to be updated, includes: If the obstacle type corresponding to the first three-dimensional grid to be updated is a dynamic obstacle and the confidence level is greater than the confidence threshold, the initial occupancy probability corresponding to the first three-dimensional grid to be updated is corrected to a first occupancy probability value, and the attenuation coefficient corresponding to the first three-dimensional grid to be updated is determined as a first attenuation value, wherein the first three-dimensional grid to be updated is any of the three-dimensional grids to be updated. When the obstacle type corresponding to the first three-dimensional grid to be updated is a transparent obstacle, the initial occupancy probability corresponding to the first three-dimensional grid to be updated is corrected to a second occupancy probability value, and the attenuation coefficient corresponding to the first three-dimensional grid to be updated is determined as a second attenuation value, wherein the second occupancy probability value is less than the first occupancy probability value, and the second attenuation value is less than the first attenuation value.

2. The method as described in claim 1, characterized in that, The step of determining the local update range of the Octomap information corresponding to the robot's operating scene based on the current linear velocity and the current depth image includes: Obtain the minimum update radius, dynamic obstacle prediction time window, and collision avoidance safety buffer radius corresponding to the robot; The base radius corresponding to the local update range is determined based on the minimum update radius of the robot, the dynamic obstacle prediction time window, the anti-collision safety buffer radius, and the current linear velocity. Based on the base radius and the current depth image, determine the types of obstacles included in the current depth image within the base radius; Based on the types of obstacles contained within the base radius in the current depth image, determine the semantic radius corresponding to the local update range; The local update range is determined based on the base radius and the semantic radius.

3. The method as described in any one of claims 1-2, characterized in that, The updated octomap information of the operating scene includes the current occupancy probability of each of the three-dimensional grids after the update. Determining the real-time navigation path of the robot within the local update range based on the updated octomap information and the A* search algorithm includes: Based on the current occupied probability and idle probability threshold corresponding to each of the three-dimensional grids, each obstacle node and each non-obstacle node within the local update range are determined. Based on the robot's historical navigation path, each obstacle node, and each non-obstacle node, the starting node and target node of the robot within the local update range are determined; According to the A* search algorithm and the preset cost function, each of the non-obstacle nodes is searched to determine the real-time navigation path between the starting node and the target node. The preset cost function is determined based on the actual cumulative cost function, the heuristic function, the slope cost function, and the rolling cost function.

4. The method as described in claim 3, characterized in that, The starting node is the last unblocked node of the historical navigation path within the scope of the local update.

5. The method as described in claim 4, characterized in that, When searching the starting node using the A* search algorithm, the actual cumulative cost of the starting node is the actual cumulative cost of the last unblocked node.

6. The method as described in claim 3, characterized in that, According to the A* search algorithm and the preset cost function, when searching for the effective neighboring nodes corresponding to the current node being searched that is in the closed list, the slope cost of the effective neighboring nodes is determined in the following way: Determine the height difference and horizontal distance between the current node and the effective neighboring nodes, wherein the effective neighboring nodes are non-obstacle nodes that are adjacent to the current node and are not in the closed list; The slope cost of the effective neighboring node is determined based on the height difference, the horizontal distance, and the slope cost function.

7. The method as described in claim 3, characterized in that, According to the A* search algorithm and the preset cost function, when searching for the effective neighboring nodes corresponding to the current node being searched in the closed list, the rolling cost of the effective neighboring nodes is determined in the following way: Determine the change in heading angle of the robot from the current node to the effective neighboring node, wherein the effective neighboring node is a non-obstacle node that is adjacent to the current node and is not in the closed list; The rolling cost of the effective neighboring node is determined based on the change in heading angle and the rolling cost function.

8. A three-dimensional real-time navigation device for a robot, characterized in that, include: The first acquisition module is used to acquire the current three-dimensional point cloud data collected by the lidar in the robot, the current depth image collected by the depth camera, and the current linear velocity of the robot. The first determining module is used to determine the local update range of the Octomap information of the running scene corresponding to the robot based on the current linear velocity and the current depth image, wherein the Octomap information of the running scene contains information of multiple three-dimensional grids; The second determining module is used to determine the observation occupancy probability and the attenuation coefficient of each three-dimensional grid to be updated based on the local update range, the current three-dimensional point cloud data and the current depth image, wherein each three-dimensional grid to be updated is located within the local update range. The first update module is used to update the occupancy probability of each of the three-dimensional grids to be updated according to the observation occupancy probability and the attenuation coefficient of each of the three-dimensional grids to be updated, so as to generate updated running scene Octomap information. The third determining module is used to determine the real-time navigation path of the robot within the local update range based on the updated Octomap information of the running scene and the A* search algorithm; The second determining module includes: The fifth determining unit is used to voxelize the current three-dimensional point cloud data within the local update range to determine each of the three-dimensional grids to be updated. The sixth determining unit is used to determine the initial occupancy probability and the observation occupancy probability for each of the three-dimensional grids to be updated; The seventh determining unit is used to perform target detection on the current depth image within the local update range to determine the obstacle type and confidence level corresponding to each of the three-dimensional grids to be updated; The eighth determining unit is used to determine the attenuation coefficient corresponding to each of the three-dimensional grids to be updated based on the obstacle type and confidence level of each grid, and to correct the initial occupancy probability of each grid. The eighth determining unit is specifically used for: If the obstacle type corresponding to the first three-dimensional grid to be updated is a dynamic obstacle and the confidence level is greater than the confidence threshold, the initial occupancy probability corresponding to the first three-dimensional grid to be updated is corrected to a first occupancy probability value, and the attenuation coefficient corresponding to the first three-dimensional grid to be updated is determined as a first attenuation value, wherein the first three-dimensional grid to be updated is any of the three-dimensional grids to be updated. When the obstacle type corresponding to the first three-dimensional grid to be updated is a transparent obstacle, the initial occupancy probability corresponding to the first three-dimensional grid to be updated is corrected to a second occupancy probability value, and the attenuation coefficient corresponding to the first three-dimensional grid to be updated is determined as a second attenuation value, wherein the second occupancy probability value is less than the first occupancy probability value, and the second attenuation value is less than the first attenuation value.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1-7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-7.

11. A computer program product, characterized in that, Includes a computer program that, when run on an electronic device, causes the electronic device to perform the method as described in any one of claims 1-7.

Images