A map-free navigation method based on global grid memory and access heat

Abstract

The present application relates to the technical field of autonomous navigation of unmanned cleaning vehicles, and particularly relates to a map-free navigation method based on global grid memory and access heat, which first establishes and continuously updates a global grid memory map under a global coordinate system, writes access heat information of a local map of an unmanned cleaning vehicle and local environment information obtained by a laser radar into the global grid memory map, and determines a guide direction and generates a sub-target by a decision network in combination with access heat information and obstacle distance information corresponding to a candidate direction, so as to realize segmented navigation. The present application overcomes the defects of existing map-free navigation methods, which mainly rely on current local observation and lack continuous use of historical exploration information, and effectively alleviates the problem of repeated exploration in a complex environment and falling into a local optimum.

Description

Technical Field

[0001] This invention relates to the field of autonomous navigation technology for unmanned cleaning vehicles, and in particular to a mapless navigation method based on global grid memory and access popularity. Background Technology

[0002] In dynamically changing scenarios, the environment is complex and changes frequently, making it difficult to pre-construct accurate, complete, and long-term effective environmental maps. Traditional navigation methods that rely on prior maps (using SLAM algorithms to first create a complete map, and then using map and LiDAR data to plan a path for navigation) are significantly limited. Unmanned cleaning vehicles need to complete autonomous navigation solely based on real-time observations from sensors such as LiDAR in the absence of prior maps. Therefore, navigation without prior maps has significant engineering application value.

[0003] Existing map-free navigation methods mainly include local rule-based methods (such as the bug algorithm) and learning-based methods. Local rule-based methods typically perform obstacle avoidance and direction adjustment based on current sensor observations. These methods are relatively simple to implement, but their adaptability to environmental structures is limited. In recent years, map-free navigation methods based on deep reinforcement learning have gradually become the main research direction in this field. These methods typically take local environmental observations, target information, and the state of the unmanned cleaning vehicle as input, and directly output navigation direction or control commands through a decision network. They exhibit strong adaptability and a certain degree of generalization ability in complex environments, and have become an important development direction for map-free navigation.

[0004] However, existing map-free navigation methods still have significant shortcomings. Many methods rely primarily on local observations at the current moment for decision-making, lacking utilization of historical information and spatial structure, resulting in insufficient feature extraction. Unmanned cleaning vehicles struggle to distinguish between unexplored areas and areas that have been repeatedly traversed, and are prone to repeated exploration in scenarios such as concave obstacles, U-shaped passages, long corridors, and dead ends. Summary of the Invention

[0005] To overcome the problems of existing mapless navigation methods that rely mainly on current local observations, lack continuous utilization of historical exploration information, and are prone to repeated exploration and getting trapped in local optima in complex environments, this invention proposes a mapless navigation method based on global grid memory and access popularity. This method can continuously accumulate environmental information without relying on prior maps, enabling unmanned cleaning vehicle navigation in unknown environments.

[0006] This invention proposes a mapless navigation method based on global grid memory and access popularity. First, a global map of the environment is constructed in a global coordinate system and rasterized to obtain a global grid memory map and initialize it. The grid status values ​​are updated on the global grid memory map based on the location of the unmanned cleaning vehicle and radar detection data. The grid status values ​​include obstacle status, passability status, and access popularity. The obstacle status and passability status are obtained by parsing the radar detection data. A local map is cropped from the global raster memory map to obtain the local map state, which is labeled with the state values ​​of each raster. The local map corresponds to the bounded rectangular area of ​​the unmanned cleaning vehicle, centered at its center and with a set size. (Local map state) Used to characterize the obstacle status, passability status, and access popularity of each grid within the coverage area; Construct a coordinate system for the unmanned cleaning vehicle as a reference to determine the relative pose information of the target. It is used to characterize the target position in the coordinate system of the unmanned cleaning vehicle, as well as the relative distance and relative azimuth between the target and the unmanned cleaning vehicle; By combining the obstacle status of the original directions, candidate directions that the unmanned cleaning vehicle can pass through are screened, the access popularity of each candidate direction is determined, and the information is summarized to form candidate direction popularity information. ; Let the decision network be based on the local map state Target relative pose information Information on the popularity of candidate directions Generate sub-targets for navigation at the current time step; Update and execute sub-goals until the unmanned cleaning vehicle reaches the target.

[0007] Preferably, the decision network also considers the obstacle distances in each candidate direction during the process of generating sub-objectives.

[0008] Preferably, the raster state value update method for the global raster memory map is as follows: The system tracks the current position, heading angle, and target position of the unmanned cleaning vehicle in real time, along with radar detection data. The grid corresponding to the center point of the unmanned cleaning vehicle on the global grid memory map is used as the center grid, and the grids adjacent to the center grid are recorded as neighboring grids. At each time step, the access popularity of the center grid is updated to the sum of the current access popularity and the set center popularity value. The access popularity of the neighboring grids is also updated to the sum of the current access popularity and the set neighbor popularity value. The set center popularity value is greater than the set neighbor popularity value.

[0009] Preferably, the grid status values ​​include obstacle-occupied channels, free space channels, access popularity channels, and unknown area channels; The obstacle-occupied passage uses 01 state values ​​to represent the obstacle state; the free space passage uses 01 state values ​​to represent the passable state; the access popularity passage represents the normalized access popularity; and the unknown area passage is used to introduce boundary values. The pixel in row u and column v on the local map is denoted as (u, v), and its unknown area channel... for: ; In the formula, and These represent the obstacle status and passable status of the grid cell containing the pixel (u,v) on the local map, respectively. This represents the set of pixels in a local map.

[0010] Preferably, the access popularity of candidate directions is obtained through the following steps: S31. Select original division directions whose obstacle distance is greater than the minimum safe distance threshold as candidate directions; S32. Determine the candidate sub-targets for each candidate direction. The candidate sub-targets are used to characterize the locations that the unmanned cleaning vehicle can reach in the corresponding candidate direction. S33. Take the access popularity corresponding to the candidate sub-target location as the access popularity of the candidate direction.

[0011] Preferably, the global coordinates of candidate sub-targets in a specified candidate direction are obtained as follows: First, obtain the relative distance between the candidate sub-targets and the unmanned cleaning vehicle.

[0012] ; in, Let be the relative distance between the k-th candidate sub-target and the unmanned cleaning vehicle, and let clip be the clipping function. This is the maximum range of the radar. For safe collision avoidance distance, The minimum relative distance is set; min is the function that takes the minimum value. The furthest distance for the set sub-target; The distance to the target is the distance to the unmanned cleaning vehicle; To allow for a distance margin exceeding the target; By combining the azimuth of the candidate direction and the relative distance between the candidate sub-target and the unmanned cleaning vehicle, the position of the candidate sub-target in the coordinate system of the unmanned cleaning vehicle is obtained, and this position is mapped to the global coordinate system to obtain the global coordinates of the candidate sub-target.

[0013] Preferably, the decision network employs a proximal policy optimization algorithm.

[0014] Preferably, the reward function used for training the decision network includes at least the goal progress reward. and duplicate access penalty ; ; ; in, For the schedule coefficient, The breadth-first search distance at time step t. The breadth-first search distance at time step t-1. The popularity index is the number of visits. The set positive threshold number; Normalized heat of the corresponding grid cell for the sub-target on the global grid memory map.

[0015] The present invention proposes a mapless navigation system based on global grid memory and access popularity, including a processor and a planner. The processor is used to execute the mapless navigation method based on global grid memory and access popularity at each time step to obtain the sub-target at the current time step. The planner combines the coordinate system of the unmanned cleaning vehicle to convert the sub-target into the motion control quantity of the unmanned cleaning vehicle and sends it to the unmanned cleaning vehicle for execution.

[0016] The present invention proposes a storage medium storing a computer program, which, when executed, is used to implement the mapless navigation method based on global grid memory and access popularity.

[0017] The advantages of this invention are: (1) The present invention proposes a mapless navigation method based on global grid memory and access heat. First, a global grid memory map is established and continuously updated in a global coordinate system. Access heat information of the vicinity of the current location of the unmanned cleaning vehicle (i.e., the local map) and local environmental information obtained by lidar observation are written into the global grid memory map. Then, the decision network combines the access heat information and obstacle distance information corresponding to the candidate directions to determine the guidance direction and generate sub-targets, thereby realizing segmented navigation. The present invention can continuously accumulate environmental information without relying on prior maps, thereby enhancing the unmanned cleaning vehicle's memory ability and spatial feature extraction ability of the explored area.

[0018] (2) In the direction decision-making process, the present invention takes into account both the suppression of repeated exploration and safety constraints, thereby improving the ability of unmanned cleaning vehicles to alleviate local optimal problems and the overall navigation success rate.

[0019] (3) By establishing and continuously updating a global grid memory map in a global coordinate system, the present invention writes the access heat information near the current location of the unmanned cleaning vehicle and the local environmental information obtained by lidar observation into the global grid memory map. Compared with the no-priority map navigation method that only relies on local observation at the current time step to make decisions, it can continuously accumulate environmental information and historical access information, and enhance the unmanned cleaning vehicle's memory of the explored area.

[0020] (4) By constructing a state space containing local map, target relative pose information and candidate direction heat information, and using a decision network to output guiding direction, the unmanned cleaning vehicle can simultaneously consider the environmental spatial structure and the degree of repeated exploration during the direction decision process. Through the constraint based on candidate direction heat information and the direction shielding mechanism based on obstacle distance, the unmanned cleaning vehicle can be prevented from repeatedly entering the explored area and the probability of choosing dangerous direction can be reduced, thereby improving navigation safety and navigation success rate.

[0021] (5) The present invention adopts a hierarchical navigation structure that combines decision network and planner. While maintaining the stability of the underlying execution, it enhances the direction decision-making ability in complex and unknown environments and improves the ability of unmanned cleaning vehicles to alleviate local optimal problems. Attached Figure Description

[0022] Figure 1 This is a flowchart of a mapless navigation method based on global grid memory and access popularity proposed in this invention; Figure 2 This is an overall architecture diagram of the mapless navigation method based on global grid memory and access popularity in an embodiment of the present invention; Figure 3 This is a success rate trend graph of the training process in an embodiment of the present invention, where k represents 10. 3 . Detailed Implementation

[0023] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0024] like Figure 1 , Figure 2 As shown, this embodiment provides a mapless navigation method based on global grid memory and access heat, suitable for unmanned cleaning vehicles that rely solely on real-time observations from sensors such as LiDAR to complete navigation tasks in unknown environments. This method aims to address the problems of existing mapless navigation methods that primarily rely on current local observations, lack continuous utilization of historical exploration information, and are prone to repeated explorations and getting trapped in local optima in complex environments.

[0025] Please see Figures 1 to 3 The hierarchical, prior-free map navigation method includes steps S1 to S5, namely, global grid memory map initialization and update, local map clipping and state space construction, decision network reasoning, candidate direction selection and sub-target generation, and low-level control execution.

[0026] S1. Obtain the current position, current heading angle, target position information and real-time lidar observation data of the unmanned cleaning vehicle, and initialize the global grid memory map, which is a gridded environment map.

[0027] Step S1 is used to complete the acquisition of initial navigation information and the establishment of the global memory structure. At the beginning of each control cycle, the system first acquires the current position, current heading angle, and target position information of the unmanned cleaning vehicle, and simultaneously acquires real-time lidar observation data.

[0028] Specifically, this step establishes a global coordinate system {W} and an unmanned cleaning vehicle body coordinate system {B}. The global coordinate system {W} is relatively stationary with respect to the unmanned cleaning vehicle's operating environment, and the unmanned cleaning vehicle body coordinate system {B} is relatively stationary with respect to the unmanned cleaning vehicle. A mapping relationship is then constructed from the unmanned cleaning vehicle body coordinate system {B} to the global coordinate system {W}. Then, using an odometry, inertial measurement unit, or other pose estimation module, the current pose of the unmanned cleaning vehicle in the global coordinate system {W} is obtained. .

[0029] ; in, and These are the x and y coordinates of the current position of the unmanned cleaning vehicle in the global coordinate system. The current heading angle of the unmanned cleaning vehicle.

[0030] Based on this, a global grid memory map is established in the global coordinate system based on the pose information of the unmanned cleaning vehicle. The global grid memory map includes at least an obstacle occupancy layer, a free space layer, and an access heat layer. The obstacle occupancy layer represents known obstacle areas, the free space layer represents known passable areas, and the access heat layer represents historical access information. During initialization, the grid values ​​of the obstacle occupancy layer, the free space layer, and the access heat layer are all set to zero to indicate that no valid environmental memory has been established yet.

[0031] Subsequently, the obstacle status of each grid on the global grid memory map will be updated based on the movement of the unmanned cleaning vehicle and radar detection. Passable status and visit popularity , (r,c) represents the r-th row and c-th column of the global raster memory map.

[0032] S2. Update the access heat information near the current location of the unmanned cleaning vehicle, and write the access heat information, free space information and obstacle information of the local area around the unmanned cleaning vehicle obtained from the current lidar observation into the corresponding area of ​​the global grid memory map.

[0033] The area near the current location of the unmanned cleaning vehicle refers to the local map centered on the vehicle (hereinafter referred to as the local map). The local map corresponds to the bounded rectangular area of ​​the unmanned cleaning vehicle with a set size, centered on its central location. The corresponding area of ​​the local map on the global raster memory map is denoted as Region A.

[0034] Step S2 is used to continuously update the global grid memory map, so that it gradually accumulates environmental information and historical access information during navigation.

[0035] Step S2 specifically includes the following sub-steps.

[0036] S21. Based on the unmanned cleaning vehicle's operating trajectory, update the access popularity of the central grid and neighboring grids corresponding to the unmanned cleaning vehicle. The central grid is the central grid corresponding to the current position of the unmanned cleaning vehicle on the global grid memory map, that is, the grid where the center point of the unmanned cleaning vehicle is located; the neighboring grids are the grids adjacent to the central grid on the global grid memory map.

[0037] In this step, at each time step, the access popularity of the center grid is updated to the sum of the current access popularity and the set center popularity value; the access popularity of the neighboring grid is updated to the sum of the current access popularity and the set neighbor popularity value; the set center popularity value is greater than the set neighbor popularity value, specifically, the center popularity value can be set to 1 and the neighbor popularity value to 0.5.

[0038] Step S21 implements a global grid memory map that follows the movement trajectory of the unmanned cleaning vehicle, updating in real time the access popularity information of the vicinity of the unmanned cleaning vehicle's current location (i.e., the local map). To facilitate the subsequent mapping of access levels from different areas to a finite interval, the access popularity information can be normalized. The normalization formula is: ; In the formula, For grid The number of visits; The normalized visit popularity, These are the preset normalization coefficients.

[0039] S22. Obtain information about the area surrounding the unmanned cleaning vehicle based on the LiDAR detection data on the vehicle, and analyze the free space information and obstacle information of the area surrounding the vehicle. The free space information is represented by the set of navigable points in each direction around the vehicle, and the obstacle information is represented by the distances to obstacles in each direction around the vehicle.

[0040] Specifically, for the ranging results returned by the lidar in each direction at the current time step, the obstacle distance in each direction can be determined first.

[0041] Obstacle distance in the i-th direction of the unmanned cleaning vehicle and azimuth They are respectively: ; ; in, This represents the maximum range of the lidar. The radar detection wave return distance in the i-th direction is the distance to the obstacle detected by the radar. The current heading angle of the unmanned cleaning vehicle. Let i be the total number of directions for the lidar, 1 ≤ i ≤ .

[0042] Thus, obstacle information can be denoted as {( );1≤i≤ }

[0043] Let all points in the i-th direction of the unmanned cleaning vehicle, from the starting point to the detectable ending point, be denoted as free space update points, and let the free space update point in the i-th direction that is a distance s from the unmanned cleaning vehicle be denoted as ( , ),satisfy: ; ; In the formula, ( , () represents the global coordinates of the unmanned cleaning vehicle. Let represent the obstacle distance in the i-th direction for the unmanned cleaning vehicle.

[0044] When an obstacle is detected, the corresponding endpoint grid is updated to an obstacle-occupied grid, and the obstacle state of the obstacle-occupied grid is updated to... =1; Within the detection range, if a grid cell contains only free space update points, then the walkable state of that grid cell is updated to... =1 indicates that passage is permitted.

[0045] Finally, the access popularity information of the local map { ; }, and free space information obtained from lidar observations { ; } and obstacle information { ; The data is uniformly written into the corresponding area of ​​the global raster memory map. A represents the coverage area of ​​the local map on the global raster memory map. Through the above process, the global raster memory map can be continuously updated as the unmanned cleaning vehicle moves, instead of only reflecting the instantaneous local observation at the current time step.

[0046] S3. A local map is obtained by cropping from the updated global grid memory map, and the local map state and the target relative pose information are used as the state input of the decision network.

[0047] In this embodiment, the decision network processes the local map state respectively. Relative pose information of the target Information on the popularity of candidate directions Encoding is performed, and the encoded results are fused to output the guidance direction, i.e., the running direction of the unmanned cleaning vehicle. This is achieved by analyzing the local map state. Target relative pose information Information on the popularity of candidate directions These factors, acting as inputs, enable the decision network to simultaneously utilize the current local environment structure, target direction information, and historical exploration information to make directional decisions. The decision network can employ any existing network, such as a PPO network.

[0048] Step S3 is used to construct the state inputs for the decision network.

[0049] First, based on the current position and heading angle of the unmanned cleaning vehicle, a local map is cropped from the updated global grid memory map. Specifically, this is a map of the bounding rectangle area of ​​the unmanned cleaning vehicle of a specified size, with the center point of the unmanned cleaning vehicle as the center point. To align the local map with the current orientation of the unmanned cleaning vehicle, the world coordinates (u,v) of any pixel (u,v) in the local map are set to (u,v). , The formula for calculating ) is: ; ; ; ; In the formula, (u,v) represents the row and column offset in the local map, i.e., the pixel in the u-th row and v-th column of the local map; Δ is the raster resolution, i.e., the area of ​​one raster cell; , ) represents the global coordinates of pixel (u,v) in the local map. The current heading angle of the unmanned cleaning vehicle. and This is the set transition coefficient.

[0050] Global coordinates are the coordinates of a specified object on a global raster memory map.

[0051] In this embodiment, the local map includes at least an obstacle occupancy channel, a free space channel, an unknown area channel, and an access popularity channel, such that pixels (u,v) on the local map are located in the grid of the global grid memory map. In the formula, the relevant formulas for pixel (u,v) are as follows: Obstacles occupy the passage: ; Free space passage: ; Popular access channels: ; Unknown area passage: ; In the formula, For grid The obstacle status represents the obstacle occupancy layer value, with 1 indicating the presence of an obstacle and 0 indicating the absence of an obstacle. For grid The passability status represents the free space layer value, with 1 indicating that it is passable and 0 indicating that it is not passable; This represents the normalized access popularity, indicating the numerical value of the access popularity layer. This represents the set of pixels in a local map. The global raster memory map is a bounded map; therefore, access popularity channels are... It also has boundary values.

[0052] In this embodiment, the state space of the decision network for: ; ; ; ; In the formula, Represents the local map state at time step t; This represents the target's relative pose information. Let {B} be the position of the target (i.e., the navigation target) in the coordinate system {B} of the unmanned cleaning vehicle. The relative distance to the target is the relative distance between the target and the unmanned cleaning vehicle at time step t. The relative azimuth angle of the target is the relative azimuth angle between the target and the unmanned cleaning vehicle at time step t. This indicates the popularity information of candidate directions; H(r,c) represents the access popularity of the candidate sub-target in the nth candidate direction at time step t, that is, the access popularity of the grid cell where the global coordinates of the candidate sub-target are located. and These represent the access popularity of candidate sub-targets in the first and second candidate directions at time step t, respectively.

[0053] Candidate sub-targets are phased target points. Each candidate sub-target corresponds to a candidate direction. The candidate direction is a valid direction selected from the original division direction. For example, the area can be divided into 8 original division directions (east, south, west, north, southeast, southwest, northeast, and northwest). Then, candidate directions are selected based on safety margin. Finally, candidate sub-targets for each candidate direction are calculated based on access popularity and obstacle distance information.

[0054] Specifically, candidate sub-targets and their popularity scores. The reasoning process is described in the following steps.

[0055] S31. During the direction filtering process, the original directions that are close to obstacles are masked based on the obstacle distance information corresponding to each original division direction.

[0056] The obstacle distance corresponding to the p-th original division direction is The equivalent radius of the unmanned cleaning vehicle is Safety margin is Then the minimum safe distance threshold is: ; The original directional shielding conditions are: ; ; All original division directions are considered as candidate directions. When none of the original division directions meet the safety conditions, the corresponding obstacle distance is retained. The largest original partition is selected as the candidate direction.

[0057] S32. Calculate the relative distance between the candidate sub-target and the unmanned cleaning vehicle in each candidate direction, as well as the position of the candidate sub-target in the global coordinate system.

[0058] The position of the candidate sub-target along the k-th candidate direction (referred to as the k-th candidate sub-target) in the global coordinate system, i.e., the global coordinates ( , ).

[0059] The relative distance between the k-th candidate sub-target and the unmanned cleaning vehicle is denoted as . ; ; in, The minimum relative distance is set. This is the maximum range of the radar, i.e., the farthest distance the radar can detect. For safe collision avoidance distance (optional maximum radius of unmanned cleaning vehicle); The furthest distance for the set sub-target; The distance to the target is the distance to the unmanned cleaning vehicle; To allow for a distance margin exceeding the target.

[0060] The position of the kth candidate sub-target in the local coordinate system of the unmanned cleaning vehicle , )for: ; ; In the formula, This represents the angle value corresponding to the k-th candidate direction. The position of the kth candidate sub-target in the global coordinate system , )for: ; .

[0061] S33. The access popularity of the region corresponding to the candidate direction is the access popularity of the candidate sub-target location.

[0062] Access popularity information for the k-th candidate direction at time step t The calculation formula is: ; in, Represents global coordinates The access popularity of the grid after normalization.

[0063] S4. Decision network combined with candidate direction popularity information and obstacle distance information { , Determine the guiding direction and generate sub-targets based on the guiding direction; Step S4 is used to complete the high-level direction decision and sub-objective generation.

[0064] Sub-goals are the current stage goals, which can be understood as the target points that need to be reached in the next time step.

[0065] The above methods can simultaneously take into account the suppression of historical repetitive exploration and local security constraints when making directional decisions.

[0066] S5. The planner outputs control quantities based on the sub-goals to drive the unmanned cleaning vehicle to move.

[0067] Step S5 is used to complete the underlying control execution. Based on the relative positional relationship between the sub-target and the current position of the unmanned cleaning vehicle, the planner outputs linear and angular velocity control values ​​and drives the unmanned cleaning vehicle to move. Specifically, the planner can use the MPPI method to output control values ​​based on the sub-target.

[0068] By combining decision networks and planners in a hierarchical manner, the decision network is responsible for guiding directional decisions in complex environments, while the planner is responsible for specific control outputs, thus balancing directional decision-making ability and motion execution stability.

[0069] In this embodiment, the decision network employs PPO (Proximity Policy Optimization), and the training reward function of the decision network includes at least the goal progress reward. Rewards for reaching the destination Collision penalty Time penalty and duplicate access penalty .in: ; ; ; In the formula, For the schedule coefficient, Let t be the BFS (Breadth First Search) distance at time step t. The BFS distance at time step t-1, The popularity index is the number of visits. The set positive threshold number; The position of the sub-target in the coordinate system of the unmanned cleaning vehicle; For sub-targets Normalized heat of the corresponding grid on the global grid memory map.

[0070] , and The value is assigned based on the unmanned cleaning vehicle's execution of the control parameters; for example: After the control quantity is executed, the unmanned cleaning vehicle reaches the sub-target. Give the set reward, for example, 80; otherwise, give 0. After the control quantity is executed, if the unmanned cleaning vehicle collides with an obstacle, then... Set a collision penalty, for example, -100, and 0 for no collision; A fixed time penalty, such as -0.1, is introduced into the reward function. By including a revisit penalty term, the decision network not only focuses on approaching the target during training but also inhibits the unmanned cleaning vehicle from repeatedly entering already explored areas.

[0071] like Figure 3The figure shows the evaluation success rate curve of the method of the present invention during training in a complex environment. The horizontal axis represents the number of training steps, and the vertical axis represents the evaluation success rate. It can be seen that the evaluation success rate generally increases with the increase of the number of training steps, indicating that the decision network can gradually learn effective directional decision-making strategies during the training process. In the early stage of training, because the unmanned cleaning vehicle has not yet fully established its decision-making ability on the environmental structure and historical visited areas, the evaluation success rate is low and fluctuates greatly; in the middle and later stages of training, the evaluation success rate gradually increases and remains at a high level. At approximately 389,844 steps, the training time is approximately 2.785 hours, the evaluation success rate reaches 0.9, and the smoothed success rate is approximately 0.8831, indicating that the method of the present invention has good convergence and high task completion ability.

[0072] Of course, those skilled in the art will recognize that the present invention is not limited to the details of the exemplary embodiments described above, but also includes the same or similar structures that can be implemented in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be considered illustrative and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

[0073] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

[0074] The technologies, shapes, and structures not described in detail in this invention are all known technologies.

Claims

1. A map-free navigation method based on global grid memory and access popularity, characterized in that: First, a global map of the environment is constructed in the global coordinate system and rasterized to obtain a global raster memory map and initialize it; Based on the location of the unmanned cleaning vehicle and radar detection data, the grid status value is updated on the global grid memory map. The grid status value includes obstacle status, passability status and access popularity. Obstacle status and passability status are obtained by analyzing radar detection data; A local map is cropped from the global raster memory map to obtain the local map state, which is labeled with the state values ​​of each raster. The local map corresponds to the bounded rectangular area of ​​the unmanned cleaning vehicle, centered at its center and with a set size. (Local map state) Used to characterize the obstacle status, passability status, and access popularity of each grid within the coverage area; Construct a coordinate system for the unmanned cleaning vehicle as a reference to determine the relative pose information of the target. It is used to characterize the target position in the coordinate system of the unmanned cleaning vehicle, as well as the relative distance and relative azimuth between the target and the unmanned cleaning vehicle; By combining the obstacle status of the original directions, candidate directions that the unmanned cleaning vehicle can pass through are screened, the access popularity of each candidate direction is determined, and the information is summarized to form candidate direction popularity information. ; Let the decision network be based on the local map state Target relative pose information Information on the popularity of candidate directions Generate sub-targets for navigation at the current time step; Update and execute sub-goals until the unmanned cleaning vehicle reaches the target.

2. The mapless navigation method based on global grid memory and access popularity as described in claim 1, characterized in that, The decision network also considers the obstacle distances in each candidate direction during the process of generating sub-objectives.

3. The mapless navigation method based on global grid memory and access popularity as described in claim 1, characterized in that, The raster state value update method for the global raster memory map is as follows: The system tracks the current position, heading angle, and target position of the unmanned cleaning vehicle in real time, along with radar detection data. The grid corresponding to the center point of the unmanned cleaning vehicle on the global grid memory map is used as the center grid, and the grids adjacent to the center grid are recorded as neighboring grids. At each time step, the access popularity of the center grid is updated to the sum of the current access popularity and the set center popularity value. The access popularity of the neighboring grids is also updated to the sum of the current access popularity and the set neighbor popularity value. The set center popularity value is greater than the set neighbor popularity value.

4. The mapless navigation method based on global grid memory and access popularity as described in claim 1, characterized in that, Grid status values ​​include obstacle-occupied channels, free space channels, access popularity channels, and unknown area channels; The obstacle-occupied passage uses 01 state values ​​to represent the obstacle state; the free space passage uses 01 state values ​​to represent the passable state; the access popularity passage represents the normalized access popularity; and the unknown area passage is used to introduce boundary values. The pixel in row u and column v on the local map is denoted as (u, v), and its unknown area channel... for: In the formula, and These represent the obstacle status and passable status of the grid cell containing the pixel (u,v) on the local map, respectively. This represents the set of pixels in a local map.

5. The mapless navigation method based on global grid memory and access popularity as described in claim 1, characterized in that, The popularity of candidate directions is obtained through the following steps: S31. Select original division directions whose obstacle distance is greater than the minimum safe distance threshold as candidate directions; S32. Determine the candidate sub-targets for each candidate direction. The candidate sub-targets are used to characterize the locations that the unmanned cleaning vehicle can reach in the corresponding candidate direction. S33. Take the access popularity corresponding to the candidate sub-target location as the access popularity of the candidate direction.

6. The mapless navigation method based on global grid memory and access popularity as described in claim 5, characterized in that, The global coordinates of candidate sub-targets along a specified candidate direction are obtained as follows: First, obtain the relative distance between the candidate sub-targets and the unmanned cleaning vehicle. in, Let be the relative distance between the k-th candidate sub-target and the unmanned cleaning vehicle, and let clip be the clipping function. This is the maximum range of the radar. For safe collision avoidance distance, The minimum relative distance is set; min is the function that takes the minimum value. The furthest distance for the set sub-target; The distance to the target is the distance to the unmanned cleaning vehicle; To allow for a distance margin exceeding the target; By combining the azimuth of the candidate direction and the relative distance between the candidate sub-target and the unmanned cleaning vehicle, the position of the candidate sub-target in the coordinate system of the unmanned cleaning vehicle is obtained, and this position is mapped to the global coordinate system to obtain the global coordinates of the candidate sub-target.

7. The mapless navigation method based on global grid memory and access popularity as described in claim 1, characterized in that, The decision network employs a proximal policy optimization algorithm.

8. The mapless navigation method based on global grid memory and access popularity as described in claim 7, characterized in that, The reward function used for training the decision network should include at least the goal progress reward. and duplicate access penalty ; in, For the schedule coefficient, The breadth-first search distance at time step t. The breadth-first search distance at time step t-1. The popularity index is the number of visits. The set positive threshold number; Normalized heat of the corresponding grid cell for the sub-target on the global grid memory map.

9. A mapless navigation system based on global grid memory and access popularity, characterized in that, It includes a processor and a planner. The processor is used to execute the mapless navigation method based on global grid memory and access heat as described in any one of claims 1-8 at each time step to obtain the sub-target at the current time step. The planner combines the body coordinate system of the unmanned cleaning vehicle to convert the sub-target into the motion control quantity of the unmanned cleaning vehicle and sends it to the unmanned cleaning vehicle for execution.

10. A storage medium, characterized in that, The system contains a computer program that, when executed, implements the mapless navigation method based on global grid memory and access heat as described in claim 7.

Images