Multi-floor robot physical object target navigation method and system based on hierarchical learning strategy

Abstract

The application discloses a kind of multi-floor robot object navigation method and system based on hierarchical learning strategy, belong to mobile robot autonomous navigation technical field, including perception and map construction module, multi-floor map representation module, strategy training module, hierarchical learning decision module and bottom navigation module, utilize robot RGBD observation, robot pose and object target class are constructed to include current floor and adjacent floor multi-floor map representation, and navigation decision is decomposed into floor switching decision and floor inner sub-target selection decision. Through hierarchical learning strategy, robot can first judge continue to navigate in current floor or switch to adjacent floor, then select specific sub-target in target floor, and by bottom navigation module executes point-to-point navigation or cross-floor navigation, so that improve the exploration efficiency of robot in multi-floor unknown environment, floor switching rationality, target search success rate and navigation process controllability, with good application prospect.

Description

Technical Field

[0001] This invention belongs to the field of autonomous navigation technology for mobile robots, specifically relating to a multi-story robot object target navigation method and system based on a hierarchical learning strategy. Background Technology

[0002] Object navigation is a crucial task in the field of autonomous navigation for mobile robots. It typically refers to a robot autonomously exploring its environment in unknown or partially unknown conditions, based on a given object category or semantic description, and ultimately reaching the vicinity of the target object. This task is widely applied in scenarios such as home service robots, warehouse inspection robots, building inspection robots, and indoor guidance robots. Unlike traditional point-to-point navigation, object navigation usually begins without knowing the exact location of the target object. The robot needs to continuously perceive its environment, build a map, and select an appropriate exploration direction or navigation target based on current observations during its movement.

[0003] In real-world indoor environments, object navigation often occurs not only on a single floor but also in architectural environments containing staircases, corridors, and multiple floors. In such multi-story environments, robots need to determine which boundary regions or semantic candidate regions on the current floor are more worthwhile to explore, and when to switch to the upper or lower floor via the stairs. The coupling between sub-object selection within a floor and the decision-making process between floors gives navigation tasks a distinctly long-term decision-making characteristic. Learning all candidate targets directly as a single global action space leads to problems such as a large action space size, sparse reward feedback, high policy training costs, and difficulty in interpreting navigation decisions.

[0004] Existing robot object navigation methods typically utilize RGB images, depth images, object detection models, visual language models, semantic maps, frontal boundaries, and local navigation strategies. When a target object is detected, the robot navigates based on its position in the image or 3D space; when no target object is detected, the next exploration point is usually selected based on occupancy maps, frontal boundaries, or semantic relevance. In recent years, visual language models and visual semantic models have also been used to determine the relevance between the current observation and the target object, thereby providing semantic guidance for robot exploration.

[0005] However, existing methods still have certain shortcomings. On the one hand, while some modular methods have good interpretability, they often rely on rule-based floor switching or zero-shot reasoning, lacking trainable intra-floor exploration strategies and inter-floor switching strategies, making it difficult to fully adapt to complex multi-floor environments. On the other hand, although end-to-end learning methods can directly map from observations to underlying actions, they usually lack explicit multi-floor map representations and do not clearly distinguish between the two levels of decision-making: "which floor to select" and "which sub-target to select within the selected floor," resulting in insufficient controllability of the multi-floor navigation process.

[0006] Furthermore, multi-story object navigation lacks large-scale, manually labeled supervised data. Relying entirely on online reinforcement learning to directly train the global policy requires extensive environmental interaction, resulting in high time and computational costs. Conversely, relying solely on imitation learning is easily limited by the quality of the data acquisition strategy, making it difficult to further improve policy performance. Therefore, it is necessary to propose a multi-story robot object navigation scheme that can utilize the multi-story structure for policy decomposition, achieve effective initialization through imitation learning, and further optimize through alternating fine-tuning via reinforcement learning. Summary of the Invention

[0007] In view of the above, the purpose of this invention is to provide a multi-floor robot object target navigation method and system based on a hierarchical learning strategy. This method addresses object target navigation tasks in multi-floor unknown environments by constructing a multi-floor map representation including the current floor and adjacent floors using robot RGBD observations, robot pose, and object target categories. Navigation decisions are decomposed into inter-floor switching decisions and intra-floor sub-target selection decisions. Through the hierarchical learning strategy, the robot can first determine whether to continue navigation on the current floor or switch to an adjacent floor, then select a specific sub-target within the target floor, and the underlying navigation module performs point-to-point navigation or cross-floor navigation.

[0008] To achieve the above-mentioned objectives, an embodiment provides a multi-story robot object target navigation system based on a hierarchical learning strategy, comprising: The perception and map building module is used to perceive environmental information and object targets and build a semantic boundary map that combines the exploration value, exploration boundary and object target perception results to represent the correlation between the current observation and the object target, as well as a staircase boundary map. The multi-floor map representation module is used to construct a multi-floor map representation containing the current floor and adjacent floors based on the exploration value map, semantic boundary map, and stair boundary map. The strategy training module is used to pre-train the inter-floor switching strategy and the intra-floor exploration strategy using imitation learning, and then alternately fine-tun the two strategies by freezing one and fine-tuning the other based on reinforcement learning. The hierarchical learning decision module uses a multi-floor map representation to output the target floor selection result through the strategy decision between floors, and combines floor validity mask to suppress floors that are unreachable or have no valid candidate targets; it also outputs the candidate sub-target selection result within the target floor through the strategy decision within the floor, and combines semantic boundary mask to suppress non-candidate areas, enabling the robot to select sub-targets for exploration or target proximity within the target floor. The underlying navigation module is used to perform cross-floor navigation based on the target floor selection result and target navigation based on the candidate sub-target selection result.

[0009] Preferably, the perception and map building module includes: The RGBD information acquisition unit is used to acquire the robot's current RGB image, depth map, and robot pose in an unknown multi-story environment, and to receive the object target category to be navigated. The object target perception unit is used to obtain the detection results corresponding to the object target category in the current RGB image, and combine the depth map and robot pose to project the detection results into the map coordinate system to form a semantic candidate region for target navigation. The exploration value perception unit is used to predict the correlation between the current RGB image and the target object through a visual language model or a visual semantic model, and convert the correlation results into exploration value; Occupation map building unit, which is used to generate the explored area, unknown area and obstacle area of ​​the current floor to form an occupation map; Boundary construction unit, which is used to obtain the boundary between explored and unknown areas based on the occupancy map, and to merge the boundary with semantic candidate areas to obtain a semantic boundary map; A stair boundary building unit is used to generate a stair boundary map to guide the robot to approach the stair when a stair is detected.

[0010] Preferably, the multi-floor map representation module includes: The current floor map representation unit is used to represent the current floor map, including at least the semantic boundary map, exploration value map, and robot state map at the current moment, and the semantic boundary map, exploration value map, and action map at the previous moment. The adjacent floor map representation unit is used to generate the corresponding adjacent floor map when there is a staircase boundary leading to the upper or lower floor, and to anchor the robot state in the adjacent floor map to the staircase connection position. The adjacent floor map includes the upper floor map and the lower floor map. A multi-level map stacking unit is used to stack the map of the lower level, the map of the current level, and the map of the upper level to obtain a multi-level map representation.

[0011] Preferably, the hierarchical learning decision module includes a floor-to-floor switching strategy unit, which is used to input the multi-floor map representation into the floor-to-floor strategy network and output a three-dimensional logit vector corresponding to the next floor, the current floor, and the previous floor. Generate floor validity masks based on the validity of each floor map: when there is no valid candidate region in the semantic boundary map of the current floor, set the mask value corresponding to the current floor to invalid; when there is no valid robot state anchor point or no valid semantic boundary candidate region in the upper or lower floor, set the mask value of the corresponding adjacent floor to invalid. The floor validity mask is used to retain the logit value of valid floors and suppress the logit value of invalid floors. After normalization, the floor with the highest probability is selected as the target floor.

[0012] Preferably, the hierarchical learning decision module further includes an in-floor exploration strategy unit, which is used to input the current floor map corresponding to the selected target floor into the in-floor strategy network and output a candidate sub-target logit map corresponding to the map grid. Generate a binary candidate mask as the semantic boundary mask based on the semantic boundary map in the current floor map: mark the non-zero positions in the semantic boundary map as valid candidate regions, and mark other positions as invalid regions; Semantic boundary masks are used to retain the logit values ​​of valid candidate regions and suppress the logit values ​​of invalid regions. The logit map after masking is normalized to obtain the probability map of candidate sub-targets, and the grid position with the highest probability is taken as the sub-target within the target floor.

[0013] Preferably, the floor-to-floor exploration strategy is executed once at a first preset time interval, and the floor-to-floor switching strategy is executed once at a second preset time interval, wherein the second preset time interval is greater than the first preset time interval, so that the update frequency of the floor-to-floor switching decision is lower than the update frequency of the floor-to-floor sub-target decision.

[0014] Preferably, the strategy training module includes an imitation learning pre-training unit, which is used to perform imitation learning-based pre-training on the inter-floor switching strategy and the intra-floor exploration strategy. For the floor switching strategy, the floor labels used as supervision data are generated by the greedy floor switching rule. The greedy floor switching rule is to switch to the adjacent floor only after all explorations on the current floor have been completed and no object target has been detected. The first training sample includes the map of the next floor, the map of the current floor, the map of the previous floor and their corresponding relative floor labels. The floor switching strategy is pre-trained by imitation learning using the first training sample. For the floor exploration strategy, the sub-target labels used as supervision data are generated by the sub-targets selected in the scene by the zero-sample object target navigation strategy or the visual language model navigation strategy. The training samples include the current floor map and its corresponding sub-target labels. The floor exploration strategy is pre-trained by imitation learning using the second training samples.

[0015] Preferably, the strategy training module includes a reinforcement learning alternating fine-tuning unit, which is used to alternately fine-tune the pre-trained inter-floor switching strategy and intra-floor exploration strategy; In each round of alternating fine-tuning, first freeze the inter-floor switching strategy and fine-tune the floor-in-floor exploration strategy, then freeze the floor-in-floor exploration strategy and fine-tune the inter-floor switching strategy. In reinforcement learning, the reward function includes at least the positive reward for successfully reaching the target object, the reward for the change in geodesic distance from the robot to the target object, and the time penalty corresponding to the decision interval.

[0016] Preferably, the underlying navigation module includes a point-to-point navigation unit and a cross-floor execution unit; When the current floor is selected as the inter-floor switching strategy, the point-to-point navigation unit converts the sub-target selected by the floor exploration strategy into a navigation target relative to the robot's current position and inputs it into the point-to-point navigation controller, which then outputs forward, turn, view adjustment, or stop actions. When the strategy for switching between floors selects the upper or lower floor, the corresponding staircase boundary is first used as the navigation target, and the point-to-point navigation controller is used to guide the robot to approach the staircase. During the cross-floor phase, waypoints are continuously generated in front of the staircase area and the point-to-point navigation controller is iteratively called to enable the robot to reach the target floor by passing through the staircase.

[0017] To achieve the above-mentioned objectives, the embodiments also provide a multi-story robot object target navigation method based on a hierarchical learning strategy, comprising the following steps: The system perceives environmental information and object targets and constructs a semantic boundary map that combines the exploration value, exploration boundary and object target perception results to represent the correlation between the current observation and the object target, as well as a staircase boundary map. Construct a multi-floor map representation that includes the current floor and adjacent floors based on the exploration value map, semantic boundary map, and staircase boundary map; After pre-training the inter-floor switching strategy and the intra-floor exploration strategy using imitation learning, the two strategies are then fine-tuned alternately by freezing one and fine-tuning the other based on reinforcement learning. Based on multi-floor map representation, the target floor selection result is output through the inter-floor switching strategy decision, and the floor validity mask is combined to suppress floors that are unreachable or have no valid candidate targets; the candidate sub-target selection result within the target floor is also output through the floor exploration strategy decision, and the non-candidate area is suppressed by combining the semantic boundary mask, so that the robot can select sub-targets for exploration or target approach within the target floor. Perform cross-floor navigation based on the target floor selection result and perform target navigation based on the candidate sub-target selection result.

[0018] Compared with the prior art, the beneficial effects of the present invention include at least the following: 1) Clear Hierarchy. This invention decomposes the decision-making process in multi-floor object target navigation into inter-floor switching decisions and intra-floor sub-target selection decisions, enabling the robot to first determine which floor to explore and then select specific navigation sub-targets within the selected floor. Compared with directly training a single global policy, this hierarchical decision-making method can reduce the complexity of the decision space and improve the interpretability of the policy training and deployment process.

[0019] 2) Complete representation. This invention constructs a multi-floor map representation that includes the current floor, the previous floor, and the next floor. In each floor map, semantic boundary maps, exploration value maps, robot state maps, historical semantic boundary maps, historical exploration value maps, and historical action maps are integrated, so that the strategy can simultaneously utilize current observations, historical decisions, and accessibility information of adjacent floors.

[0020] 3) Highly efficient training. This invention employs a strategy training method that combines imitation learning pre-training with reinforcement learning alternating fine-tuning. Imitation learning can provide effective initialization for the strategy using zero-shot navigation strategies, visual language model navigation strategies, or greedy floor switching rules; reinforcement learning alternating fine-tuning can optimize one strategy while freezing another, reducing the coupling difficulty caused by simultaneous training of intra-floor and inter-floor strategies.

[0021] 4) Reliable navigation. This invention suppresses unreachable floors and non-candidate areas through floor validity masks and semantic boundary masks, avoiding invalid actions in policy selection; at the same time, through stair boundary construction and cross-floor execution units, the robot can approach the stairs first when it needs to switch floors, and then complete cross-floor navigation through waypoint iteration or simulation state reset.

[0022] 5) High practicality. This invention integrates RGBD perception, semantic target perception, exploration value perception, multi-floor map construction, hierarchical learning decision-making, strategy training, and low-level navigation execution into a unified framework. It can be used for object target navigation tasks in multi-floor unknown environments such as home services, building inspection, warehouse management, and indoor guidance, and has good engineering application value.

[0023] Therefore, this invention is a practical and effective multi-floor robot object target navigation scheme that can improve the robot's exploration efficiency, floor switching rationality, target search success rate, and navigation process controllability in multi-floor unknown environments, and has good application prospects. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a schematic diagram of the structure of a multi-story robot object target navigation system based on a hierarchical learning strategy provided in the embodiment. Figure 2 This is a flowchart of a multi-story robot object target navigation method based on a hierarchical learning strategy provided in the embodiment; Figure 3 This is a schematic diagram of the navigation results provided in the embodiment. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not limit the scope of protection of this invention.

[0027] like Figure 1 As shown in the embodiment, the multi-layer robot object target navigation system based on a hierarchical learning strategy includes a perception and map building module, a multi-layer map representation module, a strategy training module, a hierarchical learning decision module, and a bottom-level navigation module.

[0028] In this embodiment, at the start of the task, the robot receives the target object category to be navigated through the perception and map building module, and continuously acquires RGB images, depth maps, and its own pose in a multi-story unknown environment, constructing local and semantic maps based on the current observations. Specifically, as follows... Figure 1 As shown, the perception and map building module includes an RGBD information acquisition unit, an object target perception unit, an exploration value perception unit, an occupancy map building unit, a boundary building unit, and a stair boundary building unit.

[0029] The RGBD information acquisition unit is used to obtain the robot's current RGB image, depth map, and robot pose at each moment, while also receiving the object target category to be navigated.

[0030] The object target perception unit is used to input the current RGB image into the object detection model, obtain the detection result consisting of the detection box corresponding to the object target category and the confidence score, and project the detection result into the map coordinate system by combining the depth map and robot pose to form a semantic candidate region for target navigation.

[0031] The exploration value perception unit is used to input the prompts corresponding to the current RGB image and the object target category into the visual language model or visual semantic model, predict the correlation between the current RGB image and the object target, and convert the correlation result into exploration value. Specifically, the correlation score is written into the local fan-shaped region corresponding to the robot's current observation direction according to the distribution of high in the middle and low on both sides, and fused with the historical exploration value map according to weight to obtain the updated exploration value map.

[0032] The occupancy map construction unit is used to generate the explored area, unknown area and obstacle area of ​​the current floor to form an occupancy map. Specifically, it generates the current frame point cloud based on the current depth map and robot pose, converts the current frame point cloud into a two-dimensional occupancy representation, and merges it with the historical occupancy representation to obtain the explored area, unknown area and obstacle area of ​​the current floor.

[0033] The boundary construction unit is used to obtain the boundary between the explored area and the unknown area based on the occupancy map, and to fuse the boundary with the object target perception result (i.e. semantic candidate region) to obtain a semantic boundary map containing the semantic candidate locations of the explored boundary and the object target.

[0034] The stair boundary building unit is used to generate a stair boundary map to guide the robot to approach the stair when a stair is detected. Specifically, the stair area is projected onto the top-view map, and a passable grid is searched around the projected stair area to generate the stair boundary.

[0035] In this embodiment, the multi-floor map representation module is used to construct a multi-floor map representation that includes the next floor, the current floor, and the previous floor. For example... Figure 1 As shown, the multi-floor map representation module includes the current floor map representation unit, the adjacent floor map representation unit, and the multi-floor map stacking unit.

[0036] The current floor map representation unit is used to represent the navigation-related information of the current floor as a multi-channel two-dimensional tensor. This multi-channel two-dimensional tensor includes at least the semantic boundary map at the current moment, the exploration value map at the current moment, the robot state map at the current moment, the semantic boundary map at the previous moment, the exploration value map at the previous moment, and the action map at the previous moment. The exploration value map is obtained by combining the exploration value output by the visual language model or visual semantic model with the explored area and obstacle area; the robot state map is obtained by projecting the robot's position and orientation onto the grid map; and the action map at the previous moment is obtained by encoding the sub-target position selected at the previous moment.

[0037] The adjacent floor map representation unit is used to construct an adjacent floor map that includes an upper floor map and a lower floor map. When there is a staircase boundary leading to the upper or lower floor, the corresponding adjacent floor map is generated according to the current floor map construction method, and the robot state in the adjacent floor map is anchored to the staircase connection position, and the robot orientation is set to the preset cross-floor orientation; when there is no staircase boundary leading to an adjacent floor, or when there is no valid candidate area for the corresponding adjacent floor, the adjacent floor map is set to an empty map or an invalid map.

[0038] The multi-floor map stacking unit stitches or stacks the map of the lower floor, the map of the current floor, and the map of the upper floor in the order of the floors to obtain a multi-floor map representation for decision-making between floors.

[0039] In this embodiment, the policy training module is used to pre-train a floor-to-floor switching policy and a floor-to-floor exploration policy using imitation learning, and then alternately fine-tunes the two policies based on reinforcement learning, providing trained policy parameters to the hierarchical learning decision module. Figure 1 As shown, the policy training module includes an imitation learning pre-training unit and a reinforcement learning alternating fine-tuning unit.

[0040] The imitation learning pre-training unit is used to pre-train the intra-floor exploration strategy and the inter-floor switching strategy in the first stage. For the intra-floor exploration strategy, the sub-target labels, serving as supervision data, are generated from sub-targets selected in the scene by the zero-sample object target navigation strategy or the visual-language model navigation strategy. The first training sample includes the current floor map and its corresponding sub-target labels. For the inter-floor switching strategy, the floor labels, serving as supervision data, are generated by the greedy floor switching rule. The greedy floor switching rule switches to an adjacent floor only after all explorations of the current floor have been completed (exploration to the point where there are no boundaries in the current floor) and no object target has been detected. The second training sample includes the next floor map, the current floor map, the previous floor map, and their corresponding relative floor labels. Through imitation learning pre-training, the cross-entropy loss function is used during pre-training. Both the intra-floor exploration strategy and the inter-floor switching strategy can obtain initial strategies that can be used for subsequent reinforcement learning optimization.

[0041] The reinforcement learning alternating fine-tuning unit is used in the second phase to alternately fine-tune the pre-trained intra-floor exploration strategy and inter-floor switching strategy. In each round of alternating fine-tuning, the inter-floor switching strategy is first frozen, and the intra-floor exploration strategy is fine-tuned through reinforcement learning. Then, the intra-floor exploration strategy is frozen again, and the inter-floor switching strategy is fine-tuned through reinforcement learning. This alternating fine-tuning method reduces the training coupling between inter-floor switching and intra-floor sub-objective selection, and optimizes intra-floor exploration capability and cross-floor decision-making capability, respectively.

[0042] Reinforcement learning can employ maximum entropy reinforcement learning or other methods suitable for discrete action policies. Specifically, in reinforcement learning, an intra-floor exploration policy or an inter-floor switching policy is used as the policy network to be optimized, with the current floor map or multi-floor map representation as the policy state input. The intra-floor exploration policy outputs the action probability distribution of candidate sub-objects within the floor, while the inter-floor switching policy outputs the action probability distribution of the next floor, the current floor, and the previous floor. Action sampling or selection is performed after semantic boundary masks or floor validity masks. The robot executes the corresponding navigation process based on the selected action and constructs a reward signal based on whether the target object is successfully reached, the change in geodesic distance between the robot and the target object, and the decision interval penalty. The maximum entropy reinforcement learning method introduces a policy entropy term while optimizing the expected cumulative reward, enabling the policy to maintain a certain level of exploration capability while achieving high navigation gains. During training, an alternating fine-tuning method is used: when optimizing the intra-floor exploration policy, the inter-floor switching policy parameters are frozen; when optimizing the inter-floor switching policy, the intra-floor exploration policy parameters are frozen, thereby updating the parameters of the corresponding policy networks separately and reducing the coupling difficulty caused by simultaneously optimizing two policies.

[0043] Reinforcement learning rewards include at least a positive reward for successfully reaching the target object, a reward for the change in geodesic distance between the robot and the target object, and a time penalty corresponding to the decision interval. Specifically, the reward function used in reinforcement learning is expressed as follows: : When the target object is successfully reached, the reward r is 2.5. and It is the distance between the robot and the sub-target before and after the strategy is executed. It represents the number of simulator steps between two consecutive decisions, which varies depending on the strategy chosen. When the strategy is floor-by-floor exploration... Selected as a small number of steps, preferably, When the strategy is an inter-floor switching strategy, Selected as the largest step size, preferably 0.

[0044] In this embodiment, the hierarchical learning decision module outputs the floor selection result and sub-objectives within the floor based on the multi-floor map representation and the current floor map. For example... Figure 1 As shown, the hierarchical learning decision module includes a floor-to-floor switching strategy unit and a floor-to-floor exploration strategy unit.

[0045] The inter-floor switching strategy unit is used to input the multi-floor map representation into the inter-floor strategy network and output the 3D logit vectors corresponding to the next floor, the current floor, and the previous floor. It generates a floor validity mask based on the validity of each floor map: when there is no valid candidate region in the semantic boundary map of the current floor, the mask value corresponding to the current floor is set to invalid; when there is no valid robot state anchor point or no valid semantic boundary candidate region in the upper or lower floor, the mask value of the corresponding adjacent floor is set to invalid. The floor validity mask is used to retain the logit values ​​of valid floors and suppress the logit values ​​of invalid floors. After normalization, the floor with the highest probability is selected as the target floor.

[0046] The floor exploration strategy unit is used to input the current floor map corresponding to the selected target floor into the floor strategy network and output the candidate sub-target logit map corresponding to the map grid. It generates a binary candidate mask as a semantic boundary mask based on the semantic boundary map in the current floor map: marking non-zero positions in the semantic boundary map as valid candidate regions and marking other positions as invalid regions; using the semantic boundary mask to retain the logit values ​​of valid candidate regions and suppress the logit values ​​of invalid regions, normalizing the logit map after masking to obtain the candidate sub-target probability map, and taking the grid position with the highest probability as the sub-target in the target floor.

[0047] In one embodiment, the floor exploration strategy is executed once at a first preset time interval, and the floor switching strategy is executed once at a second preset time interval, with the second preset time interval being greater than the first preset time interval. This makes the update frequency of the floor switching decision lower than the update frequency of the sub-target decision within the floor, thereby avoiding the robot frequently switching floors in a short period of time.

[0048] The intra-floor policy network can use a convolutional encoder to extract channel features from the current floor map and use a U-Net structure to output candidate sub-target logit maps. The inter-floor policy network can reuse part of the feature extraction network of the intra-floor policy network to process the lower floor map, the current floor map, and the upper floor map respectively. Then, the three floor-level features are concatenated and output floor selection logit vectors through convolutional layers and multilayer perceptrons.

[0049] In this embodiment, the underlying navigation module is used to generate the actual motion actions performed by the robot based on the decision results. Specifically, it generates underlying motion actions based on the selected sub-targets and guides the robot to the staircase boundary and complete the floor switch during cross-floor navigation. Figure 1 As shown, the underlying navigation module includes point-to-point navigation units and cross-floor execution units.

[0050] When the current floor is selected as the floor switching strategy, the point-to-point navigation unit converts the sub-target selected by the floor exploration strategy into a navigation target relative to the robot's current position and inputs it into the point-to-point navigation controller, which then outputs forward, turn, view adjustment, or stop actions.

[0051] When the strategy for switching between floors selects the upper or lower floor, the system first uses the corresponding staircase boundary as the navigation target and uses the point-to-point navigation controller to guide the robot to approach the stairs. During the cross-floor phase, waypoints can be continuously generated in front of the staircase area and the point-to-point navigation controller can be iteratively called to enable the robot to reach the target floor by passing through the stairs.

[0052] In simulation training or evaluation environments, the cross-floor execution unit can also simulate cross-floor execution by resetting the robot's state to the corresponding position on the target floor and accumulating a preset number of steps. This method can be used to verify the decision-making effect of the inter-floor switching strategy itself. In real robot deployment, the cross-floor execution unit preferably adopts a waypoint iterative guidance method, allowing the robot to move gradually along the stairs to the target floor.

[0053] like Figure 2 As shown in the embodiment, a multi-story robot object target navigation method based on a hierarchical learning strategy is also provided. This method uses the above-mentioned system and specifically includes the following steps: First, the perception and map building module performs RGBD-based perception and map building: acquiring the current RGB image, depth map and robot pose, perceiving candidate regions of objects, predicting the exploration value of the currently observed RGB image for finding objects, and building occupancy map, semantic boundary map, exploration value map and stair boundary; Then, the multi-floor map representation is constructed through the multi-floor map representation module: the adjacent floor maps are constructed based on the current floor map and the stair boundary, and the lower floor map, the current floor map and the upper floor map are stacked to obtain the multi-floor map representation; Next, the policy training module performs imitation learning pre-training and reinforcement learning alternating fine-tuning: supervised data is generated using zero-shot navigation strategy, visual language model navigation strategy or greedy floor switching rule to perform imitation learning pre-training on the floor exploration strategy and the floor switching strategy; and after pre-training, reinforcement learning alternating fine-tuning is used to optimize the selection of sub-targets within the floor and the decision-making between floors.

[0054] After training, during robot deployment, a decision based on a hierarchical learning strategy is executed: the multi-floor map representation is input into the floor switching strategy to obtain the selection result of the next floor, the current floor, or the next floor; when the current floor is selected, the current floor map is input into the floor exploration strategy to obtain the sub-objectives within the floor; when the next or next floor is selected, the corresponding staircase boundary is taken as the cross-floor sub-objective.

[0055] Finally, the execution steps based on the underlying navigation strategy are performed: the selected sub-targets within or across floors are converted into navigation targets relative to the robot's current position, and a point-to-point navigation strategy is used to generate underlying motion actions, enabling the robot to gradually approach the target or complete the floor switch. If the robot reaches the vicinity of the target and performs a stopping action, the task is completed. If the robot fails to reach the target object, the RGBD-based perception and map building, multi-layer map representation construction, hierarchical learning decision-making, and low-level navigation execution processes will be re-executed until the robot reaches the target object or reaches the preset maximum number of steps, preset time, or other task termination conditions.

[0056] The embodiments also provide navigation results using the above method, such as... Figure 3 As shown, at t=54, the floor exploration strategy unit selects the corridor's leading edge, which has higher exploration value. At t=313, the robot completes its exploration of the corridor but detects no target, and must decide whether to continue exploring the current floor or switch to another floor. Since most of the current floor has been explored and the robot is close to the stairs, the floor switching strategy unit selects the next floor. At t=333, the robot reaches the next floor. At t=350, the robot successfully perceives the target object and navigates to its location.

[0057] The specific embodiments described above illustrate the technical solution and beneficial effects of the present invention in detail. It should be understood that the above description is only the most preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, additions, and equivalent substitutions made within the scope of the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A multi-story robot object navigation system based on a hierarchical learning strategy, characterized in that, include: The perception and map building module is used to perceive environmental information and object targets and build a semantic boundary map that combines the exploration value, exploration boundary and object target perception results to represent the correlation between the current observation and the object target, as well as a staircase boundary map. The multi-floor map representation module is used to construct a multi-floor map representation containing the current floor and adjacent floors based on exploration value, semantic boundary map, and stair boundary map; The strategy training module is used to pre-train the inter-floor switching strategy and the intra-floor exploration strategy using imitation learning, and then alternately fine-tun the two strategies by freezing one and fine-tuning the other based on reinforcement learning. The hierarchical learning decision module uses a multi-floor map representation to output the target floor selection result through the strategy decision between floors, and combines floor validity mask to suppress floors that are unreachable or have no valid candidate targets; it also outputs the candidate sub-target selection result within the target floor through the strategy decision within the floor, and combines semantic boundary mask to suppress non-candidate areas, enabling the robot to select sub-targets for exploration or target proximity within the target floor. The underlying navigation module is used to perform cross-floor navigation based on the target floor selection result and target navigation based on the candidate sub-target selection result.

2. The multi-story robot object target navigation system based on a hierarchical learning strategy according to claim 1, characterized in that, The perception and map building module includes: The RGBD information acquisition unit is used to acquire the robot's current RGB image, depth map, and robot pose in an unknown multi-story environment, and to receive the object target category to be navigated. The object target perception unit is used to obtain the detection results corresponding to the object target category in the current RGB image, and combine the depth map and robot pose to project the detection results into the map coordinate system to form a semantic candidate region for target navigation. The exploration value perception unit is used to predict the correlation between the current RGB image and the target object through a visual language model or a visual semantic model, and convert the correlation results into exploration value; Occupation map building unit, which is used to generate the explored area, unknown area and obstacle area of ​​the current floor to form an occupation map; Boundary construction unit, which is used to obtain the boundary between explored and unknown areas based on the occupancy map, and to merge the boundary with semantic candidate areas to obtain a semantic boundary map; A stair boundary building unit is used to generate a stair boundary map to guide the robot to approach the stair when a stair is detected.

3. The multi-story robot object target navigation system based on a hierarchical learning strategy according to claim 1, characterized in that, The multi-floor map representation module includes: The current floor map representation unit is used to represent the current floor map, including at least the semantic boundary map, exploration value map, and robot state map at the current moment, and the semantic boundary map, exploration value map, and action map at the previous moment. The adjacent floor map representation unit is used to generate the corresponding adjacent floor map when there is a staircase boundary leading to the upper or lower floor, and to anchor the robot state in the adjacent floor map to the staircase connection position. The adjacent floor map includes the upper floor map and the lower floor map. A multi-level map stacking unit is used to stack the map of the lower level, the map of the current level, and the map of the upper level to obtain a multi-level map representation.

4. The multi-story robot object target navigation system based on a hierarchical learning strategy according to claim 1, characterized in that, The hierarchical learning decision module includes a floor-to-floor switching strategy unit, which is used to input the multi-floor map representation into the floor-to-floor strategy network and output a three-dimensional logit vector corresponding to the next floor, the current floor, and the previous floor. Generate floor validity masks based on the validity of each floor map: when there is no valid candidate region in the semantic boundary map of the current floor, set the mask value corresponding to the current floor to invalid; when there is no valid robot state anchor point or no valid semantic boundary candidate region in the upper or lower floor, set the mask value of the corresponding adjacent floor to invalid. The floor validity mask is used to retain the logit value of valid floors and suppress the logit value of invalid floors. After normalization, the floor with the highest probability is selected as the target floor.

5. The multi-story robot object target navigation system based on a hierarchical learning strategy according to claim 1, characterized in that, The hierarchical learning decision module also includes an in-floor exploration strategy unit, which is used to input the current floor map corresponding to the selected target floor into the in-floor strategy network and output a candidate sub-target logit map corresponding to the map grid. Generate a binary candidate mask as the semantic boundary mask based on the semantic boundary map in the current floor map: mark the non-zero positions in the semantic boundary map as valid candidate regions, and mark other positions as invalid regions; Semantic boundary masks are used to retain the logit values ​​of valid candidate regions and suppress the logit values ​​of invalid regions. The logit map after masking is normalized to obtain the probability map of candidate sub-targets, and the grid position with the highest probability is taken as the sub-target within the target floor.

6. The multi-story robot object target navigation system based on a hierarchical learning strategy according to claim 1, characterized in that, The floor-to-floor exploration strategy is executed once at a first preset time interval, and the floor-to-floor switching strategy is executed once at a second preset time interval, with the second preset time interval being greater than the first preset time interval, so that the update frequency of the floor-to-floor switching decision is lower than the update frequency of the floor-to-floor sub-target decision.

7. The multi-story robot object target navigation system based on a hierarchical learning strategy according to claim 1, characterized in that, The strategy training module includes an imitation learning pre-training unit, which is used to perform imitation learning-based pre-training on inter-floor switching strategies and intra-floor exploration strategies. For the floor switching strategy, the floor labels used as supervision data are generated by the greedy floor switching rule. The greedy floor switching rule is to switch to the adjacent floor only after all explorations on the current floor have been completed and no object target has been detected. The first training sample includes the map of the next floor, the map of the current floor, the map of the previous floor and their corresponding relative floor labels. The floor switching strategy is pre-trained by imitation learning using the first training sample. For the floor exploration strategy, the sub-target labels used as supervision data are generated by the sub-targets selected in the scene by the zero-sample object target navigation strategy or the visual language model navigation strategy. The training samples include the current floor map and its corresponding sub-target labels. The floor exploration strategy is pre-trained by imitation learning using the second training samples.

8. The multi-story robot object target navigation system based on a hierarchical learning strategy according to claim 1, characterized in that, The strategy training module includes a reinforcement learning alternating fine-tuning unit, which is used to alternately fine-tune the pre-trained inter-floor switching strategy and intra-floor exploration strategy. In each round of alternating fine-tuning, the inter-floor switching strategy is first frozen and the floor exploration strategy is fine-tuned through reinforcement learning, then the floor exploration strategy is frozen and the inter-floor switching strategy is fine-tuned through reinforcement learning. In reinforcement learning, the reward function includes at least the positive reward for successfully reaching the target object, the reward for the change in geodesic distance from the robot to the target object, and the time penalty corresponding to the decision interval.

9. The multi-story robot object target navigation system based on a hierarchical learning strategy according to claim 1, characterized in that, The underlying navigation module includes a point-to-point navigation unit and a cross-floor execution unit; When the current floor is selected as the inter-floor switching strategy, the point-to-point navigation unit converts the sub-target selected by the floor exploration strategy into a navigation target relative to the robot's current position and inputs it into the point-to-point navigation controller, which then outputs forward, turn, view adjustment, or stop actions. When the strategy for switching between floors selects the upper or lower floor, the corresponding staircase boundary is first used as the navigation target, and the point-to-point navigation controller is used to guide the robot to approach the staircase. During the cross-floor phase, waypoints are continuously generated in front of the staircase area and the point-to-point navigation controller is iteratively called to enable the robot to reach the target floor by passing through the staircase.

10. A multi-story robot object target navigation method based on a hierarchical learning strategy, characterized in that, Includes the following steps: The system perceives environmental information and object targets and constructs a semantic boundary map that combines the exploration value, exploration boundary and object target perception results to represent the correlation between the current observation and the object target, as well as a staircase boundary map. Construct a multi-floor map representation that includes the current floor and adjacent floors based on the exploration value map, semantic boundary map, and staircase boundary map; After pre-training the inter-floor switching strategy and the intra-floor exploration strategy using imitation learning, the two strategies are then fine-tuned alternately by freezing one and fine-tuning the other based on reinforcement learning. Based on multi-floor map representation, the target floor selection result is output through the inter-floor switching strategy decision, and the floor validity mask is combined to suppress floors that are unreachable or have no valid candidate targets; the candidate sub-target selection result within the target floor is also output through the floor exploration strategy decision, and the non-candidate area is suppressed by combining the semantic boundary mask, so that the robot can select sub-targets for exploration or target approach within the target floor. Execute cross-floor navigation based on the target floor selection result and perform target navigation based on the candidate sub-target selection result.

Images