A method and device for object layout reactive navigation based on topology intention guidance

Abstract

The application discloses a kind of object layout reactive navigation method and device based on topological intention guidance, and constructs the unified technical scheme consisting of object topological representation, global topological intention generation, local reactive control and feasible region constraint optimization.By compressing the subsequent topological descending trend in global optimal object level path into low-dimensional topological intention signal, and injecting the signal as conditional information into local controller, the robot can obtain directional guidance from global topological structure while maintaining the ability to respond sensitively to current observation, thereby realizing the transition from "reacting only on local visible information" to "making decisions considering both local perception and global trend".The scheme uses soft guidance to modulate the internal feature representation of control strategy, so it can avoid the destruction of local flexibility by traditional forced global control, and effectively suppress the misleading behavior caused by local optimal trap.

Description

Technical Field

[0001] This invention relates to the fields of robot navigation and computer vision technology, and in particular to a reactive navigation method and apparatus for object layout based on topological intent guidance. Background Technology

[0002] Visual navigation targeting objects is a crucial capability for mobile robots to perform service, inspection, and autonomous movement tasks in indoor environments. Existing object-oriented navigation methods typically rely on the semantic information, geometric relationships, and local control strategies of objects visible from the robot's current perspective to make navigation decisions.

[0003] In recent years, navigation methods based on object topology maps have been able to perform long-range planning using object instances and their connectivity relationships. Compared to dense geometric maps, these methods offer advantages such as compact representation, robustness to environmental changes, and support for open-set semantic target queries. However, existing methods still suffer from a disconnect between global planning and local control during the execution phase: while the global planning phase has obtained the topological path information from the current region to the target object, local control typically relies solely on the local costs of objects visible from the current viewpoint for reactive decisions, lacking explicit guidance on topological trends outside the field of view.

[0004] In scenarios involving symmetrical corridors, branching structures, or where the robot's initial orientation deviates from the target, making decisions based solely on the local cost of currently visible objects can easily lead to the robot continuously moving along the current orientation, even if that direction cannot reduce the actual topological distance to the target object. This results in navigation behavior that appears reasonable locally but deviates globally.

[0005] Therefore, a navigation method is needed that can retain the advantages of reactive control of object layout while injecting the results of global topology planning into the local control process in a compact form. Summary of the Invention

[0006] The purpose of this invention is to provide a reactive navigation method and apparatus for object layout based on topological intent guidance, so as to solve the problem of disconnect between global topology planning and local object layout control in existing object navigation methods, enabling the robot to navigate in the direction of decreasing topological distance even in some observable environments.

[0007] To achieve the above objectives, the present invention provides the following technical solution:

[0008] This invention discloses a reactive navigation method for object layout based on topological intent guidance, comprising the following steps:

[0009] S1: Construct an object topology map, which is represented as graph G=(N,E), where N represents the set of object instance nodes in the environment and E represents the set of edges connecting the object instances.

[0010] S2: Establish associations between object instances in the currently observed image and nodes in the object topology map G to obtain the associated node set. and on the object topology map Global planning is performed with each node as the starting point and the target node as the ending point to obtain a set of topological distances. After encoding, the object layout cost map is obtained, and the sub-target nodes are calculated simultaneously. and the next hop node ;

[0011] S3: Generate a topological intent vector based on the orientation of the next-hop node relative to the robot's current pose. The topological intent vector is used to represent the guidance information towards the direction of decreasing topological distance at the current moment;

[0012] S4: Use the object layout cost map from S2 as the main input to the local reactive controller; at the same time, use the topology intent vector generated in step S3 as conditional guidance information to input the local reactive controller, modulate the internal feature representation of the controller, and predict local waypoints.

[0013] S5: Based on the bird's-eye view of the passable map, perform feasibility correction on local waypoints to obtain the corrected execution waypoints;

[0014] S6: Control the robot's movement based on the waypoints and repeat steps S2 to S6 until the target node is reached.

[0015] As a further improvement, the node attributes in the object topology map of the present invention include at least a two-dimensional segmentation mask of the object instance and three-dimensional global coordinates.

[0016] The edges in the object topology map include intra-graph edges between different object instances co-visible in the same image, and inter-graph edges between the same object instances at different times; wherein, intra-graph edges are used to represent the spatial adjacency relationship between object instances, and inter-graph edges are used to represent the temporal association relationship of the same object instance.

[0017] As a further improvement, in step S2 of the present invention, the sub-target node Determined by the following formula:

[0018]

[0019] in, This represents the set of nodes obtained by associating the currently observed object instances with the object topology map. Represents a node Topological distance to the target node This indicates the current sub-target node.

[0020] As a further improvement, in step S2 of the present invention, the next hop node Determined by the following formula:

[0021]

[0022] in, This represents the sub-objective node obtained from global planning. To the target node The sequence of nodes on the shortest path, Indicates the next hop node.

[0023] As a further improvement, in step S2 of this invention, the generation of the object layout cost map includes: encoding the set of topological distances from each object instance in the current observation to the target node. The encoding method for the set of topological distances is any one of fixed sinusoidal position encoding, learnable embedding encoding, or a combination of both, to obtain the encoded set of topological distances; the encoded set of topological distances is projected onto the two-dimensional segmentation mask covering area of ​​the corresponding object instance to form a dense object layout cost map.

[0024] As a further improvement, in step S3 of this invention, the topological intent vector is calculated from the direction of the next-hop node relative to the robot's current local coordinate system, and is expressed as:

[0025]

[0026] in, Represents the topological intent vector. This represents the angle between the robot's current orientation and the direction of the next hop node. This indicates transpose.

[0027] As a further improvement, in step S4 of this invention, the local reactive controller introduces a topological intent vector through feature linear modulation to conditionally modulate the intermediate features extracted from the object layout cost map. The modulation form is as follows:

[0028]

[0029] in, Indicates that the local controller is in the channel Spatial location The original features of the place, Indicates the characteristics after modulation. Represented by the topological intent vector The generated scaling factor, Represented by the topological intent vector The generated bias coefficient.

[0030] As a further improvement, in step S5 of the present invention, the feasibility correction of local waypoints is performed based on the bird's-eye view traversable map, including: directly executing when the predicted local waypoint is located in a traversable area; and projecting or adjusting the predicted local waypoint to an adjacent traversable area when it is located in an obstacle area or an impassable area, so as to maintain the consistency of the local movement direction and meet the geometrically executable constraints.

[0031] This invention discloses a topology intent-guided object layout reactive navigation device, comprising: an object topology map construction unit for constructing an object topology map, wherein the object topology map is represented as graph G=(N,E), where N represents the set of object instance nodes in the environment and E represents the set of connecting edges between object instances;

[0032] Global Planning and Cost Graph Generation Unit: Used to establish associations between object instances in the currently observed image and nodes in the object topology map G, obtaining a set of associated nodes. and on the object topology map Global planning is performed with each node as the starting point and the target node as the ending point to obtain a set of topological distances. After encoding, the object layout cost map is obtained, and the sub-target nodes are calculated simultaneously. and the next hop node ;

[0033] Topology Intent Generation Unit: Used to generate topology intent vectors based on the orientation of the next-hop node relative to the robot's current pose. The topological intent vector is used to represent the guidance information towards the direction of decreasing topological distance at the current moment;

[0034] Local control unit: used to take the object layout cost map of S2 as the main input of the local reactive controller; at the same time, the topology intention vector generated in step S3 is used as conditional guidance information to input the local reactive controller to modulate the internal feature representation of the controller in order to predict local waypoints.

[0035] Feasibility Correction Unit: Used to perform feasibility correction on local waypoints based on a bird's-eye view navigable map, and obtain the corrected execution waypoints;

[0036] Execution control unit: Used to control the robot's movement based on the execution waypoints, and to repeatedly execute steps S2 to S6 until the target node is reached.

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] 1. This invention proposes a reactive navigation method for object layout based on topological intent guidance. Addressing the common technical problem of "disconnect between global planning and local control" in existing object-level visual navigation methods, it constructs a unified technical solution consisting of object topological representation, global topological intent generation, local reactive control, and feasible region constraint optimization. While existing technologies can utilize object topological maps to represent semantic targets and their connections in the environment and support long-term navigation to some extent, their local control phase typically relies primarily on currently visible objects and their corresponding costs for decision-making. This lacks explicit constraints on the future topological evolution direction outside the robot's current field of view. Therefore, in symmetrical environments, corridor environments, occluded environments, and situations where the robot's initial orientation deviates significantly from the target direction, it is prone to phenomena where local decisions appear reasonable but the overall direction continuously deviates from the target, leading to path detours, repeated trials, reduced efficiency, and even task failure. This invention compresses the subsequent topological descent trend in the globally optimal object-level path into a low-dimensional topological intent signal, and injects this signal as conditional information into the local controller. This allows the robot to maintain its sensitivity to current observations while receiving directional guidance from the global topological structure, thus achieving a shift from "responding solely based on locally visible information" to "making decisions while considering both local perception and global trends." This scheme does not directly apply rigid trajectory or action commands to the controller, but instead uses a soft-guided approach to modulate the internal feature representation of the control strategy. Therefore, it avoids the destruction of local flexibility caused by traditional forced global control and effectively suppresses misleading behavior caused by local optimum traps. Furthermore, this invention introduces a feasible domain-aware trajectory optimization mechanism based on a bird's-eye view of traversable areas. This mechanism corrects the predicted local waypoints output by the learning controller under geometric constraints, ensuring that the generated execution waypoints not only possess the correctness of target-oriented advancement but also have practical traversability, executability, smoother, and safer motion characteristics. Therefore, this invention improves the overall navigation success rate, path efficiency, trajectory stability, and environmental adaptability of robots in complex indoor environments. It is particularly suitable for applications requiring long-distance target arrival under partially observable conditions, such as semantic target navigation, indoor service robot navigation, warehouse logistics robot navigation, and intelligent inspection robot navigation. Furthermore, the "topological intent" proposed in this invention possesses excellent modularity and universal interface attributes. Its source is not limited to global planning results on object-level topology maps but can also come from manual directional instructions, semantic knowledge base reasoning results, task rule constraints, or autonomous exploration heuristics. Therefore, this invention not only improves the performance of existing visual navigation systems but also establishes a unified, compact, and scalable connection between high-level guidance information and low-level reactive control, exhibiting strong versatility, portability, scalability, and engineering deployment value.In other words, the present invention can effectively enhance the robot's globally consistent decision-making and implementation capabilities in partially observable complex environments without significantly increasing the system's representation complexity and execution burden, thereby achieving comprehensive technical effects superior to existing technologies.

[0039] 2. This invention, for the first time, explicitly introduces a global topological intention into a local reactive control framework and uses this topological intention as a feature representation of the conditional signal modulation controller, thereby structurally achieving an effective bridge between global planning and local control. Compared to existing technologies where the global planning result can only be indirectly reflected in the cost of currently visible objects, this invention enables the local controller to directly perceive the guidance information corresponding to the future topological distance descent direction. Therefore, even in situations with similar environmental structures, strong observational ambiguity, targets not within the current field of view, or misleading visible object cues, it can still maintain a forward trend consistent with the global target. Thus, this invention can significantly mitigate the problems of misjudgment, detours, and erroneous continuous progress caused by incomplete local information, improving the direction accuracy, mission success rate, and long-range path convergence capability during navigation, demonstrating significant technological advancement.

[0040] 3. This invention proposes a topology intent vector representation method based on next-hop nodes, expressing the future topology advancement direction with minimal necessary global information. Compared to schemes that directly input the complete global path, global map, or high-dimensional planning state, the topology intent vector based on next-hop nodes proposed in this invention retains only the next-stage topology distance descent information most critical for local decision-making. This results in a more compact information representation and a more focused control effect, effectively eliminating local policy ambiguity without excessively interfering with local responsiveness. This scheme avoids the problems of coupling complexity, training difficulties, and decreased generalization ability caused by high-dimensional global information, while retaining the flexibility of the local reactive controller to autonomously adjust actions based on real-time perception. Therefore, this invention can ensure global consistency while also considering local executability, which is particularly beneficial for robots to quickly complete correct turning and effective advancement when initial orientation deviation is large, the correct future path is not yet visible, or multiple candidate object cues exist locally, thus improving the system's generalization ability, stability, and interpretability.

[0041] 4. This invention proposes to perform feasibility correction on local waypoints based on a bird's-eye view (BEV) navigable map, thereby achieving a unified closed loop of learning control and geometric feasibility. Existing pure learning-based navigation strategies may produce predicted trajectories that deviate from navigable areas, get too close to obstacles, or experience local oscillations in complex environments, resulting in theoretically achievable but poorly executed performance. This invention introduces BEV feasible domain constraints to post-process and optimize the local waypoints output by the controller, making the final executed waypoints more consistent with the navigability conditions and kinematic requirements of the real environment. Therefore, this invention can reduce collision risk, improve path smoothness, execution stability, and motion safety, and enhance the deployment reliability of the system in complex scenarios and on actual robot platforms, ensuring that the navigation results are not only "correct in direction" but also "walkable, executable, and more robust."

[0042] 5. This invention employs topological intent to conditionally modulate the local reactive controller, rather than directly specifying the robot's specific actions, turns, or complete trajectory. Therefore, it can provide global directional guidance while preserving the rapid response capability of the local control strategy to environmental changes. Compared to strongly constrained path tracking or rigid sub-target-driven schemes, this invention exhibits better flexibility in the face of dynamic disturbances, local obstacles, changes in the visible area, and perceptual uncertainties, avoiding a decrease in local obstacle avoidance capabilities or rigid movements due to overly strong global commands. Thus, this invention achieves a better balance between global guidance and local response, considering both navigation efficiency and practical robustness.

[0043] 6. This invention possesses excellent modularity and scalability. The topological intent, serving as a unified interface connecting high-level guidance information and low-level reactive control, can be flexibly generated from various sources. It can originate from global planning on a known object-level topological map, or from human instructions, semantic rules, knowledge base reasoning results, or autonomous exploration heuristics. Therefore, this invention does not rely on a single information source or fixed scenario assumptions, facilitating integration with existing navigation systems, embodied intelligence systems, task planning systems, and multimodal semantic understanding modules. Consequently, this invention is not only applicable to current object navigation tasks but can also be extended to various mobile robot application scenarios with high-level task guidance requirements, demonstrating strong universal adaptability and promising industrial application prospects. Attached Figure Description

[0044] Figure 1 This is a schematic flowchart of the method of the present invention; Detailed Implementation

[0045] The following detailed description of the specific execution process of the reactive object navigation method based on topological intent guidance, using a preferred embodiment of the present invention, is provided. This embodiment is applicable to target object navigation tasks for indoor mobile robots in partially observable environments. The mobile robot may include a chassis, a vision acquisition unit, a depth acquisition unit or monocular depth estimation unit, a calculation unit, and a motion execution unit. The vision acquisition unit is used to acquire environmental images at the current moment; the calculation unit is used to perform object instance extraction, image association, global planning, topological intent calculation, local waypoint prediction, and feasibility correction; and the motion execution unit is used to control the robot's movement based on the corrected waypoints.

[0046] In this embodiment, the system can be divided into two parts: an offline phase and an online phase. The offline phase mainly completes the construction of the object topology map and the training of the local reactive controller; the online phase, while the robot is performing navigation tasks, cyclically performs current observation association, global planning, topology intent generation, local waypoint prediction, feasibility correction, and robot control until the robot reaches the target node or meets the target arrival conditions.

[0047] In some embodiments, the target arrival condition can be set as follows: the planar distance between the robot's current position and the object corresponding to the target node is less than a preset threshold; or the robot detects a target category object in its visual observation and its confidence level is higher than a preset threshold; or the robot is located within the observation area of ​​the target node and meets the stable detection condition for several consecutive frames. The above arrival determination methods can be used individually or in combination.

[0048] In this embodiment, the local reactive controller is preferably an object-relative waypoint prediction network. Its input includes at least an object layout cost map and a topological intent vector generated based on the current observations, and its output is a local waypoint in the robot coordinate system. The local reactive controller can be trained offline using simulator data before deployment. After training, it can be used in a fixed manner during the online navigation phase, or it can be updated incrementally with a small number of parameters while keeping the backbone parameters fixed.

[0049] S1: Construct an object topology map, which is represented as graph G=(N,E), where N represents the set of object instance nodes in the environment and E represents the set of connecting edges between object instances.

[0050] In step S1, a mapping process is first performed on the navigation environment to generate an object topology map that can be used for subsequent global planning and intent calculation. The object topology map is represented by a graph structure, denoted as G=(N,E), where N is a set of nodes, each node in the set corresponds to an object instance observation in the environment, and E is a set of edges used to describe the spatial connectivity and temporal correspondence between object instances.

[0051] Specifically, the robot moves in the environment along a predetermined mapping trajectory or an autonomous exploration trajectory, and continuously acquires RGB images at multiple locations. In some embodiments, depth images, camera pose information, and timestamp information are also acquired simultaneously; in other embodiments, if the robot only has a monocular camera, depth information can be obtained through an existing monocular depth estimation model, and camera pose can be obtained through visual odometry, SLAM, feedforward pose estimation networks, or other mature localization methods.

[0052] For each frame of an RGB image, an open-vocabulary or open-category object segmentation model is first used to segment the image into instances, extracting multiple object instances from the current image. The object segmentation model can be a base model, such as SAM2, FastSAM, or other models with instance segmentation capabilities. For each object instance, its two-dimensional binary mask, bounding box, pixel area, class label or class description, confidence score, and mask centroid coordinates are extracted. The two-dimensional binary mask can be denoted as (…). The centroid of the mask can be calculated from the mean, geometric center, or median position of all pixel coordinates within the mask.

[0053] Subsequently, the 3D global coordinates of each object instance are calculated. Specifically, a depth representative value is first calculated based on the depth value of the region corresponding to the instance's mask. This depth representative value can be the median, mean, or mean of the depths within the mask region after removing outliers, or the depth value at the centroid. Let the centroid pixel coordinates of this instance in the image coordinate system be (( Its depth value is ()). The camera intrinsic parameter matrix is ​​( Then, we can first calculate the three-dimensional point of the centroid of the instance in the camera coordinate system: Then, based on the current frame's camera pose transformation matrix ( Transform it to the world coordinate system to obtain the three-dimensional global coordinates of the node: In some embodiments, to improve stability, instead of using a single centroid point directly, multiple valid pixels within the mask area can be back-projected, and their geometric center or passable projection point can be taken as the three-dimensional coordinates of the node.

[0054] Therefore, each object instance can be represented as a node. In this embodiment, each node can store at least the following attributes: node number, image frame number, and two-dimensional binary mask. ), bounding box, category information, confidence level, 3D global coordinates ( ), two-dimensional ground plane projection coordinates, observation timestamps, and path cost cache related to subsequent planning.

[0055] After the nodes are generated, the edge set E is constructed. The edges include two types: edges within the same frame and edges across frames.

[0056] In this context, "frame-in-frame edges" represent the spatial adjacency relationships between multiple object instances within the same observed image. Specifically, Delaunay triangulation is performed on the centroids of all object instances in the frame, and undirected edges are established between adjacent centroids based on the triangulation results. For each frame-in-frame edge, the planar Euclidean distance or 3D Euclidean distance between the corresponding two object nodes can be used as the edge weight. Additional information such as edge length, common-view relationship, and relative direction can also be added. The advantage of using Delaunay triangulation is that it can construct sparse and stable local topological connectivity relationships without relying on a dense geometric map.

[0057] Cross-frame edges are used to characterize the correspondence between nodes that may correspond to the same physical object at different observation times. Specifically, for candidate frames recalled by adjacent time frames, nearby location frames, or location re-identification, local feature matching algorithms are first used to establish pixel or keypoint-level correspondences between images. For example, SuperPoint is used to extract keypoints and descriptors, and then LightGlue is used to perform keypoint matching. Subsequently, based on whether the matched keypoints fall into the mask region of each object instance, the number of matching points, matching ratio, or overlap relationship between different instance pairs are counted. When the matching degree between a pair of instances reaches a preset threshold, it is determined that the two belong to the same physical object and are repeated observations at different times, and cross-frame edges are established between their corresponding nodes.

[0058] In this embodiment, for cross-frame edges belonging to the same physical object, the edge weight is preferably set to zero or close to zero to indicate that the edge does not introduce additional path cost. In this way, when performing shortest path planning in the subsequent process, the path length will not be artificially increased due to repeated observations of the same object in the time dimension, thereby ensuring that the planning result reflects the topological evolution relationship of the object's layout structure itself.

[0059] After completing the above processing, the object topology map G=(N,E) is obtained. In some embodiments, to improve query efficiency, a node index, an inverted index from image frames to nodes, an index from target categories to candidate nodes, and an adjacency table or pre-computed distance table for shortest path acceleration can also be established for the graph.

[0060] S2: Establish associations between object instances in the currently observed image and nodes in the object topology map G to obtain the associated node set. and on the object topology map with Global planning is performed with each node as the starting point and the target node as the ending point to obtain a set of topological distances. After encoding, the object layout cost map is obtained, and the sub-target nodes are calculated simultaneously. and the next hop node .

[0061] In step S2, the robot is in online navigation mode. First, the vision acquisition unit acquires the current observation image and extracts object instances from the current image using the same or compatible instance segmentation method as in step S1. Each currently observed object instance serves as a query object instance, used to establish associations with historical nodes in the map.

[0062] To improve association accuracy, in some embodiments, location re-identification is first performed to retrieve a set of candidate images most similar to the current observation from the map images saved during the mapping phase. Location re-identification can be implemented using image retrieval models, global descriptor matching models, or existing lightweight location identification algorithms, outputting several candidate map image numbers. Subsequently, instance-level matching is performed only within the map nodes corresponding to these candidate images, thereby reducing matching complexity and improving accuracy.

[0063] The specific method of instance-level matching is as follows: Local keypoints and descriptors are extracted from the current observed image and candidate map images respectively to establish keypoint matching; the matched keypoints are projected onto their respective instance masks, and matching statistics between the query instance and the map instance are established based on the keypoints falling into the masks. For each currently observed object instance, the map instance node with the highest matching score and above a threshold is selected as its associated node; if multiple candidate nodes have similar scores, multiple candidate matching results can be retained. All successfully associated map nodes form the associated node set (…). ).

[0064] In other embodiments, the correspondence between the currently observed object instance and map nodes can also be established using mask features, instance-level semantic embedding, image patch features, or hybrid matching methods. This invention does not limit this approach, as long as a set of nodes associated with the map in the current observation can be obtained.

[0065] After obtaining the node set ( After that, based on the target category or target object cues, the target node is determined in the object topology map. Specifically, if a specific target object instance is given in the task, the node corresponding to that instance can be directly designated as the target node; if a target category is given in the task, all nodes belonging to that target category can be searched on the map, and one or more of the most likely target nodes can be selected as the planning endpoint. In a single-target implementation, typically one target node is selected ( ( ) as the endpoint.

[0066] Next, on graph G, with ( Each node in the array is the starting point, and the target node is the endpoint. Perform global planning with the destination as the endpoint. Dijkstra's shortest path algorithm is preferred, but A* algorithm can also be used. Floyd or other graph search algorithms. The planning results include: each query node ( ) to the target node ( The shortest path length () ), and the corresponding shortest path sequence ( ).

[0067] Furthermore, the set of topological distances corresponding to the currently visible query nodes is mapped back to the current image plane to generate an object layout cost map. Specifically, for each currently observed object instance, based on the nodes associated with it on the map ( ), obtain its topological distance scalar to the target node ( Then, encoding is performed on the scalar. In this embodiment, a fixed sinusoidal position encoding method is preferably used to map a single scalar distance into a multi-dimensional vector:

[0068]

[0069] in( The preset frequency parameter is ). Subsequently, this multidimensional encoding is spread across the pixel mask region corresponding to the current instance and written into the corresponding image channel. All instances are superimposed to form a dense object layout cost map. This cost map uses instance masks as the spatial carrier and topological distance encoding as the semantic content, enabling the local controller to perceive the relative priority of currently visible objects to the target while maintaining the relative representation of objects.

[0070] In this step, it is also necessary to calculate the sub-target nodes ( ) and the next hop node ( First, based on the current set of associated nodes ( The topological distance between each node in the array is used to select the node with the smallest topological distance as the sub-target node. This means that among the objects currently visible and successfully associated with the map, the one that is closest to the target from the perspective of the topological map is selected as the local reference object that is most likely to advance navigation at the current moment.

[0071] Then, obtain the sub-target node ( ) to the target node ( The shortest path sequence:

[0072]

[0073] Since the same physical object may correspond to multiple nodes in different observation frames, and the topological distance values ​​of adjacent nodes in the path may be the same, subsequent nodes in the path cannot be simply used as the basis for intent. To avoid this problem, this embodiment defines the "next hop node" as the first node on the path that strictly reduces the topological distance, i.e.:

[0074]

[0075] In other words, along the globally optimal path from ( The search proceeds towards the target node, and the first node encountered whose topological distance is strictly less than the distance to the current child target node is the next hop node. This design can eliminate the distance equivalence caused by repeated observations of the same physical object, thereby extracting subsequent nodes that truly represent the direction of topological progress.

[0076] After this step is completed, at least the following results will be output for use in subsequent steps: the current set of associated nodes ( ), topological distances from each associated node to the target, object layout cost graph, sub-target nodes ( ) and the next hop node ( ).

[0077] S3: Based on the direction of the next-hop node relative to the robot's current pose, generate a topological intent vector z. This topological intent vector represents the guidance information at the current moment in the direction of decreasing topological distance.

[0078] In step S3, based on the next-hop node obtained in step S2 ( And the robot's current pose, calculate the topological intent vector ( The topological intent vector is used to express, in low-dimensional form, the guiding information that "at the current moment, we should move in the direction of decreasing topological distance".

[0079] Specifically, first obtain the next hop node ( ) three-dimensional global coordinates ( ), and project it onto the ground plane to obtain two-dimensional plane coordinates ( The robot's current pose can be represented as its planar position in the world coordinate system. and heading angle ( ). Wherein, the heading angle ( It can be provided by an inertial measurement unit, odometry, visual positioning module, SLAM module or other pose estimation module.

[0080] Next, calculate the direction angle from the robot's current position to the next hop node in the world plane coordinate system:

[0081]

[0082] Then transform the orientation angle to the robot's own coordinate system to obtain the intended deflection angle relative to the current heading: To ensure angular continuity, ( Normalization to Or other preset ranges.

[0083] Finally, the deflection angle ( Represented as a unit vector:

[0084]

[0085] This yields the two-dimensional topological intention vector. The advantage of using unit vector representation is that it avoids the angle value itself being... ) and( This addresses the discontinuity issue in the neighborhood and facilitates subsequent mapping to feature modulation parameters via a multilayer perceptron.

[0086] In some embodiments, the intention vector can still be calculated using this method when the robot's current pose is obtained through coarse localization and is noisy, because this invention is not sensitive to precise meter-level path tracking; what is needed is only a relative directional offset sufficient to characterize the global topology descent direction. In other words, the main role of topology intention is to provide the local controller with directional guidance on "which side is more likely to achieve global progress," rather than directly specifying precise trajectory points.

[0087] In other embodiments, if the next-hop node is too close to the robot or the orientation calculation is unstable, a smoothing mechanism can be set, such as performing an exponential moving average on the intent vectors of multiple consecutive frames, or (…). An upper limit is set on the rate of change of ) to enhance control stability.

[0088] S4: Use the object layout cost map described in S2 as the main input to the local reactive controller; simultaneously, use the topology intent vector generated in step S3 as conditional guidance information to input the local reactive controller, modulating the internal feature representation of the controller to predict local waypoints.

[0089] In step S4, the system uses the object layout cost map generated in step S2 and the topology intent vector generated in step S3 to jointly drive the local reactive controller to predict local waypoints.

[0090] First, the object layout cost map is input into the target encoder or cost map encoder in the controller. This encoder typically consists of a convolutional network, a lightweight feature extraction network, or other two-dimensional feature extraction modules. Its function is to extract spatial features relevant to the current navigation decision from the object-level topological distance distribution. After encoding, an intermediate feature map is obtained. ,in,( ) represents the number of channels, ( ) and( ) represent the height and width of the feature map, respectively.

[0091] Then, the topology intent vector calculated in step S3 ( The input is fed into a small multilayer perceptron. This multilayer perceptron can contain one or more fully connected layers, nonlinear activation layers, and normalization layers to map the two-dimensional intention vector to modulation parameters consistent with the number of feature channels. Specifically, the multilayer perceptron outputs two sets of channel-level parameters. ,in,( ) represents the channel scaling factor, ( ) represents the channel offset coefficient.

[0092] Next, the intermediate feature map ( Perform channel-by-channel linear modulation, i.e., conditional injection using the FiLM method:

[0093]

[0094] in,( The image shows the modulated feature map. This operation means that instead of directly overwriting the original spatial features or rigidly replacing the controller output, it scales and translates the response strength of each channel, allowing the topological intent to exert a soft guidance on local control decisions at the feature level. This allows the controller to inherit the sensitivity of the original object's relative control model to local observations, while also inducing it to choose action directions that better align with the global topological descent trend.

[0095] In this embodiment, the FiLM module is preferably inserted after the target encoder and before global pooling. The reason for this insertion position is that, on the one hand, the feature map still retains spatial structure information at this time, and the intended guidance can affect the response of different spatial regions; on the other hand, FiLM only modulates intermediate features, does not change the front-end object-level cost map construction process, and does not directly interfere with the mapping relationship of the final output layer, thus having good compatibility.

[0096] Modulated feature map ( The data is then fed into the controller's subsequent pooling layer, fusion layer, and regression head, where the regression head predicts one or more local waypoints. These local waypoints are preferably defined in the robot's coordinate system, for example, the output is (...). ),in( ) represents the distance component in the robot's forward direction, The ) represents the offset component in the lateral direction. In some embodiments, a sequence of future waypoints may be predicted to improve trajectory smoothness; in other embodiments, only the first waypoint is taken as the short-term execution target.

[0097] For system training, the local reactive controller can be trained offline before deployment. Training data can be automatically generated by a simulator, for example, using the Habitat-Matterport3D dataset and corresponding navigation benchmarks. The simulator calculates the geodesic path from the origin to the target, and then discretizes or interpolates it into a supervised waypoint sequence. In the training samples, each frame observation corresponds to a current object layout cost map, a topological intent vector obtained through global object-level planning, and a real waypoint label derived from expert trajectories. The controller can be optimized using a waypoint prediction loss function consistent with that of the base object relative to the controller.

[0098] In some preferred embodiments, to ensure compatibility with the pre-trained base model, the FiLM layer can be initialized with a near-identical transformation, i.e., initialized with ( ), ( This ensures that the strategy initially closely resembles the behavior of the original controller, and then gradually learns to use topological intentions for correction during training. Furthermore, a phased training strategy can be adopted: first, freeze the main parameters of the original controller and train only the relevant parameters of FiLM; after FiLM has learned to stably utilize intention information, then fine-tune the entire network together. This strategy helps improve training stability and reduce performance degradation.

[0099] S5: Based on the bird's-eye view of the passable map, perform feasibility correction on the local waypoints to obtain the corrected execution waypoints.

[0100] In step S5, geometric feasibility correction is performed on the local waypoints predicted in step S4 to ensure that the final control command not only has global topology consistency, but also meets the passage constraints in the local environment.

[0101] First, a local bird's-eye view traversable map, i.e., a BEV traversable map, is constructed centered on the robot's current location. This map can be obtained as follows: if the robot has a depth camera, the current depth map can be back-projected into a local 3D point cloud, and the point cloud can be projected onto a ground plane grid; if the robot only has an RGB camera, the depth can be obtained first through a monocular depth estimation module, and then geometric projection can be performed. Based on ground height, obstacle height, ground normal, occupancy probability, or semantic category, the ground plane grid can be divided into traversable and impassable areas, ultimately obtaining a local BEV traversable map.

[0102] In this embodiment, the BEV traversability map can be represented as a two-dimensional grid matrix, where each grid cell corresponds to a local planar region and carries a traversability marker, cost value, distance transformation value, or safety margin value. The map coordinate origin is typically set at the center of the robot chassis, with the forward direction as the positive x-axis and the left or right direction as the y-axis. Using the known grid resolution and coordinate mapping relationship, local waypoints can be mapped from the robot coordinate system to the BEV grid coordinates.

[0103] Specifically, let the local waypoint predicted in step S4 be ( First, ( Map the location to the corresponding grid position on the BEV map and check if the location belongs to a passable area. If the location is passable, directly use the waypoint as the execution waypoint; if the location falls into an obstacle zone, an unknown zone, or is too close to an impassable boundary, feasibility correction is required.

[0104] Feasibility correction can be achieved in one or more of the following ways.

[0105] The first approach is to project along the original waypoint direction to the nearest feasible location. Specifically, this can be done by starting from the robot's origin and moving along (…). A ray search is performed in the indicated direction to find the nearest passable grid cell to the original waypoint. The coordinates corresponding to the center of this grid cell are used as the corrected execution waypoint. This method can preserve the motion direction predicted by the original controller as much as possible.

[0106] The second approach is to search for feasible locations within the neighborhood of the original waypoint. Specifically, a local search window can be constructed around the original waypoint, and all candidate grids that meet the passability conditions can be searched within the window. The grids are scored based on indicators such as distance from the original waypoint, consistency with the original direction, and safe distance from obstacle boundaries. The candidate point with the highest score is selected as the corrected execution waypoint.

[0107] The third approach is to introduce a safety cost map or a centerline cost map. By performing distance transformation, dilation erosion, or fuzzy smoothing on the passable map, passable locations that are far from the obstacle boundary and located in the center of the passage are prioritized to obtain a smoother and safer execution trajectory.

[0108] In some embodiments, the modified execution waypoint can also be obtained through a weighted fusion of the original predicted point and the best feasible neighboring point, in order to improve geometric feasibility while preserving intent-oriented characteristics. For example:

[0109]

[0110] in,( ) represents the safe and feasible point obtained through the search. () represents the preset weight. Adjusting () controls the balance between "intent guidance" and "geometric safety".

[0111] It should be noted that the feasibility correction module in this step is a deterministic safety layer during execution. It does not change the network structure and internal parameters of the local reactive controller in step S4, nor does it participate in global planning. It only performs lightweight corrections on candidate waypoints under geometric constraints after the output layer, thereby achieving the division of labor and collaboration between semantic topology decision-making and geometric executability.

[0112] S6: Control the robot's movement according to the waypoints, and repeat steps S2 to S6 until the target node is reached.

[0113] In step S6, the robot is controlled in a closed loop based on the execution waypoints obtained in step S5, and steps S2 to S6 are repeated until the target arrival conditions are met.

[0114] Specifically, let the corrected execution waypoint be ( The control module can generate speed control commands for the robot chassis based on the waypoint, such as linear velocity. ) and angular velocity ( In a preferred implementation, control can be achieved based on the distance and deflection angle of the waypoints:

[0115]

[0116] Then, speed commands are generated based on the proportional control law:

[0117]

[0118] in,( ) and( The control gain is denoted as . It can also be implemented using pure tracking control, MPC control, PID control, or a dedicated differential chassis controller. This invention does not limit this approach.

[0119] After the robot moves for a short period of time according to the current control command, it acquires new environmental images again and re-executes steps S2 (image association and global planning), S3 (topology intent generation), S4 (local waypoint prediction), and S5 (feasibility correction). Since this process is performed once in each control cycle, the present invention constitutes a closed-loop navigation process of "perception—association—planning—intent generation—local control—safety correction—execution".

[0120] In some embodiments, the control cycle can be set to a fixed frequency, such as 5Hz, 10Hz, or higher; in other embodiments, it can also be adaptively adjusted based on computing resources and chassis dynamic characteristics. To prevent abnormal situations, several protection mechanisms can be set, such as: entering an exploration or in-situ search mode when a valid node association cannot be established for multiple consecutive frames; triggering relocation or global replanning when waypoints remain stagnant for a long time or the path length does not decrease; and reducing linear speed or performing an emergency stop when a significant increase in collision risk is detected.

[0121] Once the target arrival condition is met, the loop execution of steps S2 to S6 stops, and a navigation completion signal is output. In some embodiments, the navigation trajectory, success flag, cumulative path length, execution time, node access sequence, and intermediate intent record may also be output simultaneously for system evaluation or task log storage.

Claims

1. A reactive navigation method for object layout based on topological intent guidance, characterized in that, Includes the following steps: S1: Construct an object topology map, which is represented as graph G=(N,E), where N represents the set of object instance nodes in the environment and E represents the set of connecting edges between object instances; S2: Establish associations between object instances in the currently observed image and nodes in the object topology map G to obtain the associated node set. and on the object topology map with Global planning is performed with each node as the starting point and the target node as the ending point to obtain a set of topological distances. After encoding, the object layout cost map is obtained, and the sub-target nodes are calculated simultaneously. and the next hop node ; S3: Generate a topological intent vector based on the orientation of the next-hop node relative to the robot's current pose. The topological intent vector is used to characterize the guidance information in the direction of decreasing topological distance at the current moment; S4: Use the object layout cost map described in S2 as the main input of the local reactive controller; at the same time, use the topology intent vector generated in step S3 as conditional guidance information to input the local reactive controller, modulate the internal feature representation of the controller, and predict local waypoints. S5: Based on the bird's-eye view of the passable map, perform feasibility correction on the local waypoints to obtain the corrected execution waypoints; S6: Control the robot's movement according to the waypoints and repeat steps S2 to S6 until the target node is reached.

2. The object layout reactive navigation method based on topological intent guidance as described in claim 1, characterized in that: The node attributes in the object topology map include at least the two-dimensional segmentation mask and the three-dimensional global coordinates of the object instance; The edges in the object topology map include intra-graph edges between different object instances co-occurring in the same image, and inter-graph edges between the same object instances at different times; wherein, the intra-graph edges are used to represent the spatial adjacency relationship between object instances, and the inter-graph edges are used to represent the temporal association relationship of the same object instance.

3. The object layout reactive navigation method based on topological intent guidance as described in claim 2, characterized in that: In step S2, the sub-target node Determined by the following formula: ； in, This represents the set of nodes obtained by associating the currently observed object instances with the object topology map. Represents a node Topological distance to the target node This indicates the current sub-target node.

4. The topology-intention-guided object layout reactive navigation method as described in claim 1, 2, or 3, characterized in that: In step S2, the next hop node Determined by the following formula: ； in, This represents the sub-objective node obtained from global planning. To the target node The sequence of nodes on the shortest path, Indicates the next hop node.

5. The object layout reactive navigation method based on topological intent guidance as described in claim 4, characterized in that: In step S2, the generation of the object layout cost map includes: encoding the set of topological distances from each object instance in the current observation to the target node. The encoding method of the topological distance set is any one of fixed sinusoidal position encoding, learnable embedding encoding, or a combination of both, to obtain the encoded topological distance set; the encoded topological distance set is projected and forms a dense object layout cost map within the two-dimensional segmentation mask coverage area of ​​the corresponding object instance.

6. The object layout reactive navigation method based on topological intent guidance as described in claim 5, characterized in that: In step S3, the topological intent vector is calculated from the direction of the next-hop node relative to the robot's current local coordinate system, and is expressed as: ； in, Represents the topological intent vector. This represents the angle between the robot's current orientation and the direction of the next hop node. This indicates transpose.

7. The object layout reactive navigation method based on topological intent guidance as described in claim 6, characterized in that: In step S4, the local reactive controller introduces the topological intent vector through feature linear modulation and conditionally modulates the intermediate features extracted from the object layout cost map. The modulation form is as follows: ； in, Indicates that the local controller is in the channel Spatial location The original features of the place, Indicates the characteristics after modulation. Represented by the topological intent vector The generated scaling factor, Represented by the topological intent vector The generated bias coefficient.

8. The topology-intention-guided object layout reactive navigation method as described in claim 1, 2, 3, 5, 6, or 7, characterized in that: In step S5, the feasibility correction of the local waypoints based on the bird's-eye view traversable map includes: directly executing when the predicted local waypoint is located in a traversable area; and projecting or adjusting the predicted local waypoint to an adjacent traversable area when it is located in an obstacle area or an impassable area, so as to maintain the consistency of the local movement direction and meet the geometric executability constraints.

9. A topological intention-guided object layout reactive navigation device, characterized in that, include: Object topology map construction unit: used to construct an object topology map, which is represented as graph G=(N,E), where N represents the set of object instance nodes in the environment, and E represents the set of connecting edges between object instances; Global Planning and Cost Graph Generation Unit: Used to establish associations between object instances in the currently observed image and nodes in the object topology map G, to obtain a set of associated nodes. and on the object topology map with Global planning is performed with each node as the starting point and the target node as the ending point to obtain a set of topological distances. After encoding, the object layout cost map is obtained, and the sub-target nodes are calculated simultaneously. and the next hop node ; Topology Intent Generation Unit: Used to generate a topology intent vector based on the orientation of the next-hop node relative to the robot's current pose. The topological intent vector is used to characterize the guidance information in the direction of decreasing topological distance at the current moment; Local control unit: used to take the object layout cost map described in S2 as the main input of the local reactive controller; at the same time, the topology intent vector generated in step S3 is used as conditional guidance information to input the local reactive controller to modulate the internal feature representation of the controller in order to predict local waypoints; Feasibility correction unit: used to perform feasibility correction on the local waypoints based on the bird's-eye view navigable map, and obtain the corrected execution waypoints; Execution control unit: used to control the robot's movement according to the execution waypoints, and to repeatedly execute the above steps S2 to S6 until the target node is reached.

Images