Contextual memory enhanced robot navigation method and system based on visual graph and attention mechanism

Abstract

This invention discloses a context-memory-enhanced robot navigation method and system based on a view and an attention mechanism. The method includes: acquiring sensor observation data to generate a local environment representation; constructing a view based on the local environment representation, where nodes are associated with spatial coordinates and historical visit counts, and edges are collision-free straight lines; inputting node features into an encoder-decoder network, outputting a probability distribution of candidate nodes; selecting the node with the highest probability as the next waypoint and controlling the robot's movement. By constructing a view associated with context-memory information as the environment representation and utilizing an attention mechanism to fuse the spatial coordinates and historical visit counts of nodes, the efficiency of navigation decision-making and path stability are improved, solving the problems of low computational efficiency and path oscillation in existing end-to-end planning methods.

Description

Technical Field

[0001] This invention relates to the field of robot navigation, and in particular to a context-memory-enhanced robot navigation method and system based on visualization and attention mechanisms. Background Technology

[0002] With the expanding applications of mobile robots in industrial manufacturing, warehousing and logistics, outdoor inspection, military reconnaissance, and planetary exploration, autonomous navigation technology has become a research hotspot in related fields. In practical deployments, robots typically need to move safely and efficiently from their starting position to their target location, relying solely on onboard sensors to perceive their surroundings in real time, without a pre-defined map. The core challenge of this task lies in the fact that the robot must simultaneously perform environmental exploration, path planning, and real-time obstacle avoidance within a partially observable environment, while ensuring computational efficiency meets the real-time requirements of online decision-making.

[0003] For the tasks described above, existing technical solutions can be mainly divided into two categories: traditional planning methods and learning-based methods. Traditional planning methods are represented by search-based algorithms, such as A... Algorithm, D Algorithms such as D* Lite, LPA*, and FAR Planner exist. These methods divide the environment into uniform grids or construct a topological map, using heuristic functions to search for globally or locally optimal paths within the map. These methods are structurally clear and highly interpretable, capable of generating globally optimal paths in known or structured environments. However, in unknown or large-scale complex environments, traditional planning methods suffer from the following shortcomings: First, the resolution of the grid map is directly coupled to the scale of the environment; high resolution significantly increases computational and storage overhead, making it difficult to meet real-time requirements on embedded computing platforms. Second, heuristic search strategies are prone to path oscillations and deadlocks in local dead ends or symmetrical obstacle areas, potentially causing the robot to repeatedly traverse the same area without escape. Third, these methods typically make strong assumptions about the geometric regularity of the environment; when faced with unstructured terrain or irregular obstacle distributions, path quality and planning success rates significantly decrease.

[0004] Learning-based methods represent another rapidly developing technological approach in recent years. These methods learn navigation strategies directly from sensor observation data using deep neural networks, reducing reliance on manually designed rules and pre-built maps. Typical methods include deep reinforcement learning-based approaches such as NavRL, YOPO, CADRL, and CTSAC, as well as end-to-end planners based on convolutional neural networks, such as iPlanner. Local reinforcement learning navigation methods primarily focus on short-range obstacle avoidance and action selection, avoiding the computational overhead of global search; hybrid methods first generate intermediate targets using a classical planner, then execute local navigation using a reinforcement learning strategy; and history-enhanced perception methods utilize short-term observation sequences to assist current decisions, mitigating local oscillations. However, these learning-based methods still face several key technical bottlenecks in practical applications. First, existing end-to-end planners mostly use convolutional neural networks as the perception backbone. The local receptive field of convolutional operations limits the network's ability to capture global spatial dependencies. In large indoor spaces, narrow corridors, or scenarios with complex obstacle distributions, robots often generate overly conservative or even infeasible paths. Secondly, existing methods typically use fixed-size depth images or grid maps as input, which cannot dynamically expand as the exploration range increases, making it difficult to maintain global consistency in long-distance, multi-room navigation tasks. Thirdly, most learning methods have limited environmental memory, relying only on short-term observation sequences or latent states to aid decision-making, lacking effective utilization of long-term historical access information. This leads to robots frequently re-exploring already visited areas in long-term tasks, resulting in path oscillations or getting trapped in local optima. Furthermore, the sequential processing flow of "perception-mapping-planning" in traditional modular methods also exists in some hybrid learning methods. Small errors in the upstream perception or localization modules are propagated and amplified step-by-step to the downstream planning module. Simultaneously, data transmission and sequential processing between modules introduce non-negligible system latency, reducing the robot's real-time response in dynamic environments. Finally, learning-based navigation methods generally face difficulties in migrating from simulation environments to real-world scenarios. Due to the distribution differences between simulated depth images or point clouds and real sensor data, deploying trained strategies to real robots often requires extensive fine-tuning and manual calibration, increasing system deployment costs and debugging cycles.

[0005] In summary, existing technologies cannot simultaneously meet the comprehensive requirements of low computational overhead, global environment understanding, long-term memory utilization, path oscillation avoidance, and seamless transition from simulation to reality in mobile robot autonomous navigation tasks. There is an urgent need for a real-time, robust, and scalable end-to-end autonomous navigation method that can achieve autonomous navigation in unknown and complex environments. Summary of the Invention

[0006] The purpose of this invention is to provide a context-memory-enhanced robot navigation method and system based on a view and an attention mechanism. By constructing a view with associated context memory information as an environmental representation, and using an attention mechanism to perform end-to-end fusion and global reasoning on the spatial coordinates and historical visit counts of nodes, the decision-making efficiency and path stability of long-range navigation in unknown environments are improved. This solves the problems of low computational efficiency and path oscillation caused by the lack of global spatial dependency modeling capabilities and long-term historical information utilization mechanisms in existing end-to-end planning methods.

[0007] To address the aforementioned technical problems, a first aspect of this invention provides a context-memory-enhanced robot navigation method based on visibility and attention mechanisms, comprising the following steps: S1: Acquire sensor observation data of the robot's surrounding environment and generate a local environment representation; S2: Construct a view based on the local environment representation. The view consists of a set of nodes and a set of edges. The nodes represent key passable locations extracted from the local environment representation. Each node is associated with spatial coordinates and context memory information. The context memory information is used to record the historical number of visits to the corresponding node. The edges represent passable connections between two nodes that do not intersect with obstacle areas. The node features of the view include at least the spatial coordinates and the context memory information. S3: Input the node features of the visible view into an encoder-decoder network based on an attention mechanism. The encoder fuses the spatial coordinates and the context memory information to obtain global environment features. The decoder uses the node features where the robot is currently located as a query to output the probability distribution of candidate nodes. S4: Select the node with the highest probability in the candidate node probability distribution as the next waypoint, and control the robot to move to the next waypoint.

[0008] Furthermore, the context memory information mentioned in step S2 is used to record the historical access count of the corresponding node, and its updating and utilization include the following sub-steps: S21: Monitor whether the robot has reached the node corresponding to the next waypoint; S22: When it is detected that the robot has reached the node corresponding to the next waypoint, the number of historical visits to the node is increased by a preset value to encode the historical action information into the state space; S23: The spatial coordinates and the number of historical visits are concatenated into a node feature vector, which is then input into the encoder. The encoder calculates the correlation weight between any two node feature vectors through an attention mechanism. The correlation weight is simultaneously affected by the geometric distance between the spatial coordinates and the numerical difference between the number of historical visits. The encoder outputs a global environmental feature that integrates spatial relationships and historical visit experience.

[0009] Further, in step S23, the spatial coordinates and the number of historical visits are concatenated into a node feature vector, which is then input into the encoder. The encoder calculates the correlation weight between any two node feature vectors using an attention mechanism. This correlation weight is simultaneously affected by the geometric distance between the spatial coordinates and the numerical difference between the number of historical visits. The output is a global environmental feature that integrates spatial relationships and historical visit experience, including: S231: The feature matrix composed of the feature vectors of all nodes is used as input. The feature vector of each node is composed of spatial coordinates, distance to the target location and the number of historical visits. S232: The feature matrix is ​​encoded through a multi-head self-attention mechanism. For any two nodes, the attention weight is calculated based on the combined effect of the geometric distance between the spatial coordinates of the two nodes and the numerical difference in the number of historical visits, and the global environmental features are output. S233: Using the feature vector of the robot's current node as the query and the global environment features as the key and value, calculate the correlation between the robot's current node and each candidate node through an attention mechanism; S234: Generate a probability distribution pointing to each candidate node based on the correlation.

[0010] Furthermore, after constructing the view based on the local environment representation in step S2, the process also includes: S241: Define a preset range centered on the robot's current position as a non-sparse region, and retain the complete information of all nodes within the non-sparse region; S242: For nodes located outside the non-sparse region, a clustering algorithm based on spatial distance is used to merge them to generate representative nodes, which are the nodes that replace the merged nodes after merging. S243: Combine the representative node with the nodes in the non-sparse region to obtain a sparsified view, and input the node features of the sparsified view as the node features of the view in step S3 into the encoder-decoder network.

[0011] Further, step S242, which involves fusing nodes located outside the non-sparse region using a spatial distance-based clustering algorithm to generate representative nodes, includes: S2421: Calculate the spatial distance between any two nodes located outside the non-sparse region; S2422: Merge multiple nodes whose spatial distance is less than a preset distance threshold into a representative node, wherein the spatial coordinates of the representative node are the centroid coordinates of all merged nodes; S2423: The representative node inherits the edges of all merged nodes and the context memory information of all merged nodes.

[0012] Furthermore, before constructing the view based on the local environment representation in step S2, the method further includes: S201: In the local environment representation, the local environment representation has distinguished between known free regions and unknown regions, identified the boundary points between the known free regions and the unknown regions, and used the boundary points as candidate points for frontier exploration; S202: Cluster the candidate frontier exploration points to obtain frontier exploration nodes; S203: For each candidate node, calculate the number of frontier exploration nodes contained within a preset range centered on the candidate node, and use the number as the node utility value of the candidate node, and use the node utility value as one of the input features of the encoder-decoder network.

[0013] Further, the clustering of the candidate frontier exploration points to obtain frontier exploration nodes in step S202 includes: S2021: Points that satisfy the condition of being located within a known free region and having an unknown region in their neighborhood are marked as candidate points for frontier exploration; S2022: Cluster the frontier exploration candidate points that are adjacent and whose spatial distance is less than the preset merging threshold; S2023: Select the centroid position of the spatial coordinates of all frontier exploration candidate points in each cluster as the frontier exploration node.

[0014] Furthermore, the encoder-decoder network based on the attention mechanism described in step S3 is trained using a reinforcement learning method, wherein the reward signal used to update the network parameters in the reinforcement learning method is generated by calculating at least one of the following reward terms: The target distance reward is calculated based on the change in distance between the robot's current position and the target position, and is used to encourage the robot to move towards the target position; Frontier exploration reward, which is calculated based on the number of frontier exploration nodes associated with the nodes reached by the robot, is used to incentivize the robot to explore unknown areas; A position change reward, calculated based on the amount of position change between two consecutive decision cycles of the robot, is used to prevent the robot from stalling. Stagnation penalty, which is triggered when the robot's position change is continuously lower than a preset change threshold within a preset time period, is used to reduce path oscillations caused by local optima; A completion reward is given when the robot successfully reaches the target location, which is used to mark the completion of the navigation task.

[0015] Furthermore, the reinforcement learning method employs the soft actor-critic algorithm, and the training process of the soft actor-critic algorithm includes the following sub-steps: S101: Initialize the robot's starting position and target position in the simulation environment; S102: Execute the action output by the current policy network, and interact with the simulation environment to obtain new observation data and the reward signal; S103: Store the new observation data, the reward signal, and the action as a set of interactive data in the experience playback buffer; S104: Randomly sample a batch of interaction data from the experience replay buffer, and use the batch of interaction data to update the parameters of the policy network and the value network respectively. The policy network is used to output actions, and the value network is used to evaluate the value of actions.

[0016] Accordingly, a second aspect of the present invention provides a context-memory-enhanced robot navigation system based on visualization and attention mechanisms, which navigates a robot using the aforementioned context-memory-enhanced robot navigation method based on visualization and attention mechanisms, including: The data acquisition module is used to acquire sensor observation data of the robot's surrounding environment and generate a local environment representation; A visualization construction module is used to construct a visualization based on the local environment representation. The visualization consists of a set of nodes and a set of edges. The nodes represent key passable locations extracted from the local environment representation. Each node is associated with spatial coordinates and context memory information. The context memory information is used to record the historical visit count of the corresponding node. The edges represent passable connections between two nodes that do not intersect with obstacle areas. The node features of the visualization include at least the spatial coordinates and the context memory information. The encoding and decoding module is used to input the node features of the visual image into an encoder-decoder network based on an attention mechanism. The encoder fuses the spatial coordinates and the context memory information to obtain global environmental features, and the decoder uses the node features where the robot is currently located as a query to output the probability distribution of candidate nodes. The navigation control module is used to select the node with the highest probability in the candidate node probability distribution as the next waypoint and control the robot to move to the next waypoint.

[0017] Accordingly, a third aspect of the present invention provides an electronic device, including: at least one processor; and a memory connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to cause the at least one processor to perform the above-described context memory-enhanced robot navigation method based on visibility and attention mechanisms.

[0018] Accordingly, a fourth aspect of the present invention provides a computer-readable storage medium having computer instructions stored thereon, which, when executed by a processor, implement the above-described context memory-enhanced robot navigation method based on visibility and attention mechanisms.

[0019] The above-described technical solutions of the embodiments of the present invention have the following beneficial technical effects: 1. By directly associating the historical visit count of a node with the contextual memory information of each node in the visual image, and concatenating the historical visit count with spatial coordinates to form a node feature vector, the network is input into an encoder-decoder network based on an attention mechanism. This allows the network to be influenced by both spatial geometric distance and historical visit frequency when calculating the correlation weights between nodes. As a result, long-term action experience is internalized into the intrinsic attributes of the environment representation, which improves the robot's ability to identify explored areas and the global stability of path selection in long-distance, multi-room navigation tasks. This effectively solves the path oscillation and repeated exploration problems caused by the lack of a long-term historical information utilization mechanism in existing end-to-end planning methods. 2. By introducing asymmetric graph sparsification centered on the robot's current position after the visualization is constructed, complete information of all nodes is retained in the near non-sparse regions, and a spatial distance-based clustering algorithm is used to merge multiple adjacent nodes into representative nodes in the far regions, inheriting the edge and context memory information of the merged nodes. This makes the number of nodes in the input encoder-decoder network controllable as the environment expands, improving the computational efficiency and real-time performance of navigation reasoning in large-scale environments. It also solves the problems of linear growth in computational load and insufficient real-time response capability of embedded platforms caused by the expansion of the exploration range in traditional dense environment representation. 3. By identifying the boundary points between known free regions and unknown regions from the local environment representation before constructing the visual map and clustering them to generate frontier exploration nodes, and then calculating the number of frontier exploration nodes contained within a preset range centered on each candidate node as the node utility value, the node utility value, along with spatial coordinates and context memory information, is used as the input features of the encoder-decoder network. This allows the network to simultaneously weigh the spatial accessibility, historical access frequency, and exploration value of unknown regions when outputting the probability distribution of candidate nodes. This improves the dynamic balance between exploration efficiency and goal orientation in unknown environments and solves the problem of decision lag and reduced task completion rate caused by the need for separation and coordination of exploration behavior and goal approach behavior through independent modules or complex reward designs in existing navigation methods. Attached Figure Description

[0020] Figure 1 This is a flowchart of the context memory-enhanced robot navigation method based on visibility and attention mechanism provided in this embodiment of the invention; Figure 2 This is a schematic diagram of the sensor observation data processing and local environment representation generation process provided in the embodiments of the present invention; Figure 3 This is a schematic diagram illustrating the process of constructing a view based on a local environment representation provided in an embodiment of the present invention; Figure 4 This is a flowchart illustrating the process of updating and utilizing contextual memory information provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of the internal processing flow of the encoder-decoder network based on the attention mechanism provided in the embodiment of the present invention; Figure 6 This is a schematic flowchart of the asymmetric graph sparsification process provided in an embodiment of the present invention; Figure 7 This is a block diagram of a context-memory-enhanced robot navigation system based on visibility and attention mechanisms, provided in an embodiment of the present invention.

[0021] Figure label: 1. Data acquisition module; 2. Visualization construction module; 3. Encoding / decoding module; 4. Navigation control module. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments and the accompanying drawings. It should be understood that these descriptions are merely exemplary and not intended to limit the scope of the invention. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concept of the invention.

[0023] Please refer to Figure 1 The first aspect of this invention provides a context-memory-enhanced robot navigation method based on a view and an attention mechanism. By constructing a view associated with context-memory information as an environmental representation, and utilizing an attention mechanism to perform end-to-end fusion and global reasoning on the spatial coordinates and historical visit counts of nodes, the method improves decision-making efficiency and path stability for long-range navigation in unknown environments. This solves the problems of low computational efficiency and path oscillation in existing end-to-end planning methods due to the lack of global spatial dependency modeling capabilities and long-term historical information utilization mechanisms. The method includes the following steps.

[0024] S1: Acquire sensor observation data of the robot's surrounding environment and generate a local environment representation.

[0025] In navigation tasks in unknown environments, the robot first acquires real-time observation data of its surrounding space using its onboard environmental perception sensors, including but not limited to lidar, depth cameras, or red-green-blue depth cameras. When lidar is used as the sensing device, it emits laser beams at a preset scanning frequency and receives reflected signals to obtain the three-dimensional spatial position of each sampling point in the robot's surrounding environment relative to the sensor coordinate system, forming point cloud data. This point cloud data describes the surface contours of obstacles, wall boundaries, passage openings, and the extent of open spaces. After filtering, downsampling, and outlier removal of the original point cloud data, it is uniformly transformed to the robot's body coordinate system or global navigation coordinate system through coordinate transformation. When using a depth camera or red-green-blue depth camera, the sensor simultaneously acquires color and depth images of the environmental scene. Each pixel in the depth image records the distance from the corresponding scene point to the camera's imaging plane. Based on the camera's intrinsic parameter matrix, the depth image can be converted into a three-dimensional point cloud, or the depth values ​​can be directly projected onto the robot's local coordinate system to obtain spatial occupancy information of the surrounding environment. Based on the processed sensor data, the robot divides the spatial region within its current observable range into three categories: areas where the sensors have confirmed the absence of obstacles and are passable are marked as known free regions; areas where the sensors have detected obstacles or impassable structures are marked as obstacle regions; and areas beyond the effective detection range of the sensors or not yet effectively observed are marked as unknown regions. This local environment representation is continuously updated in each perception cycle, providing basic spatial information for subsequent visualization construction.

[0026] like Figure 2As shown, the robot's environmental perception sensors scan the surrounding space to acquire raw point cloud data or depth images. After preprocessing operations such as filtering, downsampling, outlier removal, and coordinate transformation, the environmental space is divided into three categories: known free regions, obstacle regions, and unknown regions, forming a local environmental representation. This local environmental representation is continuously updated in each perception cycle as new observation data is acquired, providing the current environmental spatial state information for subsequent steps.

[0027] A significant advantage of this method at the sensor data level lies in its universal adaptability to various sensor types and robot forms. The construction of the visualization relies solely on the division of known free regions, obstacle regions, and unknown regions within the local environment representation, without depending on the raw data format of any specific sensor. Both LiDAR point cloud data and RGB depth camera depth data can be processed through the aforementioned preprocessing steps to generate a unified local environment representation. Furthermore, the nodes and edges in the visualization only describe traversable locations in space and their connections; waypoints are output as spatial coordinates, decoupled from the robot's specific kinematic model. Therefore, it is applicable to various robot forms, including wheeled robots, quadrupedal robots, and drones, without requiring the redesign of navigation strategies for different robot platforms.

[0028] S2: Construct a view based on the local environment representation. The view consists of a set of nodes and a set of edges. Nodes represent key passable locations extracted from the local environment representation. Each node is associated with spatial coordinates and context memory information. The context memory information is used to record the historical number of visits to the corresponding node. Edges represent passable connections between two nodes that do not intersect with obstacle areas. The node features of the view include at least spatial coordinates and context memory information.

[0029] After obtaining the local environment representation, the robot does not directly search for a path on the entire dense grid map. Instead, it extracts key traversable locations with navigation decision significance from the local environment representation and constructs a visual structure composed of a set of nodes and a set of edges. Each node in the node set corresponds to a key traversable location, such as the robot's current location, the target location of the navigation task, the forward exploration location at the boundary between the known and unknown regions, and topological key locations such as corners or passage entrances in free space. Unlike traditional visual representations that only record the spatial coordinates of nodes, this method also associates each node with contextual memory information. This contextual memory information is used to record the historical number of visits to the corresponding node, directly embedding the temporal dimension of action experience into the spatial dimension of the environment representation. This makes the environment representation not only describe the geometric topology of the space but also inherently contain the navigation experience accumulated by the robot in that space. After generating candidate nodes, visibility detection is performed on any two nodes: if the line segment connecting the two nodes is entirely within the known free area and does not cross obstacle areas, an edge is established between them, representing a collision-free straight line that can be connected; if the connecting line segment crosses obstacle areas or unknown areas, no connection is established. To further ensure path safety, obstacle areas can be expanded before visibility detection; if the minimum distance between the connecting line segment and the expanded obstacle boundary is less than a preset safety threshold, edge establishment is also rejected. The resulting visibility structure preserves the spatial topological connectivity of the environment in a sparse manner, and each node carries node features including at least spatial coordinates and contextual memory information, providing a compact and information-rich input for subsequent neural network inference.

[0030] like Figure 3 As shown, based on the local environment representation, key passable locations are extracted from the known free region and its boundary with the unknown region as candidate nodes. Each node is associated with spatial coordinates and context memory information. Visibility detection is used to determine whether there is a passable straight line connection between any two nodes that does not intersect with the obstacle region. If so, an edge is established. Finally, a visual structure composed of a set of nodes and a set of edges is formed, which preserves the spatial topological connectivity of the environment in a sparse manner.

[0031] It is worth emphasizing that the visibility map in this method is not a static structure built offline all at once, but rather dynamically and incrementally updated during navigation as the robot moves and new sensor observation data is acquired. In each decision cycle, new candidate nodes are generated based on the latest local environment representation, and edge connections between new nodes and existing nodes are established through visibility detection. The context memory information of visited nodes is continuously accumulated and updated. This incremental update mechanism allows the visibility map to dynamically expand as the exploration range increases, supporting continuous autonomous navigation in unknown environments, rather than being limited to known, fixed environments. Furthermore, because a local grid map is generated only in the current frame and destroyed after node extraction, memory usage remains essentially constant and does not increase linearly with the expansion of the exploration range, providing a fundamental guarantee for real-time navigation in large-scale environments.

[0032] S3: Input the node features of the visual image into the encoder-decoder network based on the attention mechanism. The encoder fuses spatial coordinates and context memory information to obtain global environment features. The decoder uses the node features of the robot's current location as the query to output the probability distribution of candidate nodes.

[0033] After constructing the current view, the node features of the view are input into an encoder-decoder network based on an attention mechanism for global reasoning and decision-making. The encoder-decoder network adopts a Transformer architecture, with self-attention computation as its core operating mechanism. Specifically, the encoder receives a feature matrix composed of feature vectors from all nodes as input. Each node's feature vector is constructed by concatenating its spatial coordinates and contextual memory information, where the historical visit count recorded in the contextual memory information represents the degree to which the node has been repeatedly visited during past navigation. The encoder calculates the correlation weight between any two node feature vectors using a multi-head self-attention mechanism. This calculation process ensures that the correlation strength between two nodes depends not only on their geometric distance in space but also on the difference in their historical visit counts, thus simultaneously fusing spatial topological relationships and accumulated temporal experience when extracting global environment features. After encoding, the decoder uses the feature vector of the robot's current node as the query and the global environment features output by the encoder as the key and value, calculating the correlation score between the robot's current node and all candidate nodes within the view through a cross-attention mechanism. This mechanism enables the robot to simultaneously perceive the global node distribution pattern and its current position when making decisions, rather than relying solely on local neighborhood information. Subsequently, the pointer network generates a probability distribution pointing to each candidate node based on the correlation score; a higher probability value indicates that the candidate node is more suitable as the next waypoint.

[0034] S4: Select the node with the highest probability in the candidate node probability distribution as the next waypoint and control the robot to move to the next waypoint.

[0035] After obtaining the probability distribution of candidate nodes, the node with the highest probability value is selected as the next waypoint, and its spatial coordinates are output to the robot's underlying motion control module. Upon receiving the waypoint coordinates, the underlying motion control module generates local motion commands based on the relative position between the robot's current pose and the waypoint, driving the robot to move along a safe path towards the waypoint. During its movement towards the waypoint, the robot continuously collects new observation data through sensors. When the robot reaches the vicinity of the current waypoint or meets preset replanning conditions, the above steps are repeated: regenerating the local environment representation, updating the visual structure and node features, and determining a new waypoint through network inference, forming a closed-loop perception and decision-making process. This process continues iteratively until the robot reaches the target position or meets the task termination conditions.

[0036] The steps S1 to S4 described above together constitute a complete end-to-end navigation architecture. Unlike traditional modular methods, this method integrates environmental perception, map construction, feature encoding, and waypoint decision-making into a unified neural network processing flow. Sensor observation data is directly mapped to waypoint outputs, eliminating the need for separate data transfer interfaces between perception, mapping, and planning modules. This end-to-end architecture eliminates the problem of upstream module errors being propagated and amplified to downstream modules in traditional modular methods, and avoids system delays introduced by data format conversion and sequential processing between modules, enabling the robot to respond quickly in dynamic environments.

[0037] By constructing a visual representation with associated contextual memory information as the environment, and internalizing long-term action experience into the inherent attributes of the spatial topology by utilizing the historical visit count of nodes, an encoder-decoder network based on an attention mechanism is used to perform end-to-end global fusion and reasoning of the spatial coordinates and contextual memory information of nodes. This allows the robot to simultaneously consider spatial accessibility and historical visit experience in every waypoint selection in an unknown environment, thereby significantly reducing path oscillation and repeated exploration in long-range navigation tasks. At the same time, replacing dense grids with sparse graph structures for decision-making calculations effectively reduces the computational overhead in large-scale environments and improves the real-time performance and path stability of autonomous navigation in unknown environments.

[0038] The following section further explains the context memory enhancement mechanism of this method by combining the specific updating and utilization of context memory information in step S2.

[0039] S21: Monitor whether the robot has reached the node corresponding to the next waypoint.

[0040] As the robot moves along the currently determined waypoint, its actual position is continuously compared and monitored with the node position corresponding to the next waypoint. This monitoring is achieved by calculating Euclidean distance; when the Euclidean distance between the robot's current position and the target node is less than a preset threshold, the robot is determined to have reached that node. This monitoring mechanism discretizes the continuous robot trajectory into access events to visible nodes, providing clear time-triggered conditions for updating contextual memory information. In actual navigation scenarios, a node in the visible view may correspond to topologically critical locations such as corridor intersections, room doorways, or passable corners next to obstacles. As the robot continuously moves from the previous node to the current waypoint, it inevitably passes through the arrival status of that node, triggering subsequent memory update operations.

[0041] S22: When the robot is detected to have reached the node corresponding to the next waypoint, the number of historical visits to that node is increased by a preset value to encode the historical action information into the state space.

[0042] When the robot detects that it has reached the node corresponding to the next waypoint, the historical visit count associated with that node is increased by a preset value, which is a fixed positive integer. In one implementation, the preset value is one. Increasing the historical visit count encodes the frequency of the robot passing through that node in the current navigation task into the state space of the visual view. This ensures that the characteristics of each node include not only its geometric position in space but also its temporal access experience. The core function of this update mechanism is to provide a basis for distinguishing between explored and unexplored areas in subsequent attention calculations: if a node or area has been visited multiple times during historical navigation but the robot has still failed to reach the target location through that area, then the historical visit count of the corresponding node in that area is high, indicating that the area may be a dead end or a path that cannot lead to the target; conversely, nodes that have not been visited or have a low visit count correspond to areas that the robot has not yet fully explored or that have potential traversal value. The above mechanism has significant advantages in long-distance multi-room navigation scenarios. For example, when a robot performs a task in an indoor environment containing multiple rooms with similar structures, if it enters the same room multiple times due to the complex distribution of local obstacles, the historical access count of the corresponding visible node of the room will gradually accumulate and increase, providing a hint for subsequent decisions that the area has been fully explored and may have no effective exit.

[0043] S23: Concatenate the spatial coordinates and the number of historical visits to form a node feature vector, which is then input into the encoder. The encoder calculates the correlation weight between any two node feature vectors through an attention mechanism. The correlation weight is affected by both the geometric distance of the spatial coordinates and the numerical difference of the number of historical visits. The output is a global environmental feature that integrates spatial relationships and historical visit experience.

[0044] After updating the historical visit count, the spatial coordinates of each node and its corresponding historical visit count are concatenated into a node feature vector, which is then input into the encoder. Before concatenation, the spatial coordinates and historical visit counts are normalized to ensure that features of different dimensions are within a similar numerical range, thus guaranteeing the numerical stability of the attention calculation. The encoder receives a feature matrix composed of the feature vectors of all nodes as input and calculates the relevance weight between any two nodes using the attention mechanism. This relevance weight is influenced by two factors: firstly, the geometric distance between the two nodes in space; nodes that are closer usually have higher path continuity and corresponding to a larger relevance weight; secondly, the numerical difference in the historical visit counts of the two nodes. If both nodes have high visit counts, it indicates that they belong to frequently explored areas, and the connection between them may have been traversed multiple times without effectively advancing the navigation task. In this case, the attention mechanism will reduce the relative importance of this connection direction in the calculation. Through the fusion calculation of the above dual constraints, the global environmental features output by the encoder not only reflect the spatial topological connectivity of the visible view, but also integrate historical access experience into the understanding of the environmental structure, so that subsequent waypoint selection can spontaneously tend to explore candidate directions that are spatially accessible and have a low historical access frequency, realizing end-to-end fusion of spatial reasoning and experience learning under a unified framework.

[0045] Through the above mechanism, the historical visit count of a node is directly associated with each node in the visual image as contextual memory information. The historical visit count is then concatenated with spatial coordinates to form a node feature vector, which is input into an encoder-decoder network based on an attention mechanism. This allows the network to be influenced by both spatial geometric distance and historical visit frequency when calculating the correlation weights between nodes. As a result, long-term action experience is internalized into an intrinsic attribute of the environment representation, improving the robot's ability to identify explored areas and the global stability of path selection in long-distance, multi-room navigation tasks. This effectively solves the path oscillation and repeated exploration problems caused by the lack of a long-term historical information utilization mechanism in existing end-to-end planning methods.

[0046] like Figure 4 As shown, the process of updating and utilizing contextual memory information includes three sub-steps: First, it monitors whether the robot has reached or is close to the node corresponding to the current waypoint. When the arrival determination is successful, the historical visit count of the node is increased by a preset value. Then, the updated historical visit count and spatial coordinates are concatenated to form a node feature vector, which is input into the encoder. When the encoder calculates the correlation weight between nodes through the attention mechanism, it is simultaneously affected by the difference in spatial geometric distance and historical visit count values, and outputs a global environmental feature that integrates spatial relationships and historical visit experience.

[0047] The following provides a further explanation of the specific processing procedure of the encoder-decoder network in step S23.

[0048] S231: The feature matrix composed of the feature vectors of all nodes is used as input. The feature vector of each node is composed of spatial coordinates, distance to the target location and historical visit count.

[0049] Before inputting the node features of the visual image into the encoder, a feature vector of uniform dimension needs to be constructed for each node. Each node's feature vector is composed of three types of information: the first is spatial coordinates, describing the node's three-dimensional spatial position or two-dimensional planar position in the environment, providing the basis for geometric distance calculation for the attention mechanism; the second is the distance to the target location, i.e., the Euclidean distance between the node and the navigation task target point or the shortest path distance along the edge of the visual image, used to provide the network with the node's target-oriented information in the global task; the third is the historical visit count, i.e., the cumulative number of times the node has been visited by the robot in this navigation task, used to provide the network with historical visit experience in that area. After concatenating these three types of information along the feature dimension, each node's feature vector forms a comprehensive representation carrying spatial positioning information, target guidance information, and temporal experience information. The feature vectors of all visual image nodes are arranged in node order to form a feature matrix, which serves as the encoder input. The number of rows in this feature matrix equals the number of nodes in the current visual image, and the number of columns equals the dimension of a single node's feature vector. Therefore, the encoder can adaptively handle visual image inputs of different sizes without requiring a fixed input size.

[0050] S232: The feature matrix is ​​encoded through a multi-head self-attention mechanism. For any two nodes, the attention weight is calculated by the combined effect of the geometric distance between the spatial coordinates of the two nodes and the numerical difference in the number of visits in the past, and the global environmental features are output.

[0051] The encoder employs a multi-head self-attention mechanism to encode the input feature matrix. This mechanism maps the feature vector of each node in the feature matrix to multiple subspaces through different linear projections. Attention weights are calculated independently in each subspace, and the outputs of each subspace are then concatenated and fused, allowing the network to capture the association patterns between nodes from different perspectives. During the computation of a single attention head, for any two nodes, the encoder calculates an attention weight based on their feature vectors. The magnitude of this weight is determined by the similarity between the feature vectors of the two nodes. Since the node feature vector contains both spatial coordinates and historical visit counts, the attention weight between two nodes is actually influenced by both spatial geometric distance and the numerical difference in historical visit counts: In the spatial dimension, the closer the geometric distance between two nodes, the stronger the continuity on the traversable path, and the higher the similarity of their feature vectors in the spatial part, usually resulting in a larger attention weight; in the temporal dimension, if both nodes have high historical visit counts, it indicates that their region has been explored multiple times but has failed to effectively advance the navigation task. When calculating attention weights, the network will inhibit the connection in that direction and tend to assign higher attention weights to edges connecting to nodes with low visit counts. Through the combined effect of the above two factors in the attention mechanism, the global environmental features output by the encoder not only encode the spatial topological connectivity of the view, but also internalize historical access experience into the understanding of the overall pattern of the environment.

[0052] S233: Using the feature vector of the robot's current node as the query and the global environment features as the key and value, the correlation between the robot's current node and each candidate node is calculated through an attention mechanism.

[0053] After the encoder extracts the global environment features, the decoder uses the feature vector of the robot's current node as the query vector and the global environment features output by the encoder as both the key vector set and the value vector set to perform cross-attention calculation. The core operation of this calculation is to perform an inner product operation between the query vector and each key vector to obtain the relevance score between the robot's current node and each candidate node within the visible view. Since the query vector comes from the robot's current node, it carries information about the robot's current spatial location and historical visit status, while the key and value vectors come from the global environment features, carrying contextual information fused with the spatial topology and historical experience of all nodes in the visible view. Therefore, the calculated relevance score reflects the relative rationality of traveling from the robot's current location to each candidate node, considering the overall global environment pattern. In a typical long corridor, multi-room indoor scenario, if the robot is currently at the doorway of a room that has been visited multiple times, when the decoder calculates the relevance of this node to candidate nodes leading to different directions, since the global environment features have already incorporated the historical visit count information for each candidate direction, candidate nodes leading to unexplored areas will receive higher relevance scores, while candidate nodes leading to repeatedly visited areas will receive relatively lower scores.

[0054] S234: Generate a probability distribution pointing to each candidate node based on the correlation.

[0055] After obtaining the relevance scores between the robot's current node and each candidate node, the pointer network generates a probability distribution pointing to each candidate node based on these relevance scores. The pointer network uses a soft maximization function to normalize the relevance scores, ensuring that the sum of the probability values ​​of all candidate nodes equals one. Each probability value represents the probability of selecting the candidate node as the next waypoint, starting from the current node. This probability distribution is directly output to the waypoint selection module without the need for additional rule filtering or manual threshold judgment. The advantage of the pointer network lies in its adaptive matching of its output dimension to the number of input visible nodes. When new nodes are added to the visible view due to the expansion of the environment exploration range, the pointer network can naturally expand its output dimension, maintaining its decision-making ability for new candidate nodes. Through the encoder's extraction of global environmental features, the decoder's calculation of the relevance between the current node and candidate nodes, and the pointer network's generation of the probability distribution, a complete end-to-end inference path from the visible view structure to the waypoint decision probability is formed.

[0056] By concatenating the spatial coordinates of each node, the distance to the target location, and the number of historical visits into a unified feature vector, and utilizing a multi-head self-attention mechanism to simultaneously perceive the spatial geometric relationship of nodes and the differences in historical visit frequencies during the encoding stage, the decoder performs global relevance calculation with the robot's current node as the query. Finally, the pointer network adaptively generates the probability distribution of candidate nodes, enabling waypoint decision-making to simultaneously complete global topological reasoning and temporal experience fusion within a unified neural network framework. This overcomes the path oscillation and repeated exploration problems caused by the independent processing of spatial planning and historical memory in traditional separate architectures.

[0057] like Figure 5 As shown, the internal processing flow of the encoder-decoder network includes four sub-steps: First, a feature vector composed of spatial coordinates, distance to the target location, and historical visit count is constructed for each node, forming a feature matrix that is input into the encoder; the encoder calculates the attention weight between any two nodes through a multi-head self-attention mechanism, which is simultaneously affected by the spatial geometric distance and the difference in historical visit count, and outputs the global environment features; the decoder uses the feature vector of the robot's current node as the query and the global environment features as the key and value, and calculates the correlation between the current node and each candidate node through the attention mechanism; the pointer network generates a probability distribution pointing to each candidate node based on the correlation, completing the end-to-end inference from the visible structure to the waypoint decision probability.

[0058] The following explains the asymmetric graph sparsity processing mechanism used to control computational complexity in this method. After the initial construction of the view, as the number of nodes in the view continues to grow with the expansion of the environment exploration range, directly inputting all nodes into the encoder-decoder network causes the computational cost of attention calculation to increase quadratically with the number of nodes, affecting the real-time performance of navigation decisions. To solve this problem, this method introduces the following graph sparsity processing step after constructing the view.

[0059] S241: Define a preset range centered on the robot's current position as a non-sparse region, and retain complete information of all nodes within the non-sparse region.

[0060] First, a predefined spatial range centered on the robot's current position is defined as the non-sparse region. This non-sparse region is typically a circular area centered on the robot's current coordinates and defined by a predefined radius, or a rectangular boundary area centered on the robot's current coordinates. The predefined radius or rectangular side length is determined comprehensively based on the robot's effective sensor detection range and real-time requirements. The non-sparse region contains all nodes in the robot's current local environment. These nodes are directly related to the robot's current obstacle avoidance decisions and local path selection. Therefore, it is necessary to retain complete information about all nodes within this region, including each node's spatial coordinates, distance to the target location, contextual memory information, and edge connections with adjacent nodes. No nodes within this region are compressed or merged. In one implementation scenario, when the robot is navigating in an indoor corridor environment, corner nodes, doorway nodes, and obstacle edge nodes within a ten-meter radius in front of it are all fully preserved, ensuring that the accuracy of local path selection is not affected by sparsity.

[0061] S242: For nodes located outside the non-sparse region, a clustering algorithm based on spatial distance is used to merge them and generate representative nodes. The representative nodes are the nodes that replace the merged nodes after merging.

[0062] For all nodes located outside the non-sparse region, i.e., distant nodes far from the robot's current position, a spatial distance-based clustering algorithm is used for fusion processing. The clustering algorithm's processing flow is as follows: First, the Euclidean spatial distance between any two nodes located outside the non-sparse region is calculated. Then, multiple nodes with a spatial distance less than a preset distance threshold are grouped into the same cluster. For each cluster, the arithmetic mean of the spatial coordinates of all nodes within the cluster is taken to calculate a centroid coordinate. This centroid coordinate is used as the spatial location of the newly generated node, which is the representative node. The role of the representative node in the view is to replace all the original nodes merged within the cluster, thereby compressing multiple spatially adjacent distant nodes into a single topological unit. The representative node inherits all edge connections between all merged original nodes and other nodes in the view. If one of the merged original nodes has an edge with another unmerged node in the view, then after sparsification, this edge is changed to an edge between the representative node and that unmerged node, thus ensuring that the topological connectivity of the entire view is not destroyed after sparsification. Simultaneously, the representative node inherits the contextual memory information of all the original nodes being merged. In one implementation, the inheritance method is to take the maximum or weighted average of the historical access counts of all the original nodes being merged as the historical access count of the representative node, to ensure that the access experience information of the remote area is not lost after sparsification. In the implementation scenario of an outdoor forest environment, the tree-to-tree passage nodes more than 50 meters away from the robot's current position are usually generated in a dense sampling manner. After the above clustering and fusion processing, multiple adjacent tree-to-tree passage nodes are merged into a topological node representing the forest area, which significantly reduces the number of nodes in the remote area.

[0063] S243: Combine the representative node with the nodes in the non-sparse region to obtain the sparsified view. Use the node features of the sparsified view as the node features of the view in step S3 to input the encoder-decoder network.

[0064] All nodes retaining complete information within the non-sparse region are combined with all representative nodes generated after clustering and fusion outside the non-sparse region to form the node set of the sparsified visual graph. Edges between nodes within the non-sparse region, edges between representative nodes, and edges between nodes in the non-sparse region and representative nodes together constitute the edge set of the sparsified visual graph. This sparsified visual graph retains complete environmental geometric details near the robot, ensuring that the accuracy of local obstacle avoidance and precise waypoint selection is not compromised; further away from the robot, topology compression reduces the node size of the graph structure, keeping the number of nodes input to the encoder-decoder network within a reasonable range. The node features of this sparsified visual graph are used as the node features input to the encoder-decoder network in subsequent steps. As the robot moves, the center position of the non-sparse region is dynamically updated according to the robot's current position. Nodes that were originally located in the non-sparse region may be moved out of the non-sparse region and participate in subsequent clustering and fusion after the robot moves away. Regions that were originally located far away and compressed into representative nodes may be expanded back into complete nodes in the non-sparse region after the robot moves closer. This achieves dynamic adaptive adjustment of the sparsity of the visual view with the robot's position.

[0065] By introducing asymmetric graph sparsification centered on the robot's current position after the visualization is constructed, complete information of all nodes is retained in the near non-sparse regions, and a spatial distance-based clustering algorithm is used to merge multiple adjacent nodes into representative nodes in the far regions, inheriting the edge and context memory information of the merged nodes. This makes the number of nodes in the input encoder-decoder network controllable as the environment expands, improving the computational efficiency and real-time performance of navigation reasoning in large-scale environments. It also solves the problems of linear growth in computational load and insufficient real-time response capability of embedded platforms caused by the expansion of the exploration range in traditional dense environment representation.

[0066] like Figure 6 As shown, the asymmetric graph sparsification process includes three sub-steps: First, a preset range centered on the robot's current position is defined as the non-sparse region, preserving all node information within this region; then, distant nodes outside the non-sparse region are fused using a spatial distance-based clustering algorithm, merging multiple nodes with a spatial distance less than a preset distance threshold into a representative node, which inherits the edge connections and contextual memory information of the merged nodes; finally, the representative node is combined with nodes within the non-sparse region to obtain the sparsified view, and its node features are used as input to the encoder-decoder network. As the robot moves, the non-sparse region is dynamically updated, achieving adaptive adjustment of the view's sparsity.

[0067] The following provides a further explanation of the specific operations of cluster fusion in step S242.

[0068] S2421: Calculate the spatial distance between any two nodes located outside the non-sparse region.

[0069] Before clustering and merging distant nodes outside the non-sparse region, it is necessary to quantify the spatial proximity between any two distant nodes. This requires calculating the Euclidean spatial distance between any two nodes outside the non-sparse region. The Euclidean spatial distance is calculated based on the difference in coordinates of the two nodes in three-dimensional space or a two-dimensional plane. In actual calculations, a distance matrix can be constructed for all nodes outside the non-sparse region. This matrix records the spatial distance between each pair of nodes, serving as the basis for subsequent clustering and merging operations. In large-scale environments, the number of distant nodes outside the non-sparse region may reach hundreds. Although calculating the distance between all node pairs increases quadratically with the number of nodes, the computational cost is far lower than the cost of directly inputting all nodes into the encoder-decoder network for attention calculation, as this calculation only involves simple geometric distance operations and does not involve forward propagation of the neural network. Therefore, it accounts for a very small percentage of the overall computational load.

[0070] S2422: Merge multiple nodes whose spatial distance is less than a preset distance threshold into a representative node. The spatial coordinates of the representative node are the centroid coordinates of all merged nodes.

[0071] Based on the spatial distance between nodes calculated in step S2421, a clustering strategy based on a distance threshold is used to merge distant nodes. The specific operation is as follows: A preset distance threshold is set, which is determined comprehensively based on the environmental scale and navigation accuracy requirements. In indoor environments, it is typically on the order of several meters, while in outdoor environments, it can be appropriately increased. Nodes located outside the non-sparse region are traversed, and multiple nodes with a spatial distance less than the preset distance threshold are grouped into the same cluster. For each cluster, the arithmetic mean of the spatial coordinates of all merged nodes within the cluster is calculated. This arithmetic mean is used as the spatial coordinates (centroid coordinates) of the newly generated representative node. This representative node replaces all the original merged nodes within the cluster, becoming a single topological unit in the sparsed view. Through the above operations, several or dozens of originally densely distributed distant nodes are compressed into a single representative node, significantly reducing the node density in the distant region. In one implementation scenario, when a robot is performing a navigation task in a large underground parking lot, multiple sampling nodes in the corners of parking spaces that are far from the robot's current position are spatially adjacent to each other and have little impact on the robot's current local obstacle avoidance decision. After the above clustering and merging, only one representative node is retained in this area to participate in global topology reasoning, which effectively reduces the number of nodes in the input network.

[0072] S2423: Represents a node that inherits the edges of all merged nodes and the context memory information of all merged nodes.

[0073] After the representative node is generated, it needs to inherit the topological connections and memory information of all the original nodes that were merged in the view, to ensure that the sparsified view still maintains complete connectivity and historical experience information. Regarding topological connections, the representative node inherits all edges connected to all the original nodes that were merged. Specifically, if any original node that was merged has an edge with another unmerged node in the view, then an edge is created between the representative node and that unmerged node in the sparsified view. If two of the merged original nodes have edges with the same unmerged node, then only one edge is retained between the representative node and that unmerged node in the sparsified view to avoid duplicate connections. Regarding contextual memory information, the representative node inherits the historical visit counts of all the original nodes being merged. In one implementation, the inheritance method is to take the maximum historical visit count among all the original nodes being merged as the historical visit count of the representative node, thus preserving the most significant experience markers of the remote region being visited. In another implementation, the inheritance method is to take the weighted average of the historical visit counts of all the original nodes being merged as the historical visit count of the representative node, thus comprehensively reflecting the average visit frequency of the region. Through the above inheritance mechanism, although the remote nodes are spatially compressed and merged, the topological connectivity structure of the original visible view is completely preserved, and the robot's experience information on visiting the region during previous navigation is not lost due to the sparsity operation.

[0074] By calculating the spatial distance to remote nodes outside the non-sparse region and clustering them according to a preset distance threshold, and then using a representative node to inherit all edge connections and context memory information of the merged nodes, the topological connectivity and historical access experience of the remote region are fully preserved while reducing the size of the visible nodes. This significantly reduces the attention computation of the subsequent encoder-decoder network, and the sparsification operation does not introduce topological breaks or memory loss, ensuring an effective balance between the real-time performance and decision quality of navigation decisions on the embedded computing platform.

[0075] The following explains the node utility value mechanism used in this method to guide the robot to actively explore unknown areas. Before constructing the visualization, information about the direction of exploration in the unknown area needs to be provided for subsequent navigation decisions.

[0076] S201: In the local environment representation, the local environment representation has distinguished between known free regions and unknown regions, identified the boundary points between known free regions and unknown regions, and used the boundary points as candidate points for frontier exploration.

[0077] In the local environment representation, the spatial region has been divided into three types: known free region, obstacle region, and unknown region. The known free region is the spatial range that the sensors have confirmed is passable, while the unknown region is the spatial range that the sensors have not yet effectively observed or whose passability status has not been determined. The boundary between the two constitutes the boundary of the current environmental perception. At the boundary between the known free region and the unknown region, all grid cells or spatial points located on one side of the known free region and adjacent to the unknown region are identified, and these boundary points are extracted as candidate points for forward exploration. These candidate points represent all potential exit directions leading to the unexplored region in the robot's current known environment. In a typical indoor navigation scenario, when the robot enters a corridor with multiple unexplored rooms, the doorway area of ​​each unexplored room forms the boundary between the known free region and the unknown region in the local environment representation. The spatial points at these doorways are identified as candidate points for forward exploration, providing the robot with multiple candidate exploration directions.

[0078] S202: Cluster the candidate points for frontier exploration to obtain frontier exploration nodes.

[0079] Because the environmental data acquired by the sensor in each sensing cycle has a certain spatial continuity, adjacent grid cells may be identified as boundary points simultaneously. Therefore, a large number of spatially adjacent front exploration candidate points are usually generated near the boundary of the same unknown region. If all these front exploration candidate points are directly used for subsequent utility value calculation, it will not only increase the computational load but also cause the boundary of the same unknown region to be counted repeatedly, resulting in a lack of distinction between the utility values ​​of different candidate nodes. To solve this problem, spatial clustering is performed on the front exploration candidate points identified in step S201. The clustering process groups adjacent front exploration candidate points with a spatial distance less than a preset merging threshold into the same cluster. For each cluster, the centroid position of the spatial coordinates of all front exploration candidate points within the cluster is calculated, and this centroid position is used as the front exploration node corresponding to the cluster. Through the above clustering operation, the dozens of front exploration candidate points that may have been generated on the boundary of a continuous unknown region are compressed into one or a few front exploration nodes. Each front exploration node represents an independent, spatially distinct unknown region exploration direction. Using the example of the aforementioned indoor corridor scenario, if the width of the doorway of an unexplored room corresponds to multiple adjacent boundary grids in the local environment representation, after clustering, only one leading exploration node is generated for the entire doorway area of ​​the room, serving as the representative position of the exploration direction of the room.

[0080] S203: For each candidate node, calculate the number of frontier exploration nodes contained within a preset range centered on the candidate node, and use the number as the node utility value of the candidate node. The node utility value is used as one of the input features of the encoder-decoder network.

[0081] After obtaining the leading exploration nodes, a quantitative indicator of the exploration value of each candidate node in the visualization is calculated. For any candidate node, a circular or spherical evaluation range with a preset radius centered on the candidate node's spatial coordinates is defined. The total number of leading exploration nodes included within this evaluation range is counted, and this number is used as the node utility value of the candidate node. The physical meaning of the node utility value is: if the robot moves to the location of the candidate node, the number of unknown area exploration directions that can be covered within the effective detection range of its sensors, i.e., the potential contribution of the candidate node to expanding the scope of environmental cognition. The node utility value, along with spatial coordinates and contextual memory information, is used as a component of the node features of the candidate node, participating in attention calculation and waypoint probability distribution generation when subsequently input into the encoder-decoder network. By incorporating the node utility value into the node features, when calculating the correlation between any two nodes, the network not only considers spatial accessibility and historical access experience, but also pays additional attention to the potential value of the candidate node in exploring unknown areas, enabling waypoint decisions to achieve a dynamic trade-off between target approach and exploration of unknown areas based on an attention mechanism. In one implementation scenario, in an indoor environment with multiple unexplored branches, a candidate node located in the middle of a corridor may have a high node utility value because its evaluation range covers multiple forward exploration nodes leading to unexplored rooms. On the other hand, a candidate node located at the end of the corridor, although closer to the target location, has no forward exploration nodes within its evaluation range and has a node utility value of zero. After comprehensively considering spatial distance, historical access count, and node utility value, the network may prioritize the former as the next waypoint in order to obtain more environmental information about unknown areas.

[0082] Before constructing the visual map, this method identifies the boundary points between known free regions and unknown regions in the local environment representation and clusters them to generate frontier exploration nodes. Then, for each candidate node, the number of frontier exploration nodes contained within a preset range centered on it is calculated as the node utility value. The node utility value, along with spatial coordinates and context memory information, are used as input features of the encoder-decoder network. This allows the network to simultaneously weigh the spatial accessibility, historical access frequency, and exploration value of unknown regions when outputting the probability distribution of candidate nodes. This improves the dynamic balance between exploration efficiency and goal orientation in unknown environments and solves the problem of decision lag and reduced task completion rate caused by the separation and coordination of exploration behavior and goal approach behavior through independent modules or complex reward designs in existing navigation methods.

[0083] The following provides a further explanation of the specific operations for generating the frontier exploration node in step S202.

[0084] S2021: Mark points that are located within a known free region and have unknown regions in their neighborhood as candidate points for frontier exploration.

[0085] Based on the local environment representation that distinguishes between known free regions and unknown regions, each grid cell or spatial sampling point in the local environment representation is individually evaluated. Points that simultaneously meet two conditions are selected and marked as candidate points for frontier exploration. The first condition is that the point is located within a known free region, meaning that the spatial location of the point has been confirmed by the sensor as passable and unobstructed. The second condition is that there is at least one point within the unknown region in the neighborhood of the point. That is, within the preset neighborhood range of the point (e.g., eight adjacent grid cells centered on the point or a spherical neighborhood of a preset radius in three-dimensional space), there is at least one spatial point belonging to an unknown region that has not yet been effectively observed by the sensor. Points that simultaneously meet both conditions are located precisely at the spatial interface between the known free region and the unknown region, representing potential exit locations for the robot to extend from the known passable region to the unexplored region. In a real-world deployment scenario, when a robot equipped with a depth camera moves in an indoor office environment, the depth camera has a limited field of view. The interior areas of the room that are not observed by the camera are marked as unknown areas, while the observed corridor areas are marked as known free areas. The two form a clear boundary line at the doorway of the room, and each spatial point located on this boundary line and on one side of the corridor is marked as a candidate point for frontier exploration.

[0086] S2022: Cluster the frontier exploration candidate points that are adjacent and whose spatial distance is less than the preset merging threshold.

[0087] Due to the spatial continuity of sensor observation data, adjacent grid cells often simultaneously satisfy the discrimination criteria for front exploration candidate points, resulting in a densely distributed set of front exploration candidate points near the boundary of unknown regions. If all front exploration candidate points are directly used for subsequent node utility value calculations, the boundary of the same continuous unknown region will be repeatedly represented by a large number of spatially adjacent candidate points, wasting computational resources and reducing the distinguishability between different candidate node utility values. Therefore, all front exploration candidate points marked in step S2021 are clustered based on spatial distance. Specifically, a preset merging threshold is set, determined comprehensively based on the resolution of the environment representation and the robot's geometric dimensions, typically a value equivalent to the robot's own radius or the sensor's minimum resolution distance. All front exploration candidate points are traversed, and spatially adjacent candidate points with a Euclidean distance less than the preset merging threshold are grouped into the same cluster, while candidate points with a spatial distance greater than or equal to the preset merging threshold belong to different clusters. Through the above clustering operation, dozens of frontier exploration candidate points that were originally densely distributed on the boundary of a continuous unknown region were grouped into a few clusters, each cluster corresponding to a spatially independent entrance direction of an unknown region.

[0088] S2023: Select the centroid of the spatial coordinates of all frontier exploration candidate points in each cluster as the frontier exploration node.

[0089] For each cluster generated in step S2022, the arithmetic mean of the spatial coordinates of all frontier exploration candidate points within the cluster is calculated. The spatial position corresponding to this arithmetic mean is used as the coordinate position of the frontier exploration node of that cluster, i.e., the centroid of the cluster is selected as the frontier exploration node. This frontier exploration node is located at the geometric center of the boundary of the unknown region corresponding to the cluster, and can evenly represent the overall spatial distribution of all frontier exploration candidate points on the boundary of the unknown region. Through the above centroid selection method, each continuous unknown region boundary ultimately generates only one or a few frontier exploration nodes. The number of nodes is comparable to the number of independent unknown region entrance directions actually existing in the environment, significantly simplifying the size of the candidate set for subsequent node utility value calculation. Continuing with the aforementioned indoor office environment scenario, after clustering and centroid selection, only one frontier exploration node is generated for the entire doorway area of ​​a room. This node is located exactly at the center of the doorway, and its spatial coordinates can accurately reflect the orientation of the room entrance, providing a clear and definite reference benchmark for the robot to determine the spatial position of the exploration direction.

[0090] By marking points that simultaneously satisfy the conditions of being located within a known free region and having unknown regions in their neighborhood as candidate points for forward exploration, and selecting the centroid as the forward exploration node after spatial clustering of adjacent candidate points, a structured abstraction of the exploration direction of the unknown region boundary is realized. This compresses the dense intersection points on the continuous boundary into a limited number of independent exploration direction nodes, significantly reducing the redundancy of subsequent node utility value calculations while ensuring the spatial representativeness of the exploration direction. This provides a concise and clear input basis for the quantitative evaluation of exploration value in waypoint decision-making.

[0091] The training method of the attention-based encoder-decoder network in this method is described below. In step S3, the attention-based encoder-decoder network is trained using a reinforcement learning method. The reward signal used to update the network parameters in the reinforcement learning method is generated by calculating at least one of the following reward terms.

[0092] The target distance reward is calculated based on the change in distance between the robot's current position and the target position, and is used to encourage the robot to move towards the target position. The target distance reward is calculated based on the change in Euclidean distance between the robot's current position and the target position set by the navigation task before and after the robot performs an action. Within each decision cycle, the first distance between the robot's position before performing a waypoint movement and the target position is recorded, as well as the second distance between the robot's position after performing the action and reaching the new position and the target position. When the second distance is less than the first distance, meaning the robot has effectively moved closer to the target within that decision cycle, a positive reward value is given, and the reward value is positively correlated with the distance reduction; the greater the proximity, the higher the reward value. When the second distance is greater than or equal to the first distance, meaning the robot has failed to effectively approach the target or has even moved further away within that decision cycle, a smaller positive reward value, zero value, or negative penalty value is given. The core function of this reward is to guide the policy network to learn and select waypoints that enable the robot to gradually approach the target position, directly encoding the target orientation into an optimizable numerical signal, so that the robot can maintain basic task orientation between exploring unknown areas and approaching the target position in unknown environments.

[0093] Frontier exploration reward is calculated based on the number of frontier exploration nodes associated with the node the robot reaches, and is used to incentivize the robot to explore unknown areas. The frontier exploration reward is calculated based on the node utility value associated with that node after the robot arrives at it. As mentioned earlier, the node utility value represents the number of frontier exploration nodes within a preset range centered on that node. Frontier exploration nodes are spatial representative points on the interface between known free areas and unknown areas. When the robot moves to a candidate node and completes the arrival determination for that node, it obtains the pre-calculated and stored node utility value for that node. If the node utility value is high, it indicates that the robot can observe or reach the entrance directions of multiple unknown areas from the node's location. This movement significantly expands the robot's cognitive range of the environment, and a positive reward value positively correlated with the node utility value is given. If the node utility value is zero, it indicates that there are no unknown area boundaries to explore around that node, and this movement does not contribute to the expansion of the environmental cognitive boundary; the frontier exploration reward is zero. The purpose of this reward is to incentivize the strategy network to actively select nodes with high exploration value as waypoints during training, preventing the robot from prematurely abandoning the exploration of unknown branch areas in the environment simply because of the reward for the target distance, thereby improving the global task completion rate in complex and unknown environments.

[0094] Position change reward, calculated based on the robot's position change between two consecutive decision cycles, is used to prevent robot stagnation. The position change reward is calculated based on the robot's actual spatial movement between two consecutive decision cycles. At the end of each decision cycle, the Euclidean displacement between the robot's position after executing an action and its initial position before the action is calculated. This displacement is multiplied by a preset small reward coefficient to obtain the position change reward value for that decision cycle. When the robot effectively executes a waypoint movement action and generates actual displacement, the position change reward is positive, and the reward value is proportional to the displacement. When the robot fails to generate effective displacement within a decision cycle due to obstacles, an infeasible path, or invalid actions output by the network, the position change reward is zero. The purpose of this reward is to encourage the policy network to output effective actions that reliably drive the robot to generate spatial displacement from the underlying mechanism, and to suppress stagnation caused by continuously outputting invalid actions.

[0095] Stagnation penalty is triggered when the robot's position change remains below a preset threshold for a predetermined period of time. It is used to reduce path oscillations caused by local optima. Stagnation penalty is a negative reward signal triggered when the robot's position change remains below the preset threshold for a continuous, predetermined time period. In actual deployment, a fixed-length sliding time window is set, during which the robot's total displacement is accumulated. When the total displacement within the time window falls below the preset threshold, the robot is determined to be in a stagnant state, triggering the stagnation penalty with a fixed negative penalty value. The absolute magnitude of this penalty value is set according to the task's tolerance for stagnant behavior. Stagnation typically indicates that the robot may be stuck in a local dead end, surrounded by obstacles, or that the policy network outputs a sequence of actions that causes back-and-forth oscillations. By applying a penalty signal, the policy network is forced to identify such states as negative experiences to be avoided during training. This reward complements the position change reward; the position change reward positively encourages effective movement, while the stagnation penalty negatively inhibits prolonged stagnation. Together, they reduce the probability of the robot getting stuck in local optima and path oscillations in complex environments.

[0096] A completion reward is given when the robot successfully reaches the target position, signifying the completion of the navigation task. The completion reward is a one-time positive reward signal given when the robot successfully reaches the target position set in the navigation task. When the Euclidean distance between the robot's current position and the target position is less than a preset successful arrival threshold, the navigation task is considered complete, and a positive completion reward value significantly larger than other reward items is given. The purpose of this completion reward is to provide the policy network with a sparse but clear task success sign, enabling the policy network to identify the effective action sequence that ultimately achieves the task goal from a large amount of exploration experience during training. The high reward value of the completion reward is then propagated back to earlier decision steps in the task execution process through the value function propagation mechanism in reinforcement learning, guiding the policy network to gradually optimize the complete navigation behavior chain from the starting position to the target position.

[0097] By constructing a multi-objective reward signal system composed of target distance reward, frontier exploration reward, position change reward, stagnation penalty and completion reward, five training signals—target tendency guidance, unknown area exploration incentive, effective movement encouragement, stagnation behavior inhibition and task completion mark—are synergistically optimized within a unified reinforcement learning framework. This enables the trained encoder-decoder network to adaptively balance the decision conflict between moving towards the target and exploring unknown areas in an unknown environment, effectively suppressing path oscillations and local stagnation behaviors, and improving the completion rate of long-range navigation tasks and the convergence stability of policy training.

[0098] Furthermore, the reinforcement learning in this method employs the soft actor-critic algorithm, achieving end-to-end training of the encoder-decoder network parameters through alternating optimization of the policy network and value network. The training process of the soft actor-critic algorithm includes the following sub-steps.

[0099] S101: Initialize the robot's starting position and target position in the simulation environment.

[0100] Before starting reinforcement learning training, the robot's initial state needs to be set in the simulation environment for each training round. Specifically, a starting position and a navigation task target position are randomly generated or generated according to preset rules in the simulation environment. The virtual robot model is placed at the starting position, and the target position coordinates are provided to the robot as the target point information for the navigation task. The simulation environment includes virtual geometric models corresponding to real-world application scenarios, such as indoor corridors, multi-room layouts, underground tunnels, forest paths, or disaster relief scenarios. The spatial topology of the simulation environment, such as obstacle distribution, passage width, and room connectivity, can be randomly configured according to training requirements or generated according to a preset map template. By using diverse combinations of starting and target positions in different training rounds, the policy network can be exposed to rich spatial scenes and navigation task instances during training, avoiding overfitting to specific starting conditions or fixed environmental layouts, thereby improving the generalization ability of the trained policy in unknown real-world environments. In one implementation, the simulation environment is constructed using a three-dimensional physical simulation platform. The robot virtual model is equipped with a simulated lidar or simulated depth camera as a virtual sensor. The sensor simulation parameters include scanning frequency, detection distance, field of view, and noise model to reduce the distribution difference between simulation data and real sensor data.

[0101] S102: Execute the actions output by the current policy network, and interact with the simulation environment to obtain new observation data and reward signals.

[0102] Within each decision cycle, the robot's current state is composed of the node features of the visual view and the features of the node the robot is currently at. The policy network receives the current state as input and outputs an action, which corresponds to the decision to select a node from the candidate node set of the visual view as the next waypoint. The robot's virtual model executes this action, moving towards the selected waypoint node in the simulation environment. During the movement, virtual sensors continuously collect environmental data. Upon reaching the waypoint node, the local environmental representation is updated based on the newly collected sensor data, a new visual view is constructed, and node features are extracted to form new observation data. Simultaneously, based on information such as whether the robot effectively approaches the target position, reaches a node with the coverage of the leading exploration node, generates effective displacement, triggers a stagnation state, and successfully reaches the target position within the decision cycle, a reward signal corresponding to the decision cycle is calculated according to multiple preset reward items. The new observation data, reward signal, and executed action together constitute an interaction record for the decision cycle. This interaction record fully describes the next state and immediate reward obtained by the robot after taking a specific action in the current state, providing single-step experience samples for subsequent parameter updates of the policy network and value network.

[0103] S103: Store the new observation data, reward signals, and actions as a set of interactive data in the experience playback buffer.

[0104] The interaction data generated in each decision cycle—namely, new observation data, reward signals, and executed actions—is stored as a complete interaction data set in the experience replay buffer. This experience replay buffer is a fixed-capacity data storage structure. When the number of interaction data sets stored in the buffer reaches its capacity limit, newly stored data sets will overwrite the oldest data sets in a first-in, first-out (FIFO) order. The purpose of using the experience replay buffer is to randomly recombine temporally generated, highly correlated interaction data sets, breaking the temporal coupling between training samples, thereby meeting the fundamental requirement of independent and identically distributed neural network parameter updates for training samples in deep reinforcement learning. In actual training, the buffer typically stores interaction data generated from multiple different training rounds, different environmental configurations, and different navigation stages simultaneously, ensuring that each batch of data sampled from the buffer contains diverse state transition patterns.

[0105] S104: Randomly sample a batch of interaction data from the experience replay buffer, and use the batch of interaction data to update the parameters of the policy network and the value network respectively. The policy network is used to output actions, and the value network is used to evaluate the value of actions.

[0106] During each parameter update, a batch of interaction data is randomly sampled from the experience replay buffer, with the number of data sets set to a preset batch size. Using this sampled data, the parameters of the policy network and value network are updated respectively. The value network comprises two structurally identical but parameter-independent subnetworks, used to estimate state value and action value, respectively. During training, the value network parameters are updated using gradient descent to minimize temporal difference error, enabling the value network to more accurately assess the expected cumulative reward of taking an action in the current state. The parameter update direction of the policy network is to maximize the action value estimate output by the value network, i.e., to encourage the policy network to output actions that yield higher expected cumulative rewards under the value network's evaluation. Within the soft actor-critic algorithm framework, the optimization objective of the policy network also includes an additional entropy regularization term. This term encourages the action probability distribution output by the policy network to maintain a certain degree of randomness, thereby enhancing the policy's exploration ability in unknown environments and preventing the policy from prematurely converging to a local optimum. By alternately updating the parameters of the value network and the policy network, the policy network gradually learns the ability to select waypoints that balance target approach, exploration of unknown areas, and avoidance of stagnation, given the characteristics of visible nodes and contextual memory information.

[0107] By employing a soft actor-critic algorithm to train the encoder-decoder network end-to-end in a simulation environment, the temporal correlation of interactive data is broken by utilizing an experience replay buffer. Furthermore, a balance is achieved between maximizing expected cumulative reward and maintaining policy exploration capability through alternating optimization of the policy network and value network. This enables the trained network parameters to output robust waypoint decisions when facing unknown real-world environments. No manual annotation of trajectory data is required during training, and the parameter update process of the policy network exhibits convergence stability.

[0108] It is worth further elaborating that the introduction of the context memory mechanism in this method provides an important theoretical guarantee for the convergence of reinforcement learning policies. In partially observable unknown environments, the state space relying solely on current sensor observation data often does not satisfy the Markov property, making it difficult for reinforcement learning algorithms based on value function iteration to guarantee convergence. This method encodes long-term action experience into the state space by using the historical visit counts of nodes, so that the current state not only contains spatial observation information but also historical visit records, and the state space re-satisfies the Markov property. Under this condition, when using the soft actor-critic algorithm for policy training, the temporal difference learning process of the value network has a theoretical guarantee of convergence, overcoming the defect of existing deep reinforcement learning methods in complex unknown environments where policy convergence is unstable due to partial observations.

[0109] This method has been thoroughly validated experimentally in both simulation environments and on real robot platforms. In terms of simulation environments, test scenarios covered various types, including indoor corridors, forests, garages, and underground tunnels. Experimental results show that, compared with existing end-to-end planning methods, the navigation path length generated by this method is reduced by an average of 10% to 20%, the trajectory smoothness score is improved by 5% to 15%, and the curvature variance and jerk root mean square of accelerometers are significantly reduced, effectively lowering the robot's control costs and mechanical wear. Regarding training efficiency, by using sparse visual representations instead of the original depth images as network input, the abstraction level of the graph structure is higher than that of pixel-level representations, reducing the dimensionality of the input space. Simultaneously, contextual memory information provides historical priors for the state space, eliminating the need for the policy network to learn the dynamic characteristics of the environment from scratch. The training convergence speed is improved by approximately 15% to 20% compared to baseline methods based on convolutional neural networks. In terms of real-world deployment, the trained network parameters were directly deployed to a real robot platform without any fine-tuning, successfully completing autonomous navigation tasks in complex indoor environments, validating the method's zero-simulation-to-real-world transfer capability. In terms of real-time performance, after combining graph sparsification processing, the single inference time is about 5 milliseconds in an indoor environment with about 750 visible nodes, and about 10 milliseconds in a forest environment with about 1500 visible nodes. The single complete planning time is about 0.12 seconds, which meets the real-time navigation requirements of embedded computing platforms.

[0110] Accordingly, please refer to Figure 7 A second aspect of this invention provides a context-memory-enhanced robot navigation system based on visualization and attention mechanisms. The system navigates a robot using the aforementioned context-memory-enhanced robot navigation method based on visualization and attention mechanisms, including: Data acquisition module 1 is used to acquire sensor observation data of the robot's surrounding environment and generate a local environment representation; The visualization construction module 2 is used to construct a visualization based on the local environment representation. The visualization consists of a set of nodes and a set of edges. Nodes represent key passable locations extracted from the local environment representation. Each node is associated with spatial coordinates and context memory information. The context memory information is used to record the historical number of visits to the corresponding node. Edges represent passable connections between two nodes that do not intersect with the obstacle area. The node features of the visualization include at least spatial coordinates and context memory information. The encoding and decoding module 3 is used to input the node features of the visual image into the encoder-decoder network based on the attention mechanism. The encoder fuses spatial coordinates and context memory information to obtain global environment features, and the decoder uses the node features of the robot's current location as a query to output the probability distribution of candidate nodes. Navigation control module 4 is used to select the node with the highest probability in the candidate node probability distribution as the next waypoint and control the robot to move to the next waypoint.

[0111] Accordingly, a third aspect of the present invention provides an electronic device, including: at least one processor; and a memory connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to cause the at least one processor to perform the above-described context memory-enhanced robot navigation method based on visibility and attention mechanisms.

[0112] Accordingly, a fourth aspect of the present invention provides a computer-readable storage medium having computer instructions stored thereon, which, when executed by a processor, implement the above-described context memory-enhanced robot navigation method based on visibility and attention mechanisms.

[0113] The embodiments of this invention aim to protect a context-memory-enhanced robot navigation method and system based on visibility and attention mechanisms, which has the following effects: 1. By directly associating the historical visit count of a node with the contextual memory information of each node in the visual image, and concatenating the historical visit count with spatial coordinates to form a node feature vector, the network is input into an encoder-decoder network based on an attention mechanism. This allows the network to be influenced by both spatial geometric distance and historical visit frequency when calculating the correlation weights between nodes. As a result, long-term action experience is internalized into the intrinsic attributes of the environment representation, which improves the robot's ability to identify explored areas and the global stability of path selection in long-distance, multi-room navigation tasks. This effectively solves the path oscillation and repeated exploration problems caused by the lack of a long-term historical information utilization mechanism in existing end-to-end planning methods. 2. By introducing asymmetric graph sparsification centered on the robot's current position after the visualization is constructed, complete information of all nodes is retained in the near non-sparse regions, and a spatial distance-based clustering algorithm is used to merge multiple adjacent nodes into representative nodes in the far regions, inheriting the edge and context memory information of the merged nodes. This makes the number of nodes in the input encoder-decoder network controllable as the environment expands, improving the computational efficiency and real-time performance of navigation reasoning in large-scale environments. It also solves the problems of linear growth in computational load and insufficient real-time response capability of embedded platforms caused by the expansion of the exploration range in traditional dense environment representation. 3. By identifying the boundary points between known free regions and unknown regions from the local environment representation before constructing the visual map and clustering them to generate frontier exploration nodes, and then calculating the number of frontier exploration nodes contained within a preset range centered on each candidate node as the node utility value, the node utility value, along with spatial coordinates and context memory information, is used as the input features of the encoder-decoder network. This allows the network to simultaneously weigh the spatial accessibility, historical access frequency, and exploration value of unknown regions when outputting the probability distribution of candidate nodes. This improves the dynamic balance between exploration efficiency and goal orientation in unknown environments and solves the problem of decision lag and reduced task completion rate caused by the need for separation and coordination of exploration behavior and goal approach behavior through independent modules or complex reward designs in existing navigation methods.

[0114] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0115] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0116] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0117] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0118] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A context-memory-enhanced robot navigation method based on visibility and attention mechanisms, characterized in that, Includes the following steps: S1: Acquire sensor observation data of the robot's surrounding environment and generate a local environment representation; S2: Construct a view based on the local environment representation. The view consists of a set of nodes and a set of edges. The nodes represent key passable locations extracted from the local environment representation. Each node is associated with spatial coordinates and context memory information. The context memory information is used to record the historical number of visits to the corresponding node. The edges represent passable connections between two nodes that do not intersect with obstacle areas. The node features of the view include at least the spatial coordinates and the context memory information. S3: Input the node features of the visible view into an encoder-decoder network based on an attention mechanism. The encoder fuses the spatial coordinates and the context memory information to obtain global environment features. The decoder uses the node features where the robot is currently located as a query to output the probability distribution of candidate nodes. S4: Select the node with the highest probability in the candidate node probability distribution as the next waypoint, and control the robot to move to the next waypoint.

2. The method according to claim 1, characterized in that, The context memory information mentioned in step S2 is used to record the historical access count of the corresponding node, and its updating and utilization include the following sub-steps: S21: Monitor whether the robot has reached the node corresponding to the next waypoint; S22: When it is detected that the robot has reached the node corresponding to the next waypoint, the number of historical visits to the node is increased by a preset value to encode the historical action information into the state space; S23: The spatial coordinates and the number of historical visits are concatenated into a node feature vector, which is then input into the encoder. The encoder calculates the correlation weight between any two node feature vectors through an attention mechanism. The correlation weight is simultaneously affected by the geometric distance between the spatial coordinates and the numerical difference between the number of historical visits. The encoder outputs a global environmental feature that integrates spatial relationships and historical visit experience.

3. The method according to claim 2, characterized in that, In step S23, the spatial coordinates and the number of historical visits are concatenated into a node feature vector, which is then input into the encoder. The encoder calculates the correlation weight between any two node feature vectors using an attention mechanism. This correlation weight is influenced by both the geometric distance between the spatial coordinates and the numerical difference in the number of historical visits. The output is a global environmental feature that integrates spatial relationships and historical visit experience, including: S231: The feature matrix composed of the feature vectors of all nodes is used as input. The feature vector of each node is composed of spatial coordinates, distance to the target location and the number of historical visits. S232: The feature matrix is ​​encoded through a multi-head self-attention mechanism. For any two nodes, the attention weight is calculated based on the combined effect of the geometric distance between the spatial coordinates of the two nodes and the numerical difference in the number of historical visits, and the global environmental features are output. S233: Using the feature vector of the robot's current node as the query and the global environment features as the key and value, calculate the correlation between the robot's current node and each candidate node through an attention mechanism; S234: Generate a probability distribution pointing to each candidate node based on the correlation.

4. The method according to claim 1, characterized in that, After constructing the view based on the local environment representation in step S2, the method further includes: S241: Define a preset range centered on the robot's current position as a non-sparse region, and retain the complete information of all nodes within the non-sparse region; S242: For nodes located outside the non-sparse region, a clustering algorithm based on spatial distance is used to merge them to generate representative nodes, which are the nodes that replace the merged nodes after merging. S243: Combine the representative node with the nodes in the non-sparse region to obtain a sparsified view, and input the node features of the sparsified view as the node features of the view in step S3 into the encoder-decoder network.

5. The method according to claim 4, characterized in that, Step S242, which involves fusing nodes located outside the non-sparse region using a spatial distance-based clustering algorithm to generate representative nodes, includes: S2421: Calculate the spatial distance between any two nodes located outside the non-sparse region; S2422: Merge multiple nodes whose spatial distance is less than a preset distance threshold into a representative node, wherein the spatial coordinates of the representative node are the centroid coordinates of all merged nodes; S2423: The representative node inherits the edges of all merged nodes and the context memory information of all merged nodes.

6. The method according to claim 1, characterized in that, Before constructing the view based on the local environment representation in step S2, the method further includes: S201: In the local environment representation, the local environment representation has distinguished between known free regions and unknown regions, identified the boundary points between the known free regions and the unknown regions, and used the boundary points as candidate points for frontier exploration; S202: Cluster the candidate frontier exploration points to obtain frontier exploration nodes; S203: For each candidate node, calculate the number of frontier exploration nodes contained within a preset range centered on the candidate node, and use the number as the node utility value of the candidate node, and use the node utility value as one of the input features of the encoder-decoder network.

7. The method according to claim 6, characterized in that, Step S202, which involves clustering the candidate frontier exploration points to obtain frontier exploration nodes, includes: S2021: Points that satisfy the condition of being located within a known free region and having an unknown region in their neighborhood are marked as candidate points for frontier exploration; S2022: Cluster the frontier exploration candidate points that are adjacent and whose spatial distance is less than the preset merging threshold; S2023: Select the centroid position of the spatial coordinates of all frontier exploration candidate points in each cluster as the frontier exploration node.

8. The method according to claim 1, characterized in that, The encoder-decoder network based on the attention mechanism described in step S3 is trained using a reinforcement learning method. The reward signal used to update the network parameters in this reinforcement learning method is generated by calculating at least one of the following reward terms: The target distance reward is calculated based on the change in distance between the robot's current position and the target position, and is used to encourage the robot to move towards the target position; Frontier exploration reward, which is calculated based on the number of frontier exploration nodes associated with the nodes reached by the robot, is used to incentivize the robot to explore unknown areas; A position change reward, calculated based on the amount of position change between two consecutive decision cycles of the robot, is used to prevent the robot from stalling. Stagnation penalty, which is triggered when the robot's position change is continuously lower than a preset change threshold within a preset time period, is used to reduce path oscillations caused by local optima; A completion reward is given when the robot successfully reaches the target location, which is used to mark the completion of the navigation task.

9. The method according to claim 8, characterized in that, The reinforcement learning method employs the soft actor-critic algorithm, and the training process of the soft actor-critic algorithm includes the following sub-steps: S101: Initialize the robot's starting position and target position in the simulation environment; S102: Execute the action output by the current policy network, and interact with the simulation environment to obtain new observation data and the reward signal; S103: Store the new observation data, the reward signal, and the action as a set of interactive data in the experience playback buffer; S104: Randomly sample a batch of interaction data from the experience replay buffer, and use the batch of interaction data to update the parameters of the policy network and the value network respectively. The policy network is used to output actions, and the value network is used to evaluate the value of actions.

10. A context-memory-enhanced robot navigation system based on visibility and attention mechanisms, characterized in that, Navigating a robot using the context-memory-enhanced robot navigation method based on visibility and attention mechanisms as described in any one of claims 1-9 includes: The data acquisition module is used to acquire sensor observation data of the robot's surrounding environment and generate a local environment representation; A visualization construction module is used to construct a visualization based on the local environment representation. The visualization consists of a set of nodes and a set of edges. The nodes represent key passable locations extracted from the local environment representation. Each node is associated with spatial coordinates and context memory information. The context memory information is used to record the historical visit count of the corresponding node. The edges represent passable connections between two nodes that do not intersect with obstacle areas. The node features of the visualization include at least the spatial coordinates and the context memory information. The encoding and decoding module is used to input the node features of the visual image into an encoder-decoder network based on an attention mechanism. The encoder fuses the spatial coordinates and the context memory information to obtain global environmental features, and the decoder uses the node features where the robot is currently located as a query to output the probability distribution of candidate nodes. The navigation control module is used to select the node with the highest probability in the candidate node probability distribution as the next waypoint and control the robot to move to the next waypoint.

Images