A Brain-Inspired Navigation Method Based on DSI Decoupling Characterization and Composite Potential Field Path Optimization
Patent Information
- Application Number
- CN202610775148.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-06-01
- Publication Date
- 2026-06-26
Smart Images
Figure CN122281941A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of mobile robot navigation technology, and in particular to a brain-like navigation method based on DSI decoupling characterization and composite potential field path optimization. Background Technology
[0002] In recent years, goal-oriented decision-making methods based on successor representations, cognitive maps, and vector navigation have attracted attention. These methods attempt to establish compact representations based on reachability structures between states and directly drive navigation through the relationships between these representations. However, existing related schemes generally suffer from three drawbacks: First, there is often a lack of direct closed loop between representation learning and geometric path quality, which makes greedy navigation prone to outputting reachable but not optimal jagged paths. Secondly, existing post-processing optimizations typically only make one-time repairs to the path, and the optimization results cannot be back-accumulated into the state representation, making it difficult to form long-term gains that are "more accurate the more they are used". Third, in the presence of obstacles, local geometric distortions, and multiple overlapping navigation tasks, relying solely on local greedy decisions can easily lead to detours, redundant backtracking, and decreased path efficiency. Summary of the Invention
[0003] In view of the above situation, the main objective of this invention is to propose a brain-like navigation method based on DSI decoupling characterization and composite potential field path optimization to solve the above-mentioned technical problems.
[0004] This invention proposes a brain-like navigation method based on DSI decoupling characterization and composite potential field path optimization, the method comprising the following steps: Step 1: Discretize the environment and collect state sequences through random walks; calculate the successor representation matrix using the state sequences; calculate the successor information based on the successor representation matrix and take the positive value to obtain the positive successor information matrix; decompose and reduce the dimensionality of the positive successor information matrix to obtain the DSI representation; based on the DSI representation, approximate the value function with the DSI inner product, select the neighbor with the highest value and exclude the source from the previous step, repeat until the target state is reached to generate a navigation path sequence. Step 2: Define the total potential energy function through the navigation path sequence; use the total potential energy function to iteratively update the node coordinates through gradient descent, while introducing obstacle information, anchoring and freezing nodes that approach obstacles, and using the anchor points to optimize the path segments independently to obtain the optimal path sequence. Step 3: Extract the optimal state sequence from the optimal path sequence as a supervision signal to construct the total correction loss; use the total correction loss to update the DSI representation using gradient descent to obtain the corrected DSI representation; store the corrected DSI representation to obtain the correction experience buffer pool. Step 4: Extract the corrected DSI representation from the calibration experience buffer pool, repeat the online navigation, path optimization and calibration steps, and obtain the optimal brain-like navigation sequence after a preset number of iterations.
[0005] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. This invention constructs a low-dimensional DSI representation that simultaneously characterizes state reachability relationships and goal-oriented value information. This compresses the high-dimensional, dense state transition relationships in the environment into a computationally achievable and storable compact vector representation, enabling the robot to maintain effective encoding of global topology and target orientation information at a lower representation dimension. Compared to making decisions directly in the original state transition space, this invention reduces navigation computational complexity and storage overhead, and improves the compactness and transferability of the navigation representation.
[0006] 2. In the online navigation phase, this invention utilizes the inner product of the DSI representation vector and the target state DSI weight vector to approximate the value function, and quickly generates an initial navigation path based on the value comparison of candidate neighbor states. This eliminates the need for a global search of the entire state graph or progressively unfolding complex planning, thereby improving online decision-making efficiency. This approach enables the robot to quickly select local actions based on the target state, offering advantages such as fast response speed, clear decision-making process, and ease of real-time deployment.
[0007] 3. This invention further introduces a composite potential energy optimization mechanism after the initial path generation, and combines obstacle safety constraints to continuously adjust and segment the path nodes. This ensures that the resulting path, while maintaining consistency between the start and end points, effectively reduces local jagged edges, redundant backtracking, and unnecessary detours, while also preventing the path from crossing obstacles or excessively approaching obstacle boundaries. Therefore, this invention achieves a geometrically smoother, more compact, and safer navigation path, improving the robot's path quality and reliability in complex environments.
[0008] 4. This invention further maps the optimal path output by the path optimization module into an optimal state sequence, and uses this sequence as a supervisory signal to reverse-correct the DSI representation. This ensures that the path optimization results are no longer confined to a one-time post-processing stage, but are embedded in the state representation layer, directly improving the quality of initial decisions in subsequent navigation tasks. Through this mechanism, this invention achieves a closed-loop coupling of "path optimization—representation correction—re-navigation," enhancing the ability of the state representation to characterize high-quality path structures and improving the system's long-term adaptability to target-oriented navigation tasks.
[0009] 5. This invention constructs a correction experience buffer pool and combines experience replay and multi-round iterative update mechanisms to continuously accumulate, reuse, and globally integrate optimization experience obtained from different starting point-target pairs. This allows the DSI representation to continuously approach a better solution in multiple navigation tasks. As iterations proceed, the deviation between the initial path and the optimized path gradually decreases, and the correction required by the path optimization module gradually diminishes, ultimately forming a navigation system that tends towards a steady state. Therefore, this invention not only improves the performance of a single navigation attempt but also enhances the convergence, stability, and continuous optimization capability of the entire navigation system. Attached Figure Description
[0010] Figure 1 This is a flowchart of the brain-like navigation method based on DSI decoupling characterization and composite potential field path optimization proposed in this invention; Figure 2 This is a flowchart of the offline learning and decoupling of DSI representation for brain-like navigation based on DSI decoupling representation and composite potential field path optimization proposed in this invention. Figure 3 This is a flowchart of the path optimization process for brain-like navigation based on DSI decoupling characterization and composite potential field path optimization proposed in this invention. Figure 4 This is a flowchart of the DSI online correction algorithm for brain-like navigation based on DSI decoupling characterization and composite potential field path optimization proposed in this invention. Detailed Implementation
[0011] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.
[0012] These and other aspects of the embodiments of the present invention will become clear from the following description and accompanying drawings. In these descriptions and drawings, some specific embodiments of the present invention are specifically disclosed to illustrate some ways of implementing the principles of the embodiments of the present invention; however, it should be understood that the scope of the embodiments of the present invention is not limited thereto.
[0013] Please see Figure 1 This invention proposes a brain-like navigation method based on DSI decoupling characterization and composite potential field path optimization. The method includes the following steps: Step 1: Discretize the environment and collect state sequences through random walks; calculate the successor representation matrix using the state sequences; calculate the successor information based on the successor representation matrix and take the positive value to obtain the positive successor information matrix; decompose and reduce the dimensionality of the positive successor information matrix to obtain the DSI representation; based on the DSI representation, approximate the value function with the DSI inner product, select the neighbor with the highest value and exclude the source from the previous step, repeat until the target state is reached to generate a navigation path sequence. In step 1, the environment is discretized, and a state sequence is collected through random walks. The successor representation matrix is calculated using the state sequence. Based on the successor representation matrix, subsequent information is calculated and positively evaluated to obtain a positive successor information matrix. The positive successor information matrix is then decomposed and its dimensionality reduced to obtain a DSI representation. Based on the DSI representation, the value function is approximated using the DSI inner product, the neighbor with the highest value is selected, and sources from the previous step are excluded. This process is repeated until the target state is reached to generate a navigation path sequence. The specific steps are as follows: Step 101: The mobile robot executes a random exploration strategy in the environment to obtain a state sequence; Step 102: Calculate the cumulative expected number of visits with discounts in the form of the state sequence. Rewrite the expected number of visits accumulated by discounts from the expected form to the probabilistic form; Based on the properties of Markov chains, in the process of rewriting the cumulative expected number of visits from the expected form to the probabilistic form, the infinite series is written in the form of a matrix to obtain the successor representation matrix. Step 103: Calculate the steady-state access probability of each state under the preset default strategy; Subsequent information is obtained by calculating the steady-state access probability of each state; Within the linear reinforcement learning framework, the optimal function is calculated by utilizing the steady-state access probability and subsequent information of each state. Based on the optimal function, negative correlations in the successor information are filtered out to retain the positive half of the successor information in the successor representation matrix, thus obtaining the positive successor information matrix. Step 104: The positive successor information matrix is projected into a low-dimensional space using constrained nonnegative matrix decomposition to obtain the DSI representation vector and the DSI weight vector. Step 105: Based on the DSI representation vector and DSI weight vector, the inner product of the DSI vectors approximates the value function, and the value function is used to construct navigation decision rules. The mobile robot starts from the current state, repeatedly applies navigation decision rules until it reaches the target state, and records the sequence of states it has passed through to obtain the navigation path sequence.
[0014] For details, please refer to Figure 2In the process of a mobile robot executing a random exploration strategy in the environment to obtain a state sequence, the following relationship exists: ; in, This represents the discrete state sequence collected by the robot during its random walk, i.e., the state sequence. This represents the total number of steps in the random walk. This represents the state sampling function in the state sequence. Both represent the status information of the mobile robot; In the process of calculating the cumulative expected number of visits with discounts in the form of expectations using the state sequence, the following relationship exists: ; in, This indicates the cumulative expected number of visits for the discount. Indicates the initial state. This represents the discount factor at time step t. Indicates the target state or successor state. This represents the mathematical expectation under the random walk strategy. This indicates processing via the Kronecker function. This represents the state at time step t. Indicates the initial state. Indicates the time step index; In rewriting the cumulative expected number of visits for discounts from expected form to probabilistic form, the following relationship exists: ; in, This represents the state access probability function; This represents the conditional probability of reaching the target state or subsequent state after t time steps from the starting state under a preset default strategy or a random walk strategy, i.e., the state transition probability at t time steps. It represents the conditional probability of transitioning from the initial state to the target state or the successor state in one step under a preset default policy or a random walk policy, i.e., the one-step state transition probability. This represents the number of times the initial state transitions to the target state or successor state in the sequence. This represents the total number of transitions triggered from the initial state; Based on the properties of Markov chains, in the process of rewriting the cumulative expected number of visits from the expected form to the probabilistic form, and in the process of writing the infinite series in the form of a matrix to obtain the successor representation matrix, the following relationship exists: ; in, Represents the successor representation matrix. Represents the identity matrix; Let f(x) represent a random matrix with a row sum of 1 and a spectral radius not exceeding 1. The discount factor determines the effective time horizon of the subsequent representation matrix, and It typically takes the value 0.99; In calculating the steady-state access probability of each state under the preset default strategy (or random walk strategy), the following relationship exists: ; in, This represents the empirical steady-state access probability. This indicates the number of times the initial state appears in the state sequence; In the process of calculating subsequent information based on the steady-state access probability of each state, the following relationship exists: ; in, Indicates subsequent information; This represents a logarithmic measure of discounted reachability from the initial state to the target state or successor state. A larger value indicates greater reachability from the trigger state. The easier to reach ; This represents a self-information item, indicating normalization compensation for frequently accessed states; This represents the empirical steady-state access probability; Within the linear reinforcement learning framework, the process of calculating the optimal function using the steady-state access probability and subsequent information of each state corresponds to the following relationship: ; in, The parameter representing the penalty weight for deviating from the default policy. Indicates the target state. This represents the optimal value function. This indicates the expected number of visits accumulated under the default policy with discounts. This indicates that processing is performed using the successor representation function under the default strategy. This represents the steady-state access probability under the default policy. This indicates that under the default policy, the process is handled using the steady-state access probability function. This indicates subsequent information under the default policy. This indicates that processing will be performed using a subsequent information function under the default strategy; In the process of filtering out negative correlations in the successor information according to the optimal function, and retaining the positive half of the successor information in the successor representation matrix to obtain the positive successor information matrix, the following relationship exists: ; in, Represents the positive successor information matrix. This indicates processing via a positive successor information function; In the process of projecting the positive successor information matrix into a low-dimensional space using constrained nonnegative matrix factorization to obtain the DSI representation vector and DSI weight vector, the following relationship exists: ; in, The DSI representation vector representing the initial state. The DSI weight vector represents the target state or successor state. Indicates the transpose symbol. Indicates approximate equality; In obtaining the DSI representation vector and DSI weight vector, the quality of the decomposition is quantified by constructing a total decomposition quality loss, and the total decomposition quality loss is minimized iteratively through gradient descent. The expression for the total decomposition quality loss is: ; in, This represents the total decomposition quality loss function. This represents the reconstruction error loss term. This represents the weighting coefficient of the decorrelation loss term. This indicates the removal of the relevant loss item. The weight coefficients of the regularization loss term are represented. This represents the regularization loss term; The reconstruction error loss term is constructed using the squared error, and its expression is: ; The expression for the dereferenced loss term is: ; in, The first DSI representation vector represents the... Dimensions and the The Pearson correlation coefficient of the dimension across all states. Indicates the first The set of values in a dimension Indicates the first The set of values in a dimension Indicates the first Values in a dimension Indicates the first The average value of all states in the dimension. Indicates the first Values in a dimension Indicates the first The average value of all states in the dimension; The expression for the regularization loss term is: ; in, The DSI weight vector representing the initial state; In the process of approximating the value function based on the inner product of the DSI representation vector, DSI weight vector, and DSI vector, and constructing navigation decision rules using the value function, the following relationship exists: ; in, Represents the transformation matrix. This represents the candidate transformation matrix to be solved. The matrix that represents the minimum value of the objective function. Represents the set of state transition samples. The DSI weight vector represents the target state. Indicates positive successor information. This indicates a direct proportional relationship. Indicates navigation decision rules, This represents the subset of all reachable neighbors of the initial state, excluding the states at time step t-1. This means selecting the state from the set of candidate neighbor states that maximizes the value of the subsequent objective function. A DSI representation vector representing the target state or successor state.
[0015] Furthermore, in this step, once the offline learning phase is completed, the robot can achieve target-oriented spatial navigation. In the subsequent online navigation phase, constraints ensure that the agent only considers the direction of travel when making decisions at each step, eliminating the shortest-cycle back-and-forth oscillations, while not affecting the possibility of the robot revisiting the state in subsequent steps (i.e., only immediate backtracking is prohibited, but long-distance revisiting is not), thus achieving a balance between avoiding local oscillations and preserving path flexibility.
[0016] Step 2: Define the total potential energy function through the navigation path sequence; use the total potential energy function to iteratively update the node coordinates through gradient descent, while introducing obstacle information, anchoring and freezing nodes that approach obstacles, and using the anchor points to optimize the path segments independently to obtain the optimal path sequence. In step 2, a total potential energy function is defined based on the navigation path sequence. Using this function, node coordinates are iteratively updated via gradient descent. Simultaneously, obstacle information is introduced, and nodes approaching obstacles are anchored and frozen. The path is then segmented and independently optimized using these anchor points to obtain the optimal path sequence. The specific steps are as follows: Step 201: Define the total potential energy function based on the navigation path sequence; Step 202: Based on the total potential energy function, merge the gradient of the local continuous potential energy and the gradient of the global coordinated potential energy to obtain the total potential energy gradient. Using the total potential gradient, the coordinates of all intermediate nodes are iteratively updated through gradient descent to obtain a smoothed path node sequence. Step 203: Discretize the collected environmental obstacle information into a set of obstacle coordinates; use the set of obstacle coordinates and the smoothed path node sequence to calculate the distance from each smoothed path node to the nearest obstacle; Anchoring conditions are constructed using the distance from each smoothed path node to the nearest obstacle and a preset safety threshold. Nodes that meet the anchoring conditions are selected from the smoothed path node sequence to obtain the anchor point set; Step 204: Use the anchor points in the anchor point set to divide the smoothed path into multiple sub-segments, and perform gradient descent optimization of the potential energy function on each sub-segment to obtain the optimal path sequence.
[0017] For details, please refer to Figure 3 In defining the total potential energy function based on the navigation path sequence, the following relationship exists: ; in, Indicates a sequence of navigation paths; Both represent path nodes, where, As the starting point of the path, The endpoint of the path; This represents the total number of path nodes. This represents the j-th path node. Indicates the index of the path node. Represents two-dimensional Euclidean space. Represents the total potential energy function; This represents the local continuous potential energy, which is the sum of the Euclidean distances between all adjacent nodes on the path. This represents the balance coefficient; the smaller the value, the more the path focuses on eliminating local jagged edges. It represents the global coordination potential energy, and its physical meaning is the total arc length of the path; The definition of local continuous potential energy is: ; in, This represents the (j+1)th path node; The expression for the local continuous potential gradient is: ; in, This represents the local continuous potential gradient. This indicates calculating the gradient with respect to the j-th path node; The definition of global coordination potential energy is: ; in, Indicates the index of the path node; This represents the topological distance decay weight, and its physical meaning is the overall resilience of the path chain: it provides a weak attraction to all node pairs on the path, causing the entire path to contract towards the geometric center; This represents the m-th path node; This represents the attenuation coefficient, typically set to 0.02; weight. The design, which decays with topological distance, ensures that the strong attraction of near-end nodes guarantees path smoothing, while the weak attraction of far-end nodes prevents catastrophic path collapse. The expression for the global coordinated potential gradient is: ; in, This represents the global coordination potential gradient. This represents a symbolic function determined by the relative order of the indices m and j of path nodes; the function's purpose is to: for path nodes in the sequence that are ranked... Subsequent nodes are subjected to forward attraction, while nodes preceding them are subjected to backward attraction, thus shrinking the path while maintaining the order of nodes. In the process of merging the local continuous potential energy gradient and the global coordinated potential energy gradient according to the total potential energy function to obtain the total potential energy gradient, the following relationship exists: ; in, This represents the total potential energy gradient. This represents the local continuous potential gradient. Represents the global coordinated potential gradient; In the process of using the total potential energy gradient to iteratively update the coordinates of all intermediate nodes through gradient descent to obtain the smoothed path node sequence, the following relationship exists: ; in, Indicates the first The navigation path sequence obtained after rounds of iteration, Indicates the first The navigation path sequence obtained after rounds of iteration, Indicates the learning rate. This represents the total potential energy gradient; after each update, the starting point is forcibly fixed. and the end point The coordinates remain unchanged; In the process of discretizing the collected obstacle information into a set of obstacle coordinates, and then using this set of obstacle coordinates to calculate the distance from each smoothed path node to the nearest obstacle, the following relationship exists: ; in, Represents the set of obstacle coordinates. All represent obstacle sampling points. This indicates the total number of obstacle sampling points. This indicates the first obstacle sampling point. This indicates the second obstacle sampling point. Indicates the first One obstacle sampling point, Indicates the distance to the nearest obstacle sampling point. This indicates finding the obstacle sampling point that minimizes the distance within the set of obstacle coordinates. Indicates the first The position coordinates of the j-th path node after round of iteration. This represents the h-th obstacle sampling point; In the process of constructing anchoring conditions using the distance from each smoothed path node to the nearest obstacle and a preset safety threshold, the following relationship exists: ; in, Indicates the anchoring condition. Indicates the first The distance from the j-th path node after smoothing following rounds of iterations to the nearest obstacle sampling point. Indicates the safety threshold. Indicates the first The distance from the j-th path node after smoothing following rounds of iterations to the nearest obstacle sampling point; when a certain path node The distance to the obstacle is reduced to a safe threshold. If the distance continues to decrease, anchor the node (freeze its coordinates and prevent it from participating in subsequent updates), with a safety threshold. It can be determined based on the robot's equivalent radius, the environmental discrete resolution, and the safety margin. Let the side length or discrete resolution of the environmental mesh be... The robot's equivalent radius is Safety margin is Then it is acceptable. In a normalized mesh environment, if the edge length of each mesh is taken as... ,but The range of values can be set to The preferred value is ; The anchoring mechanism serves to immediately anchor a path node at a safe distance once it approaches an obstacle boundary, preventing the path from crossing the obstacle while maintaining a safety margin. ; In the process of selecting nodes that meet the anchoring conditions from the smoothed path node sequence to obtain the anchor point set, the following relationship exists: ; in, Indicates the first The set of path nodes that have been anchored after rounds of iteration. Indicates the first The set of path nodes that have been anchored after rounds of iteration. Indicates the first The set of path node indices that satisfy the anchoring condition after rounds of iteration. This function represents the process of sorting the elements of a collection from smallest to largest. In the process of optimizing the potential function gradient descent for each sub-segment to obtain the optimal path sequence, the following relationship exists: ; in, Indicates the first After rounds of iteration, the r-th path segment, Indicates the first After rounds of iteration, the r-th path segment, This represents a gradient optimization operation. Indicates the first The starting fixed endpoint in the round iteration, Indicates the first The terminating endpoint in the round iteration, Indicates the first The starting fixed endpoint in the round iteration, Indicates the first The terminating endpoint in the round iteration, This represents the index of a path segment. The advantage of this segmentation strategy is that nodes near obstacles become new fixed endpoints after being anchored, decomposing a global optimization problem into multiple local optimization subproblems. Each subproblem is solved independently in free space without obstacle interference, which ensures the obstacle safety of the global path and reduces the computational complexity of optimization.
[0018] Step 3: Extract the optimal state sequence from the optimal path sequence as a supervision signal to construct the total correction loss; use the total correction loss to update the DSI representation using gradient descent to obtain the corrected DSI representation; store the corrected DSI representation to obtain the correction experience buffer pool. In step 3, the optimal state sequence is extracted from the optimal path sequence as a supervision signal to construct the total correction loss; the DSI representation is updated using gradient descent using the total correction loss to obtain the corrected DSI representation; the corrected DSI representation is stored to obtain the correction experience buffer pool. The specific steps are as follows: Step 301: Map the path nodes in the optimal path sequence to discrete states to obtain the optimal state sequence; Step 302: Construct the total correction loss based on the optimal state sequence, DSI representation vector, and DSI weight vector; use the total correction loss to update the DSI representation vector and DSI weight vector using gradient descent to obtain the corrected DSI representation vector and corrected DSI weight vector. Step 303: Store the corrected DSI representation vector, the corrected DSI weight vector, and the optimal state sequence to obtain the correction experience buffer pool.
[0019] For details, please refer to Figure 4 In the process of mapping path nodes in the optimal path sequence to discrete states to obtain the optimal state sequence, the following relationship exists: ; in, Represents the optimal state sequence. Each represents a discrete state in the optimal state sequence. This represents the discrete state mapped to the first path node. This represents the discrete state mapped to the second path node. This represents the discrete state mapped to the Mth path node. Let represent the discrete state mapped to the j-th path node. This represents selecting the state that minimizes the distance among all discrete states. Representing discrete states, Represents the set of all discrete states in the environment. This represents the j-th path node in the optimal path sequence. Represents the physical coordinates corresponding to discrete states; on the optimal state sequence (i.e., the optimal path), for each pair of adjacent states... The correct navigation decision is from Move to This means that... When the goal is to achieve it, The navigation value should be higher than The other neighbors; this relationship constitutes the supervisory signal for correcting the DSI vector; A total correction loss is constructed based on the optimal state sequence, DSI representation vector, and DSI weight vector. This total correction loss is then used to update the DSI representation vector and DSI weight vector using gradient descent, resulting in the corrected DSI representation vector and DSI weight vector. The expression for the total correction loss is: ; in, Indicates the total correction loss. This represents the order consistency loss. The weights represent the vector steady-state loss. Represents the vector steady-state loss. The weight representing the geodesic deviation loss, Indicates geodesic deviation loss; The ranking consistency loss and vector steady-state loss are constructed using the DSI representation vector and DSI weight vector. The expressions for the ranking consistency loss and vector steady-state loss are as follows: ; in, This represents the state of a non-optimal candidate neighbor. The DSI representation vector representing the non-optimal candidate neighbor state. Let the set of non-optimal neighbors of the j-th path node be denoted as . The DSI representation vector represents the optimal discrete state mapped to the (j+1)th path node; Indicates the interval parameter, and ; The DSI representation vector representing the discrete state. The original DSI representation vector representing the discrete state. The DSI weight vector represents the discrete state. The original DSI weight vector represents the discrete state; for each step on the optimal path Define its non-optimal neighbor set as ,Right now Eliminate the optimal next step from all reachable neighbors The subset after; the margin parameter requires that the inner product value of the optimal next step must not only be greater than that of the non-optimal neighbors, but also be at least larger than that of the non-optimal neighbors. The margin; The hinge loss mechanism is adopted: when the sorting is correct and the margin is sufficient, the loss is zero; when the sorting is incorrect or the margin is insufficient, a positive loss is generated to drive gradient correction. The intuitive meaning is: for each decision point on the optimal path, check whether the current DSI vector will allow the greedy navigation to choose the correct direction. If it chooses the wrong direction (the inner product of a non-optimal neighbor is larger), a penalty signal is generated to force the DSI vector to be adjusted to the correct direction. This ensures that the correction process is an incremental fine-tuning of the original DSI vector rather than a full rewrite, thus maintaining the integrity of the global topological semantic information; Order consistency loss This only guarantees the correctness of local decisions at each step, but does not directly constrain the global path length. To further enhance the quality of the global path, a geodesic bias loss is introduced, the expression for which is: ; in, This indicates the actual length of the path obtained by applying the navigation decision rule to the current DSI vector. The hierarchical weights represent the control over the intensity of the failure penalty. The hierarchical weights represent the control over the intensity of the penalty for deviation. This represents the hierarchical weights that control the optimal reward intensity. Indicates the maximum allowed number of steps. This represents the geometrically optimal path length, serving as a truth reference. After the correction loss function is designed, the total correction loss is calculated for each navigation experiment (which includes a "start-target" pair and the corresponding optimal path). right and The gradient, and for and Perform gradient descent updates; In the process of storing the corrected DSI representation vector, the corrected DSI weight vector, and the optimal state sequence to obtain the correction experience buffer, the following relationship exists: ; in, This indicates the correction of the experience buffer pool. Indicates the amount of navigation experience. Both represent the target state in navigation experience. All represent the optimal state sequence in navigation experience; the mechanism serves three purposes: First, it avoids overfitting: preventing the DSI vector from being optimized only for a single target and losing its navigation ability for other targets; Second, it ensures global consistency: the correction gradients from different targets counterbalance each other after averaging, maintaining the topological encoding consistency of the DSI vector across the entire state space; Third, it facilitates incremental accumulation: as the robot performs more and more navigation tasks, With continuous expansion, the correction signal covers more and more "start-point-target" pairs, and the global accuracy of the DSI vector continues to improve.
[0020] Step 4: Extract the corrected DSI representation from the calibration experience buffer pool, repeat the online navigation, path optimization and calibration steps, and obtain the optimal brain-like navigation sequence after a preset number of iterations; In step 4, the corrected DSI representation is extracted from the calibration experience buffer pool, and the online navigation, path optimization, and calibration steps are repeated. After a preset number of iterations, the optimal brain-like navigation sequence is obtained. The specific steps are as follows: Step 401: Extract the DSI representation vector and DSI weight vector from the calibration experience buffer pool, and execute steps 105 to 302 in an iterative manner. After a preset number of iterations (the navigation path sequence obtained in step 105 approximates the optimal path sequence obtained in step 204), the system reaches steady state. Step 402: After the system reaches steady state, the navigation path sequence obtained in the last round is taken as the optimal brain-like navigation sequence.
[0021] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.
[0022] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.
[0023] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.
Claims
1. A brain-like navigation method based on DSI decoupling characterization and composite potential field path optimization, characterized in that, The method includes the following steps: Step 1: Discretize the environment and collect state sequences through random walks; use the state sequences to calculate the successor representation matrix. The successor information is then calculated based on the successor representation matrix and positiveened to obtain the positive successor information matrix. The DSI representation is obtained by dimensionality reduction through positive successor information matrix decomposition. Based on DSI representation, the value function is approximated by the DSI inner product. The neighbor with the highest value is selected and the source from the previous step is excluded. This process is repeated until the target state is reached to generate a navigation path sequence. Step 2: Define the total potential energy function through the navigation path sequence; use the total potential energy function to iteratively update the node coordinates through gradient descent, while introducing obstacle information, anchoring and freezing nodes that approach obstacles, and using the anchor points to optimize the path segments independently to obtain the optimal path sequence. Step 3: Extract the optimal state sequence from the optimal path sequence as a supervision signal to construct the total correction loss; use the total correction loss to update the DSI representation using gradient descent to obtain the corrected DSI representation; store the corrected DSI representation to obtain the correction experience buffer pool. Step 4: Extract the corrected DSI representation from the calibration experience buffer pool, repeat the online navigation, path optimization and calibration steps, and obtain the optimal brain-like navigation sequence after a preset number of iterations.
2. The brain-like navigation method based on DSI decoupling characterization and composite potential field path optimization according to claim 1, characterized in that, In step 1, the environment is discretized, and a state sequence is collected through random walks; Calculate the successor representation matrix using the state sequence; then calculate the successor information based on the successor representation matrix and take the positive value to obtain the positive successor information matrix. The DSI representation is obtained by dimensionality reduction through positive successor information matrix decomposition. Based on DSI representation, the value function is approximated using the DSI inner product. The neighbor with the highest value is selected and the source from the previous step is excluded. This process is repeated until the target state is reached to generate a navigation path sequence. The specific steps are as follows: Step 101: The mobile robot executes a random exploration strategy in the environment to obtain a state sequence; Step 102: Calculate the cumulative expected number of visits with discounts in the form of the state sequence. Rewrite the expected number of visits accumulated by discounts from the expected form to the probabilistic form; Based on the properties of Markov chains, in the process of rewriting the cumulative expected number of visits from the expected form to the probabilistic form, the infinite series is written in the form of a matrix to obtain the successor representation matrix. Step 103: Calculate the steady-state access probability of each state under the preset default strategy; Subsequent information is obtained by calculating the steady-state access probability of each state; Within the linear reinforcement learning framework, the optimal function is calculated by utilizing the steady-state access probability and subsequent information of each state. Based on the optimal function, negative correlations in the successor information are filtered out to retain the positive half of the successor information in the successor representation matrix, thus obtaining the positive successor information matrix. Step 104: The positive successor information matrix is projected into a low-dimensional space using constrained nonnegative matrix decomposition to obtain the DSI representation vector and the DSI weight vector. Step 105: Based on the DSI representation vector and DSI weight vector, the inner product of the DSI vectors approximates the value function, and the value function is used to construct navigation decision rules. The mobile robot starts from the current state, repeatedly applies navigation decision rules until it reaches the target state, and records the sequence of states it has passed through to obtain the navigation path sequence.
3. The brain-like navigation method based on DSI decoupling characterization and composite potential field path optimization according to claim 2, characterized in that, In the process of a mobile robot executing a random exploration strategy in the environment to obtain a state sequence, the following relationship exists: ； in, Represents a state sequence. This represents the total number of steps in the random walk. This represents the state sampling function in the state sequence. Both represent the status information of the mobile robot; In the process of calculating the cumulative expected number of visits with discounts in the form of expectations using the state sequence, the following relationship exists: ； in, This indicates the cumulative expected number of visits for the discount. Indicates the initial state. This represents the discount factor at time step t. Indicates the target state or successor state. This represents the mathematical expectation under the random walk strategy. This indicates processing via the Kronecker function. This represents the state at time step t. Indicates the initial state. Indicates the time step index; In rewriting the cumulative expected number of visits for discounts from expected form to probabilistic form, the following relationship exists: ； in, This represents the state access probability function. This represents the conditional probability of reaching the target state or subsequent state after t time steps, given a preset default strategy or a random walk strategy. This represents the conditional probability that, under a preset default policy or a random walk policy, the starting state can be transitioned to the target state or successor state in one step. This represents the number of times the initial state transitions to the target state or successor state in the sequence. This represents the total number of transitions triggered from the initial state; Based on the properties of Markov chains, in the process of rewriting the cumulative expected number of visits from the expected form to the probabilistic form, and in the process of writing the infinite series in the form of a matrix to obtain the successor representation matrix, the following relationship exists: ； in, Represents the successor representation matrix. Represents the identity matrix. Represents a random matrix. Indicates the discount factor; In the process of calculating the steady-state access probability of each state under the preset default policy, the following relationship exists: ； in, This represents the empirical steady-state access probability. This indicates the number of times the initial state appears in the state sequence; In the process of calculating subsequent information based on the steady-state access probability of each state, the following relationship exists: ； in, Indicates subsequent information, This represents the empirical steady-state access probability; Within the linear reinforcement learning framework, the process of calculating the optimal function using the steady-state access probability and subsequent information of each state corresponds to the following relationship: ； in, The parameter representing the penalty weight for deviating from the default policy. Indicates the target state. This represents the optimal value function. This indicates the cumulative expected number of visits for the discount. This indicates processing through a successor representation function. Represents the steady-state access probability. This indicates that the process is handled through the steady-state access probability function. Indicates subsequent information, This indicates that the information will be processed through a subsequent information function.
4. The brain-like navigation method based on DSI decoupling characterization and composite potential field path optimization according to claim 3, characterized in that, In the process of filtering out negative correlations in the successor information according to the optimal function, and retaining the positive half of the successor information in the successor representation matrix to obtain the positive successor information matrix, the following relationship exists: ； in, Represents the positive successor information matrix. This indicates processing via a positive successor information function; In the process of projecting the positive successor information matrix into a low-dimensional space using constrained nonnegative matrix factorization to obtain the DSI representation vector and DSI weight vector, the following relationship exists: ； in, The DSI representation vector representing the initial state. The DSI weight vector represents the target state or successor state. Indicates the transpose symbol. Indicates approximate equality; In obtaining the DSI representation vector and DSI weight vector, the quality of the decomposition is quantified by constructing a total decomposition quality loss, and the total decomposition quality loss is minimized iteratively through gradient descent. The expression for the total decomposition quality loss is: ； in, This represents the total decomposition quality loss function. This represents the reconstruction error loss term. This represents the weighting coefficient of the decorrelation loss term. This indicates the removal of the relevant loss item. The weight coefficients of the regularization loss term are represented. This represents the regularization loss term; The reconstruction error loss term is constructed using the squared error, and its expression is: ； The expression for the dereferenced loss term is: ； in, The first DSI representation vector represents the... Dimensions and the The Pearson correlation coefficient of the dimension across all states. Indicates the first The set of values in a dimension Indicates the first The set of values in a dimension Indicates the first Values in a dimension Indicates the first The average value of all states in the dimension. Indicates the first Values in a dimension Indicates the first The average value of all states in the dimension; The expression for the regularization loss term is: ； in, The DSI weight vector representing the initial state; In the process of approximating the value function based on the inner product of the DSI representation vector, DSI weight vector, and DSI vector, and constructing navigation decision rules using the value function, the following relationship exists: ； in, Represents the transformation matrix. This represents the candidate transformation matrix to be solved. The matrix that represents the minimum value of the objective function. Represents the set of state transition samples. The DSI weight vector represents the target state. Indicates positive successor information. This indicates a direct proportional relationship. Indicates navigation decision rules, This represents the subset of all reachable neighbors of the initial state, excluding the states at time step t-1. This means selecting the state from the set of candidate neighbor states that maximizes the value of the subsequent objective function. A DSI representation vector representing the target state or successor state.
5. The brain-like navigation method based on DSI decoupling characterization and composite potential field path optimization according to claim 4, characterized in that, In step 2, a total potential energy function is defined using the navigation path sequence. Using this function, node coordinates are iteratively updated via gradient descent. Simultaneously, obstacle information is introduced, and nodes approaching obstacles are anchored and frozen. The path is then segmented and independently optimized using these anchor points to obtain the optimal path sequence. The specific steps are as follows: Step 201: Define the total potential energy function based on the navigation path sequence; Step 202: Based on the total potential energy function, merge the gradient of the local continuous potential energy and the gradient of the global coordinated potential energy to obtain the total potential energy gradient. Using the total potential gradient, the coordinates of all intermediate nodes are iteratively updated through gradient descent to obtain a smoothed path node sequence. Step 203: Discretize the collected environmental obstacle information into a set of obstacle coordinates; Using the set of obstacle coordinates and the smoothed path node sequence, calculate the distance from each smoothed path node to the nearest obstacle; Anchoring conditions are constructed using the distance from each smoothed path node to the nearest obstacle and a preset safety threshold. Nodes that meet the anchoring conditions are selected from the smoothed path node sequence to obtain the anchor point set; Step 204: Use the anchor points in the anchor point set to divide the smoothed path into multiple sub-segments, and perform gradient descent optimization of the potential energy function on each sub-segment to obtain the optimal path sequence.
6. The brain-like navigation method based on DSI decoupling characterization and composite potential field path optimization according to claim 5, characterized in that, In defining the total potential energy function based on the navigation path sequence, the following relationship exists: ； in, Indicates a navigation path sequence. Both represent path nodes. This represents the total number of path nodes. This represents the j-th path node. Indicates the index of the path node. Represents two-dimensional Euclidean space. This represents the total potential energy function. Represents local continuous potential energy. Represents the balance coefficient. Represents the global coordination potential energy; The definition of local continuous potential energy is: ； in, This represents the (j+1)th path node; The expression for the local continuous potential gradient is: ； in, This represents the local continuous potential gradient. This indicates calculating the gradient with respect to the j-th path node; The definition of global coordination potential energy is: ； in, Indicates the index of the path node. Indicates the topological distance decay weight. This represents the m-th path node. Indicates the attenuation coefficient; The expression for the global coordinated potential gradient is: ； in, This represents the global coordination potential gradient. A symbolic function representing the order determined by the relative order of the index m and index j of a path node; In the process of merging the local continuous potential energy gradient and the global coordinated potential energy gradient according to the total potential energy function to obtain the total potential energy gradient, the following relationship exists: ； in, This represents the total potential energy gradient. This represents the local continuous potential gradient. This represents the global coordinated potential gradient.
7. The brain-like navigation method based on DSI decoupling characterization and composite potential field path optimization according to claim 6, characterized in that, In the process of using the total potential energy gradient to iteratively update the coordinates of all intermediate nodes through gradient descent to obtain the smoothed path node sequence, the following relationship exists: ； in, Indicates the first The navigation path sequence obtained after rounds of iteration, Indicates the first The navigation path sequence obtained after rounds of iteration, Indicates the learning rate. This represents the total potential energy gradient; In the process of discretizing the collected obstacle information into a set of obstacle coordinates, and then using this set of obstacle coordinates to calculate the distance from each smoothed path node to the nearest obstacle, the following relationship exists: ； in, Represents the set of obstacle coordinates. All represent obstacle sampling points. This indicates the total number of obstacle sampling points. This indicates the first obstacle sampling point. This indicates the second obstacle sampling point. Indicates the first One obstacle sampling point, Indicates the distance to the nearest obstacle sampling point. This indicates finding the obstacle sampling point that minimizes the distance within the set of obstacle coordinates. Indicates the first The position coordinates of the j-th path node after round of iteration. This represents the h-th obstacle sampling point; In the process of constructing anchoring conditions using the distance from each smoothed path node to the nearest obstacle and a preset safety threshold, the following relationship exists: ； in, Indicates the anchoring condition. Indicates the first The distance from the j-th path node after smoothing following rounds of iterations to the nearest obstacle sampling point. Indicates the safety threshold. Indicates the first The distance from the j-th path node to the nearest obstacle sampling point after smoothing following rounds of iterations; In the process of selecting nodes that meet the anchoring conditions from the smoothed path node sequence to obtain the anchor point set, the following relationship exists: ； in, Indicates the first The set of path nodes that have been anchored after rounds of iteration. Indicates the first The set of path nodes that have been anchored after rounds of iteration. Indicates the first The set of path node indices that satisfy the anchoring condition after rounds of iteration. This function represents the process of sorting the elements of a collection from smallest to largest. In the process of optimizing the potential function gradient descent for each sub-segment to obtain the optimal path sequence, the following relationship exists: ； in, Indicates the first After rounds of iteration, the r-th path segment, Indicates the first After rounds of iteration, the r-th path segment, This represents a gradient optimization operation. Indicates the first The starting fixed endpoint in the round iteration, Indicates the first The terminating endpoint in the round iteration, Indicates the first The starting fixed endpoint in the round iteration, Indicates the first The terminating endpoint in the round iteration, Indicates the index of the path segment.
8. The brain-like navigation method based on DSI decoupling characterization and composite potential field path optimization according to claim 7, characterized in that, In step 3, the optimal state sequence is extracted from the optimal path sequence as a supervision signal to construct the total correction loss; the DSI representation is updated using gradient descent using the total correction loss to obtain the corrected DSI representation; the corrected DSI representation is stored to obtain the correction experience buffer pool. The specific steps are as follows: Step 301: Map the path nodes in the optimal path sequence to discrete states to obtain the optimal state sequence; Step 302: Construct the total correction loss based on the optimal state sequence, DSI representation vector, and DSI weight vector; use the total correction loss to update the DSI representation vector and DSI weight vector using gradient descent to obtain the corrected DSI representation vector and corrected DSI weight vector. Step 303: Store the corrected DSI representation vector, the corrected DSI weight vector, and the optimal state sequence to obtain the correction experience buffer pool.
9. The brain-like navigation method based on DSI decoupling characterization and composite potential field path optimization according to claim 8, characterized in that, In the process of mapping path nodes in the optimal path sequence to discrete states to obtain the optimal state sequence, the following relationship exists: ； in, Represents the optimal state sequence. Each represents a discrete state in the optimal state sequence. This represents the discrete state mapped to the first path node. This represents the discrete state mapped to the second path node. This represents the discrete state mapped to the Mth path node. Let represent the discrete state mapped to the j-th path node. This represents selecting the state that minimizes the distance among all discrete states. Representing discrete states, Represents the set of all discrete states in the environment. This represents the j-th path node in the optimal path sequence. Represents the physical coordinates corresponding to discrete states; A total correction loss is constructed based on the optimal state sequence, DSI representation vector, and DSI weight vector. This total correction loss is then used to update the DSI representation vector and DSI weight vector using gradient descent, resulting in the corrected DSI representation vector and DSI weight vector. The expression for the total correction loss is: ； in, Indicates the total correction loss. This represents the order consistency loss. The weights represent the vector steady-state loss. Represents the vector steady-state loss. The weight representing the geodesic deviation loss, Indicates geodesic deviation loss; The ranking consistency loss and vector steady-state loss are constructed using the DSI representation vector and DSI weight vector. The expressions for the ranking consistency loss and vector steady-state loss are as follows: ； in, This represents the state of a non-optimal candidate neighbor. The DSI representation vector representing the non-optimal candidate neighbor state. Let the set of non-optimal neighbors of the j-th path node be denoted as . Let represent the DSI representation vector of the optimal discrete state mapped to the (j+1)th path node. Indicates the interval parameter. The DSI representation vector representing the discrete state. The original DSI representation vector representing the discrete state. The DSI weight vector represents the discrete state. The original DSI weight vector representing the discrete state; The expression for geodesic deviation loss is: ； in, This indicates the actual length of the path obtained by applying the navigation decision rule to the current DSI vector. The hierarchical weights represent the control over the intensity of the failure penalty. The hierarchical weights represent the control over the intensity of the penalty for deviation. This represents the hierarchical weights that control the optimal reward intensity. Indicates the maximum allowed number of steps. Indicates the geometrically optimal path length; In the process of storing the corrected DSI representation vector, the corrected DSI weight vector, and the optimal state sequence to obtain the correction experience buffer, the following relationship exists: ； in, This indicates the correction of the experience buffer pool. Indicates the amount of navigation experience. Both represent the target state in navigation experience. Both represent the optimal state sequence in navigation experience.
10. The brain-like navigation method based on DSI decoupling characterization and composite potential field path optimization according to claim 9, characterized in that, In step 4, the corrected DSI representation is extracted from the calibration experience buffer, and the online navigation, path optimization, and calibration steps are repeated. After a preset number of iterations, the optimal brain-like navigation sequence is obtained. The specific steps are as follows: Step 401: Extract the DSI representation vector and DSI weight vector from the calibration experience buffer pool, and execute steps 105 to 302 in an iterative manner. After a preset number of iterations, the system reaches steady state. Step 402: After the system reaches steady state, the navigation path sequence obtained in the last round is taken as the optimal brain-like navigation sequence.