Mobile robot adaptive navigation method for continuous tasks

Abstract

The application discloses a kind of mobile robot adaptive navigation methods for continuous task, the method of the application organizes as task stream for continuous incoming navigation task, obtains the state observation of robot and task semantic information;Potential task variables are constructed by task encoder;Dynamic knowledge space is established based on Dirichlet process mixture model, online update is carried out using memorized variational inference, and the birth and merging of knowledge cluster are realized;Potential task variables are input as context condition into downstream soft actor critic policy network, and adaptive navigation action is generated.The application can continuously accumulate and reuse navigation knowledge in long-term task stream, effectively alleviate catastrophic forgetting, improve the learning efficiency and navigation success rate of new task, and ensure the long-term stable operation of mobile robot under continuous task conditions.

Description

Technical Field

[0001] This invention belongs to the field of autonomous navigation and reinforcement learning decision-making technology for mobile robots, specifically an adaptive navigation method for mobile robots in continuous tasks. The method utilizes task semantic representation, dynamic knowledge space modeling, memorized variational inference, and conditional reinforcement learning strategy training to achieve continuous knowledge accumulation, knowledge reuse, and adaptive updating of navigation strategies for mobile robots in multiple consecutive navigation tasks. Background Technology

[0002] Mobile robots have wide applications in smart manufacturing, warehousing and logistics, service robots, inspection operations, and hazardous environment detection. As the application environment of robots gradually expands from regular static scenarios to open, dynamic, and unstructured scenarios, robots often need to switch continuously between different tasks and maintain stable autonomous navigation capabilities under new environments, new obstacle distributions, and new target requirements.

[0003] Traditional mobile robot navigation methods typically rely on pre-built environment maps, explicit obstacle models, and fixed path planning strategies. When the environment changes or the task expands, the system needs to frequently remap, replan, or readjust parameters, making it difficult to fully utilize navigation experience gained from past tasks. For robots that need to operate long-term, simply relying on fixed models or single-task training methods cannot meet the needs of continuous learning and rapid adaptation.

[0004] Reinforcement learning methods can directly learn the mapping relationship between states and actions through the interaction between an agent and its environment, reducing reliance on precise environmental models and showing great potential for application in the navigation and control of mobile robots. However, most existing reinforcement learning navigation methods are trained for single tasks or fixed task distributions. When a robot faces multiple tasks sequentially, directly training the original policy network on new tasks can easily lead to a decline in the performance of previously learned tasks, i.e., catastrophic forgetting. Training independent policies for each task, on the other hand, leads to increased training costs, inability to reuse knowledge, and complex policy deployment.

[0005] While multi-task reinforcement learning can improve policy generalization by accessing multiple tasks simultaneously, it typically assumes that all tasks are given during the training phase and can be repeatedly accessed. This assumption is inconsistent with the long-term operation of real robots, as real-world tasks often arrive sequentially, and the number, types, and boundaries of tasks may change over time. Continuous learning methods can alleviate the forgetting problem to some extent; for example, some existing techniques have explored the application of knowledge transfer, meta-learning, or lifelong learning in robot navigation. However, common fine-tuning, regularization (such as Elastic Weight Consolidation, EWC), or experience replay methods still suffer from problems such as insufficient task representation, fixed knowledge structure, and difficulty in balancing the preservation of old knowledge with adaptation to new tasks. Summary of the Invention

[0006] In view of this, the purpose of this invention is to provide an adaptive navigation method for mobile robots for continuous tasks, so as to solve the problems of catastrophic forgetting, insufficient knowledge transfer ability and low efficiency of adapting to new tasks in long-term continuous navigation tasks, and realize continuous knowledge accumulation and adaptive updating of navigation strategies.

[0007] To achieve the above objectives, the present invention provides the following technical solution: An adaptive navigation method for a mobile robot for continuous tasks includes the following steps: Step 1: Obtain the state observation information and task semantic information of the mobile robot in the long-term task flow, and construct a joint input; Step 2: Construct a task encoder to jointly map the state observation information and task semantic information into potential task variables; Step 3: Construct a dynamic knowledge space based on the Dirichlet process hybrid model, treating the potential task variables as samples generated by several knowledge clusters, and enabling the number of knowledge clusters to adaptively change with the progress of the long-term task flow; Step 4: Update the dynamic knowledge space online based on memorized variational inference, calculate the soft attribution probability of the current latent task variable to each knowledge cluster, and incrementally update the knowledge cluster parameters using sufficient statistics; Step 5: Construct a knowledge generation mechanism and a knowledge merging mechanism based on the soft attribution probability. When the current sample cannot be fully explained by the existing knowledge cluster, a new knowledge cluster is generated. When there is redundancy between existing knowledge clusters, merging is performed. Step 6: Construct an upstream joint training objective so that the latent variables output by the task encoder simultaneously meet the requirements of knowledge cluster alignment, task semantic preservation, and dynamic environmental prediction. Step 7: Input the latent task variables as contextual conditions into the downstream soft actor critic algorithm policy network and value network to construct a conditional reinforcement learning navigation policy, so that the policy generates differentiated navigation actions according to different task contexts.

[0008] Furthermore, in step one, the state observation information Represented as: in: This represents the Euclidean distance from the robot to the global target point; and Indicates the robot's position relative to the target point. direction and Positional deviation in direction; and This represents the robot's relative velocity to the target point in two directions; and This indicates the distance information of the robot relative to the local guide point; and These represent the robot's linear velocity and angular velocity at the current moment, respectively. This indicates lidar observation information; The task semantic information, after being encoded by a pre-trained language model, forms a semantic embedding, which, together with the state observation, constitutes the input of the task encoder, as follows: in: Indicates the input to the task encoder; Indicates state observation; It represents the semantic information of the task.

[0009] Furthermore, in step two, the learning condition distribution of the task encoder is as follows: in: Indicates the task encoder parameters; and These represent the mean and standard deviation of the latent distribution output by the encoder, respectively. Indicates state observation; Represents task semantic information; Represents potential task variables; Represents a diagonal matrix; Constructing latent task variables using reparameterization: in: This represents a random noise vector that follows a standard normal distribution. This indicates the Hadamard element-wise multiplication operation.

[0010] Furthermore, in step three, the process of constructing the dynamic knowledge space is represented as follows: in: Represents the prior distribution of the parameters of the knowledge cluster; This represents the mixed weight distribution induced by the stick-breaking process; Represents category distribution; This represents the data generation distribution given cluster parameters; Denotes the first sample obtained from the prior distribution. Each knowledge cluster parameter; Represent the prior distribution hyperparameters; This represents a weight vector composed of the mixed weights of each knowledge cluster; Indicates the first The knowledge cluster affiliation variable corresponding to each potential variable; Indicate latent variables; For the concentration parameters of the Dirichlet process; The Dirichlet process mixture model models each knowledge cluster as a Gaussian distribution: in: and They represent the first The central location and distribution range of a knowledge cluster in the latent space.

[0011] Furthermore, in step four, the memorized variational inference adopts a variational distribution in factorized form: in: Representing latent variables The posterior distribution of cluster affiliation; This represents the posterior distribution of the variable representing the folding stick; Indicates the first The posterior distribution of the parameters of a knowledge cluster; The set of cluster affiliation variables representing all latent variables; This represents the knowledge cluster affiliation variable of the i-th latent variable; This represents the set of stick-breaking variables during the stick-breaking process; This represents the folding rod variable corresponding to the k-th knowledge cluster; and They represent the first The central location and distribution range of a knowledge cluster in the latent space; This indicates the number of latent variable samples currently used for variational inference; This indicates the number of active knowledge clusters in the current dynamic knowledge space.

[0012] Furthermore, in step four, the current latent variables... Belongs to the The posterior probability of a knowledge cluster Calculated using the following formula: in: Represents the variational distribution Expectation calculation; Indicates the first Mixed weights for knowledge clusters; Utilizing sufficient statistics , , Each knowledge cluster is incrementally updated; among which: Description of the The current effective sample quality of each knowledge cluster; and Describe the first First-order and second-order statistical information of a knowledge cluster sample.

[0013] Furthermore, in step five, the triggering condition for the knowledge generation mechanism is the maximum belonging probability of the current sample to the existing knowledge cluster. The current sample's maximum probability of belonging to an existing knowledge cluster is below a set threshold. Represented as: in: Representing latent variables Belongs to the The posterior probability of a knowledge cluster; Indicates the number of knowledge clusters; The execution condition of the knowledge merging mechanism is the marginal likelihood gain before and after the merging. Greater than zero; the marginal likelihood gain before and after the merger Represented as: in: and Let each represent a sufficient statistic for a candidate knowledge cluster; This represents the marginal likelihood approximation obtained from the statistics.

[0014] Furthermore, in step six, the comprehensive training objective of the upstream joint training objective... for: in: To dynamically predict losses; For semantic reconstruction loss; The average KL loss over all samples; , and Let represent the weight coefficients of the environmental dynamic prediction loss, the language semantic reconstruction loss, and the KL constraint term, respectively; and: in: This represents the weighted KL alignment loss for the i-th potential sample; Representing latent variables Belongs to the The posterior probability of a knowledge cluster; Indicates the Kullback-Leibler divergence; This represents the distribution of learning conditions of the task encoder; and They represent the first The central location and distribution range of a knowledge cluster in the latent space; and Let represent the semantic embedding of the i-th reconstructed task output by the language semantic decoder and the original task semantic embedding of the sample, respectively. and These represent the predicted state for the next moment output by the dynamic prediction module and the actual state for the next moment obtained through interaction with the environment, respectively. This indicates the number of active knowledge clusters in the current dynamic knowledge space; This indicates the number of samples in the current training batch.

[0015] Furthermore, in step seven, the optimization objective of the conditional reinforcement learning navigation strategy is: in: Indicates the observation status; Indicates an action; As latent variables; Indicates the discount factor; For the reward function; Represents policy entropy; Temperature is a parameter used to balance strategy exploration and profit utilization; The critic network updates by minimizing the Bellman residual: in: Indicates the observed state at the next moment; Indicates the first The soft action value function is obtained by evaluating state, action, and potential task variables by a network of critics; Indicates the first Parameters of a network of critics; The target value is calculated by the target network: in: This indicates the action sampled according to the conditional policy at the next moment; Indicates the network parameters of the target commentator. The latent variables that represent the state and task semantics at the next moment; The policy network updates by maximizing action value while maintaining appropriate policy entropy; its loss function is expressed as: in: This represents the temperature parameter, used to balance policy entropy and action value. The parameter is The distribution of conditional strategies; This represents a reparameterized action sampling function based on a policy network; This represents the distribution of state transition samples stored in the experience replay pool; Indicates motion sampling noise It follows a standard normal distribution.

[0016] Furthermore, it also includes step eight: constructing a long-term task flow consisting of multiple navigation tasks in sequence to simulate the process of a mobile robot continuously facing different task conditions in an open environment; after continuous training and testing of the method, its continuous learning ability is evaluated using the final average success rate, average forgetting rate, and average forward transfer index; wherein, the final average success rate is used to measure the overall navigation ability of the system on each task after completing all task training, the average forgetting rate is used to measure the degree of decline in the performance of old tasks caused by new task training, and the average forward transfer is used to measure the promoting effect of historical knowledge on the learning efficiency and performance of subsequent tasks.

[0017] The beneficial effects of this invention are as follows: The adaptive navigation method for mobile robots used in continuous tasks, as described in this invention, has the following technical advantages: (1) Effectively suppress catastrophic forgetting and achieve continuous knowledge accumulation: By constructing a dynamic knowledge space based on the Dirichlet process hybrid model, the latent variables corresponding to different navigation tasks are modeled as knowledge clusters that can be adaptively grown or merged; combined with memorized variational inference to update sufficient statistics online, the knowledge of old tasks can be retained without replaying all historical data; the average forgetting rate of the method of this invention in long-term task flow is significantly lower than that of traditional fine-tuning and conventional continuous learning methods. (2) Promote positive knowledge transfer and improve the learning efficiency of new tasks: The potential task variables output by the task encoder are used as contextual conditions to input the downstream SAC policy network, enabling the policy to adaptively adjust the action output according to different task types; the existing knowledge clusters in the dynamic knowledge space can provide structured priors for new tasks and accelerate policy convergence; the superiority of the method of this invention in historical knowledge reuse is reflected in the cross-task performance matrix and forward transfer statistics. (3) Ensuring long-term operational stability and overall navigation success rate: The upstream joint training objectives (knowledge cluster alignment, semantic preservation, and dynamic prediction) enable latent variables to have both clusterability and controllability; the conditional policy network reduces the mutual interference of shared parameters between different tasks; the method of this invention can improve the average success rate in multiple consecutive navigation tasks, and the knowledge space visualization shows that similar tasks form a similar distribution, demonstrating good task organization and generalization capabilities. Attached Figure Description

[0018] To make the objectives, technical solutions, and beneficial effects of this invention clearer, the following figures are provided for illustration: Figure 1 This is a flowchart of the adaptive navigation method for mobile robots used in continuous tasks according to the present invention; Figure 2 A conceptual diagram illustrating long-term task flow learning and continuous knowledge accumulation; Figure 3 This is a framework diagram of the adaptive navigation method for mobile robots used in continuous tasks in this embodiment; Figure 4 This is a schematic diagram of the task encoder structure; Figure 5 A schematic diagram of a long-term navigation task flow for a mobile robot; Figure 6 The curves showing the average success rate of different methods in long-term task flows; Figure 7 This is a curve showing the phased success rate changes of each navigation task in a long-term task flow. Figure 8 Heatmap of cross-task performance matrix for different methods in long-term task flows; Figure 9Forgetting and forward transfer statistics for different methods in long-term task flows; (a) Forgetting value for each task; (b) Forward transfer value for each task; Figure 10 Visualization results of the phased evolution of the knowledge space; (a) Visualization results of the phased evolution of the knowledge space in the long-term task flow; (b) Visualization results of the correspondence between the final knowledge space distribution and specific task scenarios; Figure 11 The structure for deploying the trained strategy to a real mobile robot platform or real-world environment for verification; (a) navigation process in a static obstacle scenario; (b) navigation process in a narrow passage scenario; (c) navigation process in a wall-around scenario. Detailed Implementation

[0019] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, so that those skilled in the art can better understand and implement the present invention. However, the embodiments described are not intended to limit the present invention.

[0020] This embodiment aims to propose an adaptive navigation method for mobile robots in continuous tasks, which addresses the problems faced by mobile robots in continuous task scenarios, such as catastrophic forgetting, insufficient knowledge transfer ability, and low efficiency in adapting to new tasks.

[0021] Specifically, in this embodiment, the long-term navigation task is first represented as a task flow arriving in chronological order. A task encoder is then used to jointly encode robot state observation information and task semantic information to obtain latent task variables that characterize the features of the current task. Next, a dynamic knowledge space is established based on the Dirichlet Process Mixture Model (DPMM) to probabilistically model the latent task variables, enabling the knowledge space to automatically generate, assign, and merge knowledge clusters according to changes in task distribution. Furthermore, a memorized variational inference mechanism is introduced to update the knowledge space online without repeatedly accessing all historical task data, thereby achieving historical knowledge retention and new task knowledge expansion. Finally, the latent task variables are used as contextual conditions input to the reinforcement learning policy network to construct a conditional navigation control policy, enabling the mobile chassis to adaptively generate navigation actions based on the current task type and environmental state. Compared with existing single-task reinforcement learning, multi-task reinforcement learning, and conventional continuous learning methods, this embodiment can continuously organize and reuse existing knowledge in a long-term task flow, reduce policy interference between old and new tasks, and improve the success rate, knowledge transfer capability, and long-term operational stability of navigation tasks.

[0022] like Figure 1As shown in the figure, this embodiment describes an adaptive navigation method for mobile robots performing continuous tasks. The method comprises several stages: long-term task flow input construction, task encoder generation of latent task variables, dynamic knowledge space modeling, online updating of memorized variational inference, knowledge generation and merging, upstream joint training, downstream conditional SAC policy execution, and long-term task flow performance evaluation. This process starts with continuously arriving navigation tasks, forms a reusable task context through the upstream knowledge organization module, and then the downstream policy execution module completes the navigation action output.

[0023] like Figure 2 As shown, the difference between long-term task flow learning and traditional multi-task training lies in the fact that mobile robots face a sequence of tasks arriving sequentially in time, rather than a fixed set of tasks that can be accessed simultaneously during the training phase. Upon arrival of each navigation task, the robot needs to retain knowledge from past tasks while learning the current task, and apply transferable experience to subsequent tasks. As the task flow continues, the knowledge space should gradually form a stable, structured distribution to support knowledge reuse, knowledge expansion, and task adaptation.

[0024] like Figure 3 As shown, the overall framework of the mobile robot adaptive navigation method for continuous tasks in this embodiment consists of an upstream task reasoning and knowledge retention module, a downstream policy execution module, and a long-term task flow interaction environment. The upstream module receives state observations and task semantic information, generates latent task variables through a task encoder, and organizes task knowledge using a dynamic knowledge space. The downstream module inputs the latent task variables as contextual conditions into the SAC policy network and value network, outputting the mobile robot's navigation actions. The upstream and downstream modules are interconnected through latent task variables, enabling the task representation results to directly influence the policy learning process.

[0025] Specifically, the adaptive navigation method for mobile robots used in this embodiment for continuous tasks includes the following steps.

[0026] Step 1: Obtain state observation information and task semantic information from the long-term task flow. Specifically, obtain the state observation information and task semantic information of the mobile robot in the long-term task flow, and construct a joint input consisting of physical state observations and task language embeddings.

[0027] Assume the long-term navigation task flow arrives sequentially in time, and the mobile robot at time... Get current state observation and task semantic information State observation Used to describe the immediate relationship between the robot and the target, obstacles, and environmental boundaries; task semantic information. Used to describe the task type, objective requirements, and scenario differences of the current navigation task.

[0028] Specifically, the state observation vector can be represented as: in: This represents the Euclidean distance from the robot to the global target point; and Indicates the robot's position relative to the target point. direction and Positional deviation in direction; and This represents the robot's relative velocity to the target point in two directions; and This indicates the distance information of the robot relative to the local guide point; and These represent the robot's linear velocity and angular velocity at the current moment, respectively. This represents lidar observation information. The lidar scan can be uniformly divided into several angular intervals, and the nearest obstacle distance within each interval is taken to form a low-dimensional lidar feature, thereby reducing the dimensionality of the original sensor data and retaining obstacle avoidance related information.

[0029] Task semantic information can be obtained from a textual description of the task, such as a linguistic description of the navigation objective, obstacle distribution, passage structure, or task difficulty. This task semantic information is then encoded into semantic embeddings by a pre-trained language model. Together with state observations, they constitute the input to the task encoder: in: Indicates the input to the task encoder; Indicates state observation; It represents the semantic information of the task.

[0030] Step 2: Construct a task encoder and generate latent task variables. Specifically, the state observation information and task semantic information are jointly mapped into latent task variables, which simultaneously represent the environmental state, task objective, and semantic differences of the current task.

[0031] like Figure 4 As shown, the task encoder receives status observations. and task semantic embedding Furthermore, it outputs the mean and covariance parameters of the latent distribution through multi-layer nonlinear mapping. Unlike directly outputting discrete task labels, this embodiment represents the current task as a continuous latent distribution, enabling it to simultaneously express task differences and task uncertainty.

[0032] The task encoder learns the following conditional distribution: in: Indicates the task encoder parameters; and These represent the mean and standard deviation of the latent distribution output by the encoder, respectively. Indicates state observation; Represents task semantic information; Represents potential task variables; This represents a diagonal matrix. To make the latent variable sampling process differentiable, this embodiment uses a reparameterization method to construct the latent task variables: in: Let represent a random noise vector that follows a standard normal distribution; This represents the Hadamard element-wise multiplication operation. The resulting latent task variables... It serves both as a modeling object for the upstream dynamic knowledge space and as contextual input for the downstream conditional policy network. This design enables the task encoder to do more than just compress information; it generates task representations that facilitate knowledge attribution, knowledge expansion, and policy conditionalization.

[0033] Step 3: Construct a dynamic knowledge space based on the Dirichlet process hybrid model. Treat latent task variables as samples generated from several knowledge clusters, and ensure that the number of knowledge clusters adapts to the progression of the long-term task flow.

[0034] To adapt to the characteristics of long-term task flows, such as unknown task numbers, variable task boundaries, and continuously changing task distribution, this embodiment employs a Dirichlet process hybrid model to construct a dynamic knowledge space. The dynamic knowledge space treats the latent variables output by the task encoder as samples generated from several knowledge clusters, allowing the number of knowledge clusters to adaptively change as the task flow progresses. The generation process of the dynamic knowledge space can be represented as follows: in: Represents the prior distribution of the parameters of the knowledge cluster; This represents the mixed weight distribution induced by the stick-breaking process; Represents category distribution; This represents the data generation distribution given cluster parameters; Denotes the first sample obtained from the prior distribution. Each knowledge cluster parameter; Represent the prior distribution hyperparameters; This represents a weight vector composed of the mixed weights of each knowledge cluster; Indicates the first The knowledge cluster affiliation variable corresponding to each potential variable; Indicate latent variables; The concentration parameters for the Dirichlet process.

[0035] For continuous latent variables, this embodiment models each knowledge cluster as a Gaussian distribution: in: and They represent the first The modeling method identifies the central location and distribution range of each knowledge cluster within the latent space. This approach allows the dynamic knowledge space to both preserve existing task patterns and adaptively expand when new task patterns emerge.

[0036] Step 4: Update the dynamic knowledge space online based on memoized variational inference. Calculate the soft attribution probability of latent task variables to each knowledge cluster, and incrementally update the knowledge cluster parameters using sufficient statistics.

[0037] When new potential task variables arrive, this embodiment uses memoized variational inference to approximate their knowledge attribution, and achieves online updates by maintaining the posterior parameters and sufficient statistics of each knowledge cluster. The variational distribution adopts the following factorization form: in: Representing latent variables The posterior distribution of cluster affiliation; This represents the posterior distribution of the variable representing the folding stick; Indicates the first The posterior distribution of the parameters of a knowledge cluster; The set of cluster affiliation variables representing all latent variables; Indicates the first The knowledge cluster affiliation variable of each latent variable; This represents the set of stick-breaking variables during the stick-breaking process; Indicates the first The folding rod variable corresponding to each knowledge cluster; and They represent the first The central location and distribution range of a knowledge cluster in the latent space; This indicates the number of latent variable samples currently used for variational inference; This represents the number of active knowledge clusters in the current dynamic knowledge space. By maximizing the lower bound of evidence, the variational distribution approximates the true posterior distribution.

[0038] For the current latent variables It belongs to the first The unnormalized affiliation weights of each knowledge cluster are: After further normalization, the sample is obtained Belongs to the The posterior probability of a knowledge cluster: Subsequently, this embodiment incrementally updates each knowledge cluster using sufficient statistics: in, Description of the The current effective sample quality of each knowledge cluster and The first-order and second-order statistical information of the cluster samples are described respectively. Since the update process relies on cluster-level statistics rather than all historical samples, knowledge preservation and updating can be completed without repeatedly accessing the complete historical task data.

[0039] Step 5: Construct a knowledge generation mechanism and a knowledge merging mechanism based on soft attribution probability. When the current sample cannot be fully explained by the existing knowledge cluster, a new knowledge cluster is generated. When there is redundancy between existing knowledge clusters, merging is performed, thereby realizing the dynamic evolution of the knowledge space.

[0040] In this embodiment, the triggering condition for the knowledge generation mechanism is the maximum belonging probability of the current sample to an existing knowledge cluster. Below a set threshold; the maximum probability of the current sample belonging to an existing knowledge cluster. Represented as: in: Representing latent variables Belongs to the The posterior probability of a knowledge cluster; Indicates the number of knowledge clusters. When When the value is below the set threshold, it indicates that the current potential variable cannot be fully explained by the existing knowledge cluster. The system then includes it in the candidate new generation set and attempts to generate a new knowledge cluster.

[0041] When two existing knowledge clusters are highly similar in the latent space, this embodiment determines whether to perform the merging based on the marginal likelihood gain before and after merging: in: and Let each represent a sufficient statistic for a candidate knowledge cluster; This represents the marginal likelihood approximation calculated from statistics. In this embodiment, the execution condition of the knowledge merging mechanism is the marginal likelihood gain before and after merging. Greater than zero, that is, if If the value is greater than zero, the merging operation is accepted, thereby compressing redundant knowledge clusters and maintaining a compact knowledge space structure.

[0042] Step 6: Construct an upstream joint training objective so that the latent variables output by the task encoder simultaneously meet the requirements of knowledge cluster alignment, task semantic preservation, and dynamic environmental prediction.

[0043] To ensure that the latent variables output by the task encoder simultaneously possess task discriminability, knowledge clusterability, and control relevance, this embodiment constructs an upstream joint training objective. This upstream joint training objective includes a knowledge cluster alignment term, a task semantic preservation term, and an environmental dynamic prediction term. The knowledge cluster alignment term is used to constrain the encoder output distribution to maintain consistency with the knowledge cluster distribution in the dynamic knowledge space, and its form can be expressed as: in: Indicates the first Weighted KL alignment loss for potential samples; Representing latent variables Belongs to the The posterior probability of a knowledge cluster; This represents the Kullback-Leibler divergence, used to measure the difference between two probability distributions; This represents the distribution of learning conditions of the task encoder; and They represent the first The central location and distribution range of a knowledge cluster in the latent space.

[0044] Furthermore, to improve the interpretability and controllability of latent variables, a generative auxiliary task is introduced in the upstream module, defining the semantic reconstruction loss and dynamic prediction loss as follows: in: and These represent the outputs of the language semantic decoder. The semantic embedding of the reconstruction task and the original task semantic embedding of the sample; and These represent the predicted state for the next moment output by the dynamic prediction module and the actual state for the next moment obtained through interaction with the environment, respectively. This indicates the number of active knowledge clusters in the current dynamic knowledge space; This indicates the number of samples in the current training batch.

[0045] The task semantic preservation term is used to maintain the ability of latent variables to express task semantic information, while the environmental dynamic prediction term is used to enhance the control relevance of latent variables on state transition patterns. The combined upstream training objective is: in: To dynamically predict losses; For semantic reconstruction loss; The average KL loss over all samples; , and These represent the weight coefficients of the environmental dynamic prediction loss, the linguistic semantic reconstruction loss, and the KL constraint term, respectively. Through the above joint objective, the latent variables output by the task encoder can form a separable, aggregateable structure in the knowledge space that is relevant to the control task.

[0046] Step 7: Construct a conditional SAC navigation policy. Input the latent task variables as contextual conditions into the downstream Soft Actor-Critic (SAC) policy network and value network to construct a conditional reinforcement learning navigation policy, enabling the policy to generate differentiated navigation actions based on different task contexts.

[0047] In the downstream strategy execution module, this embodiment will include potential task variables. As a contextual condition, with state observation The SAC policy network and value network are input together. Therefore, the traditional policy... Extended to a conditional policy: The optimization objective of the downstream control strategy can be written as: in: Indicates the observation status; Indicates an action; As latent variables; Indicates the discount factor; For the reward function; Represents policy entropy; The temperature parameter is used to balance strategy exploration and reward utilization. Compared to the standard SAC, this formula differs in that both policy entropy and policy distribution are defined as condition variables. Down.

[0048] The parameters of the Q network are updated by minimizing the Bellman residuals: in: Indicates the observed state at the next moment; Indicates the first The soft action value function is obtained by evaluating state, action, and potential task variables by a network of critics; Indicates the first Parameters of a network of critics; The target value is calculated by the target network: in: This indicates the action sampled according to the conditional policy at the next moment; Indicates the network parameters of the target commentator. The potential variables represent the state and task semantics at the next moment.

[0049] The policy network updates by maximizing action value while maintaining appropriate policy entropy; its loss function can be expressed as: in: This represents the temperature parameter, used to balance policy entropy and action value. The parameter is The distribution of conditional strategies; This represents a reparameterized action sampling function based on a policy network; This represents the distribution of state transition samples stored in the experience replay pool; Indicates motion sampling noise It follows a standard normal distribution.

[0050] Through the aforementioned conditional policy learning method, downstream policies can adaptively adjust action outputs according to different task contexts, reducing direct interference between different tasks on shared policy parameters and improving knowledge transfer and adaptability to new tasks in long-term task flows.

[0051] like Figure 5As shown, this embodiment constructs a long-term task flow consisting of multiple navigation tasks in sequence to simulate the process of a mobile robot continuously facing different obstacle structures, different target locations, and different dynamic changing conditions in an open environment. Each navigation task arrives sequentially in a fixed order; the robot proceeds to the next task after completing training on the current task, without relying on all tasks being accessible simultaneously.

[0052] At the start of each training round, the positions of the mobile robot, target point, and obstacles are randomly generated within a set range. If the task includes dynamic obstacles or dynamic target points, their movement direction and speed are also randomly set, and they return to the area upon reaching the environmental boundary using a reflection method. Through this randomization process, the same task can exhibit different instance distributions during training, thereby improving the policy's adaptability to changes within the task.

[0053] In complex area navigation tasks, this embodiment can also set local guide points. Local guide points are used to break down long or complex global navigation tasks into several easier-to-explore intermediate stages. When the robot approaches a local guide point, it receives an additional reward, thereby reducing the difficulty of exploring complex environments by relying solely on global objectives.

[0054] The reward function guides the robot to complete target navigation safely. Rewards include target arrival rewards, collision penalties, proximity rewards, velocity smoothing penalties, and local guide point rewards. Through this composite reward design, the robot can learn navigation strategies that balance goal achievement, obstacle avoidance safety, and smooth motion in long-term task flows.

[0055] Step 8: Conduct long-term task flow experiments and real-world validation. A long-term task flow consisting of multiple navigation tasks is constructed. The method is continuously trained and tested, and its continuous learning ability is evaluated using metrics such as final average success rate, average forgetting rate, and average forward transfer. Specifically, in this step, a long-term task flow consisting of multiple navigation tasks is constructed sequentially to simulate the process of a mobile robot continuously facing different task conditions in an open environment. After continuous training and testing of the method, its continuous learning ability is evaluated using the final average success rate, average forgetting rate, and average forward transfer. The final average success rate measures the system's overall navigation ability on each task after completing all task training; the average forgetting rate measures the degree of performance degradation caused by new task training on old tasks; and the average forward transfer measures the promoting effect of historical knowledge on the learning efficiency and performance of subsequent tasks.

[0056] To verify the effectiveness of the method described in this embodiment, multiple baseline methods were compared, including single-task training, direct fine-tuning, multi-task training, and conventional continuous learning methods. Evaluation metrics included final average success rate, average forgetting rate, and average forward transfer. Specifically, the final average success rate measures the system's overall navigation capability across all tasks after completing training for all tasks; the average forgetting rate measures the degree of performance degradation caused by training on new tasks on older tasks; and the average forward transfer measures the positive impact of historical knowledge on the learning efficiency and performance of subsequent tasks.

[0057] like Figure 6 As shown, the average success rate curves of different methods in long-term task flows reflect the changes in overall performance as the task progresses during training. The figure shows that the method in this embodiment can maintain a high task success rate even in the later stages of continuous task training, indicating that the dynamic knowledge space and conditional policy structure can effectively alleviate the performance degradation problem in long-term task flows.

[0058] like Figure 7 As shown, the success rate curves for each navigation task reflect the learning and retention of individual tasks in the long-term task flow. Each task maintains a relatively stable success rate after completing its corresponding training phase, indicating that this embodiment can effectively maintain the performance of older tasks while learning new ones.

[0059] like Figure 8 As shown, the cross-task performance matrix heatmap illustrates the model's test performance on all known tasks and some subsequent tasks after completing each task training phase. Performance changes at different points in the graph reflect interference relationships between tasks, knowledge retention effectiveness, and cross-task generalization ability.

[0060] like Figure 9 As shown, forgetting statistics and forward transfer statistics quantify the degree of performance degradation in old tasks and the degree to which historical knowledge promotes learning in new tasks, respectively. The results in the figure demonstrate that this embodiment can reduce forgetting in long-term task flows and promote the transfer of goal-approaching and basic obstacle avoidance knowledge learned in early tasks to subsequent tasks.

[0061] like Figure 10 As shown, the visualization results of the phased evolution of the knowledge space demonstrate the distribution changes of latent task variables during the training process. The final correspondence between the knowledge space distribution and the specific task scenario shows the organization of task knowledge. The figure shows that similar tasks form a similar distribution in the latent space, while tasks with significant differences are assigned to different knowledge regions, indicating that the dynamic knowledge space constructed in this embodiment has good task organization capabilities.

[0062] like Figure 11As shown, the trained strategy is deployed to a real mobile robot platform or a real-world environment for verification. The figure illustrates the robot's navigation process in different real-world scenarios, such as static obstacles, narrow passages, and wall-navigation. The results demonstrate that the method of this invention not only enables continuous knowledge accumulation in simulated task flows but also possesses adaptability to real-world environments.

[0063] In summary, this embodiment, through the collaborative design of a task encoder, dynamic knowledge space, memorized variational inference, and conditional SAC policy network, enables a mobile robot to continuously organize and reuse historical knowledge in a long-term navigation task flow that arrives in chronological order, and to quickly adapt when new tasks arrive. Experimental evaluation results can verify the effectiveness of the invention from multiple perspectives, including average success rate, forgetting suppression, forward transfer, knowledge space structure, and navigation process in real-world environments.

[0064] Compared with the prior art, this embodiment has at least the following beneficial effects: (1) In this embodiment, the mobile robot navigation task is regarded as a long-term task flow that arrives continuously in chronological order. It does not rely on all tasks being accessible at the same time, thus avoiding the dependence of traditional multi-task reinforcement learning on fixed task sets and repeated access to historical tasks. This is more in line with the application characteristics of mobile robots running continuously in real open environments and tasks constantly changing.

[0065] (2) In this embodiment, the robot state observation and task semantic information are fused by the task encoder to construct a potential task representation with task discrimination capability; further, a dynamic knowledge space is established based on DPMM, and knowledge attribution, generation, updating and merging are realized by combining memorized variational inference, so that the system can continuously organize existing knowledge and expand new task knowledge in long-term task flow, thereby alleviating catastrophic forgetting and improving the adaptability to new tasks.

[0066] (3) In this embodiment, potential task variables are introduced as contextual conditions into the downstream SAC policy network and value network, enabling the navigation strategy to adaptively adjust the action output according to different task types and environmental states, thereby reducing the mutual interference between different tasks on shared policy parameters. At the same time, the method is comprehensively evaluated by indicators such as final average success rate, average forgetting rate, and average forward transfer, which can intuitively reflect its effectiveness in task switching, knowledge retention, and knowledge transfer.

[0067] The above-described embodiments are merely preferred embodiments provided to fully illustrate the present invention, and the scope of protection of the present invention is not limited thereto. Equivalent substitutions or modifications made by those skilled in the art based on the present invention are all within the scope of protection of the present invention. The scope of protection of the present invention is defined by the claims.

Claims

1. An adaptive navigation method for a mobile robot for continuous tasks, characterized in that: Includes the following steps: Step 1: Obtain the state observation information and task semantic information of the mobile robot in the long-term task flow, and construct a joint input; Step 2: Construct a task encoder to jointly map the state observation information and task semantic information into potential task variables; Step 3: Construct a dynamic knowledge space based on the Dirichlet process hybrid model, treating the potential task variables as samples generated by several knowledge clusters, and enabling the number of knowledge clusters to adaptively change with the progress of the long-term task flow; Step 4: Update the dynamic knowledge space online based on memorized variational inference, calculate the soft attribution probability of the current latent task variable to each knowledge cluster, and incrementally update the knowledge cluster parameters using sufficient statistics; Step 5: Construct a knowledge generation mechanism and a knowledge merging mechanism based on the soft attribution probability. When the current sample cannot be fully explained by the existing knowledge cluster, a new knowledge cluster is generated. When there is redundancy between existing knowledge clusters, merging is performed. Step 6: Construct an upstream joint training objective so that the latent variables output by the task encoder simultaneously meet the requirements of knowledge cluster alignment, task semantic preservation, and dynamic environmental prediction. Step 7: Input the latent task variables as contextual conditions into the downstream soft actor critic algorithm policy network and value network to construct a conditional reinforcement learning navigation policy, so that the policy generates differentiated navigation actions according to different task contexts.

2. The adaptive navigation method for mobile robots for continuous tasks according to claim 1, characterized in that: In step one, the state observation information Represented as: in: This represents the Euclidean distance from the robot to the global target point; and Indicates the robot's position relative to the target point. direction and Positional deviation in direction; and This represents the robot's relative velocity to the target point in two directions; and This indicates the distance information of the robot relative to the local guide point; and These represent the robot's linear velocity and angular velocity at the current moment, respectively. This indicates lidar observation information; The task semantic information, after being encoded by a pre-trained language model, forms a semantic embedding, which, together with the state observation, constitutes the input of the task encoder, as follows: in: Indicates the input to the task encoder; Indicates state observation; It represents the semantic information of the task.

3. The adaptive navigation method for mobile robots for continuous tasks according to claim 1, characterized in that: In step two, the learning condition distribution of the task encoder is as follows: in: Indicates the task encoder parameters; and These represent the mean and standard deviation of the latent distribution output by the encoder, respectively. Indicates state observation; Represents task semantic information; Represents potential task variables; Represents a diagonal matrix; Constructing latent task variables using reparameterization: in: This represents a random noise vector that follows a standard normal distribution. This indicates the Hadamard element-wise multiplication operation.

4. The adaptive navigation method for mobile robots for continuous tasks according to claim 1, characterized in that: In step three, the process of constructing the dynamic knowledge space is represented as follows: in: Represents the prior distribution of the parameters of the knowledge cluster; This represents the mixed weight distribution induced by the stick-breaking process; Represents category distribution; This represents the data generation distribution given cluster parameters; Denotes the first sample obtained from the prior distribution. Each knowledge cluster parameter; Represent the prior distribution hyperparameters; This represents a weight vector composed of the mixed weights of each knowledge cluster; Indicates the first The knowledge cluster affiliation variable corresponding to each potential variable; Indicate latent variables; For the concentration parameters of the Dirichlet process; The Dirichlet process mixture model models each knowledge cluster as a Gaussian distribution: in: and They represent the first The central location and distribution range of a knowledge cluster in the latent space.

5. The adaptive navigation method for mobile robots for continuous tasks according to claim 1, characterized in that: In step four, the memorized variational inference adopts a variational distribution in factorized form: in: Representing latent variables The posterior distribution of cluster affiliation; This represents the posterior distribution of the variable representing the folding stick; Indicates the first The posterior distribution of the parameters of a knowledge cluster; The set of cluster affiliation variables representing all latent variables; This represents the knowledge cluster affiliation variable of the i-th latent variable; This represents the set of stick-breaking variables during the stick-breaking process; This represents the folding rod variable corresponding to the k-th knowledge cluster; and They represent the first The central location and distribution range of a knowledge cluster in the latent space; This indicates the number of latent variable samples currently used for variational inference; This indicates the number of active knowledge clusters in the current dynamic knowledge space.

6. The adaptive navigation method for a mobile robot for continuous tasks according to claim 5, characterized in that: In step four, the current latent variables Belongs to the The posterior probability of a knowledge cluster Calculated using the following formula: in: Represents the variational distribution Expectation calculation; This represents the mixed weights of the k-th knowledge cluster; Utilizing sufficient statistics , , Each knowledge cluster is incrementally updated; among which: Description of the The current effective sample quality of each knowledge cluster; and Describe the first First-order and second-order statistical information of a knowledge cluster sample.

7. The adaptive navigation method for a mobile robot for continuous tasks according to claim 1, characterized in that: In step five, the trigger condition for the knowledge generation mechanism is the maximum probability of the current sample belonging to an existing knowledge cluster. The current sample's maximum probability of belonging to an existing knowledge cluster is below a set threshold. Represented as: in: Representing latent variables Belongs to the The posterior probability of a knowledge cluster; Indicates the number of knowledge clusters; The execution condition of the knowledge merging mechanism is the marginal likelihood gain before and after the merging. Greater than zero; the marginal likelihood gain before and after the merger Represented as: in: and Let each represent a sufficient statistic for a candidate knowledge cluster; This represents the marginal likelihood approximation obtained from the statistics.

8. The adaptive navigation method for a mobile robot for continuous tasks according to claim 1, characterized in that: In step six, the comprehensive training objective of the upstream joint training objective... for: in: To dynamically predict losses; For semantic reconstruction loss; The average KL loss over all samples; , and Let represent the weight coefficients of the environmental dynamic prediction loss, the language semantic reconstruction loss, and the KL constraint term, respectively; and: in: Indicates the first Weighted KL alignment loss for potential samples; Representing latent variables Belongs to the The posterior probability of a knowledge cluster; This represents the Kullback-Leibler divergence, used to measure the difference between two probability distributions; This represents the distribution of learning conditions of the task encoder; and They represent the first The central location and distribution range of a knowledge cluster in the latent space; and These represent the outputs of the language semantic decoder. The semantic embedding of the reconstruction task and the original task semantic embedding of the sample; and These represent the predicted state for the next moment output by the dynamic prediction module and the actual state for the next moment obtained through interaction with the environment, respectively. This indicates the number of active knowledge clusters in the current dynamic knowledge space; This indicates the number of samples in the current training batch.

9. The adaptive navigation method for a mobile robot for continuous tasks according to claim 1, characterized in that: In step seven, the optimization objective of the conditional reinforcement learning navigation strategy is: in: Indicates the observation status; Indicates an action; As latent variables; Indicates the discount factor; For the reward function; Represents policy entropy; Temperature is a parameter used to balance strategy exploration and profit utilization; The critic network updates by minimizing the Bellman residual: in: Indicates the observed state at the next moment; Indicates the first The soft action value function is obtained by evaluating state, action, and potential task variables by a network of critics; Indicates the first Parameters of a network of critics; The target value is calculated by the target network: in: This indicates the action sampled according to the conditional policy at the next moment; Indicates the network parameters of the target commentator. The latent variables that represent the state and task semantics at the next moment; The policy network updates by maximizing action value while maintaining appropriate policy entropy; its loss function is expressed as: in: This represents the temperature parameter, used to balance policy entropy and action value. The parameter is The distribution of conditional strategies; This represents a reparameterized action sampling function based on a policy network; This represents the distribution of state transition samples stored in the experience replay pool; Indicates motion sampling noise It follows a standard normal distribution.

10. The adaptive navigation method for a mobile robot for continuous tasks according to claim 1, characterized in that: The method also includes step eight: constructing a long-term task flow consisting of multiple navigation tasks in sequence to simulate the process of a mobile robot continuously facing different task conditions in an open environment; after continuous training and testing of the method, its continuous learning ability is evaluated using the final average success rate, average forgetting rate, and average forward transfer index; wherein, the final average success rate is used to measure the overall navigation ability of the system on each task after completing all task training, the average forgetting rate is used to measure the degree of decline in the performance of old tasks caused by new task training, and the average forward transfer is used to measure the promoting effect of historical knowledge on the learning efficiency and performance of subsequent tasks.

Images