Mobile robot visual navigation method based on space-time attention and asynchronous element variation strategy

Abstract

The invention discloses a mobile robot visual navigation method based on space-time attention and an asynchronous element variation strategy. The method comprises the following steps: S100, defining a state space and an action space of a visual navigation task; s200, hierarchical semantic information modeling and context vectors are introduced; s300, adopting a space-time adaptive convolution attention model, fusing causal expansion convolution and a time sequence adaptive multi-dimensional attention mechanism, and dynamically capturing long-term space-time dependence characteristics of the environment; s400, constructing an asynchronous meta-variation strategy algorithm, and modeling the uncertainty of navigation strategy parameters by combining an asynchronous parallelized strategy gradient updating and meta-variation inference method; s500, designing a multi-target reward function and training a navigation strategy; and S600, migrating the trained navigation strategy model to a real machine. The visual navigation method provided by the invention can effectively solve the defects of an existing navigation method in the aspects of processing long-term space-time dependence and generalization, and realizes efficient autonomous navigation of the mobile robot in an unknown complex environment.

Description

Technical Field

[0001] This invention relates to the field of autonomous navigation technology for mobile robots, specifically a visual navigation method for mobile robots based on spatiotemporal attention and asynchronous metavariable strategies. Background Technology

[0002] With the rapid development of robotics technology, mobile robots are increasingly widely used in many fields such as security inspection, warehousing and logistics, and home services. This places higher demands on the autonomous visual navigation capabilities of mobile robots in complex and ever-changing environments. In practical application scenarios, robots need to be able to autonomously plan the path from the starting point to the target point based on visual information and adapt to various changes in unknown environments.

[0003] Traditional mobile robot navigation methods typically rely on pre-built environment maps or fixed navigation rules, making it difficult to cope with complex and ever-changing real-world environments. In recent years, deep reinforcement learning technology has brought new possibilities to robot visual navigation, enabling robots to learn navigation strategies directly from raw visual input. However, current deep reinforcement learning-based visual navigation methods struggle to effectively capture and model long-term spatiotemporal dependencies in the environment, leading to unstable navigation strategy performance in complex environments. Furthermore, existing models often rely on specific training datasets, exhibiting poor generalization ability and requiring retraining or extensive fine-tuning when facing new environments or tasks.

[0004] Therefore, a visual navigation method is needed that can effectively capture long-term spatiotemporal dependencies and has strong generalization ability, so that mobile robots can achieve adaptive, efficient and stable autonomous navigation in unknown and complex environments. Summary of the Invention

[0005] The purpose of this invention is to provide a visual navigation method for mobile robots based on spatiotemporal attention and asynchronous metavariable strategies, so as to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A visual navigation method for mobile robots based on spatiotemporal attention and asynchronous metavariable strategy is proposed, and its steps are as follows:

[0008] S100: Defines the state space and action space of the visual navigation task, acquires visual input data from the environment, and performs preprocessing.

[0009] S200: Introduces hierarchical semantic information modeling and context vectors to enhance the semantic understanding ability and decision-making efficiency of intelligent agents in navigation tasks;

[0010] S300: Employs a spatiotemporal adaptive convolutional attention model, which integrates causal dilated convolution and temporally adaptive multidimensional attention mechanism to dynamically capture the long-term spatiotemporal dependency features of the environment.

[0011] S400: Construct an asynchronous metavariable policy algorithm, combining asynchronous parallelized policy gradient update with metavariable inference method to model the uncertainty of navigation policy parameters;

[0012] S500: Design a multi-objective reward function and train the navigation strategy;

[0013] S600: Transfer the trained navigation strategy model to the actual device.

[0014] As a further aspect of the present invention: the state space includes the word embeddings of the current RGB image and the target object, and the action space includes a discrete set of navigation actions.

[0015] As a further aspect of the present invention: S500 specifically involves training an agent in a simulation environment using the spatiotemporal adaptive convolutional attention model and the asynchronous metavariable policy algorithm, optimizing policy parameters through a multi-objective reward function, and ultimately achieving efficient autonomous navigation in complex environments.

[0016] As a further aspect of the present invention: S200 specifically involves hierarchical semantic modeling, which enables the robot to identify semantic and spatial relationships between objects. By utilizing parent-child object relationships, the robot is guided to prioritize the exploration of potential target areas, thereby improving navigation efficiency. The context vector combines the visibility, location, and semantic similarity of objects with target objects, helping the robot make more accurate navigation decisions based on this information. By integrating these two aspects, the robot can more intelligently select target areas and reduce ineffective exploration.

[0017] As a further aspect of the present invention: S100 specifically includes:

[0018] S110: Define the state space. The robot uses the Kinect camera 5 to capture the current single-frame RGB image and the corresponding depth image to capture dynamic changes in the environment. These raw image data may undergo preliminary processing by the model described in S200 before being input into the policy network. In order for the robot to understand its navigation target, the target described in natural language will be converted into a fixed-dimensional numerical vector through a pre-trained word vector model. These two parts of information will be concatenated into a unified state vector, which will be used as the input to the subsequent policy network.

[0019] S120: Define the motion space and design a set of motion instructions that the robot can execute. This motion space is usually set as a discrete set, namely, moving forward in a straight line for 0.25 meters, rotating 30 degrees to the left, rotating 30 degrees to the right, tilting the camera up 30 degrees, tilting the camera down 30 degrees, and an action called Done that indicates that the robot has found the target.

[0020] As a further aspect of the present invention: S200 specifically includes:

[0021] S210: A general indoor scene knowledge graph is constructed. When the robot receives a navigation target, it can query the knowledge graph to find its possible parent objects. During navigation, if the target is not visible, but its parent object is visible or its possible location is known, the strategy can be guided to prioritize exploring the area where the parent object is located.

[0022] S220: At each time step of robot navigation, the system extracts a series of contextual information from the current visual observation, for each object detected in the scene. j A context vector C can be constructed. j Context vector C j =[b,x c ,y c bbox, CS j ], where: b indicates whether the target object is visible in the field of view, (x c ,y c ) is an object o j Normalized center coordinates in the image; bbox is the object o j The normalized area of ​​the bounding box in the image; CS j It is an object o j The semantic similarity between the target and the current navigation target is obtained by calculating the cosine similarity between the GloVe word embedding vectors: Among them, g(o j g(t) and g(t) represent objects o j The GloVe word embedding vector of the target object t.

[0023] As a further aspect of the present invention, S300 specifically includes:

[0024] S310: Applying causal dilated convolution to process visual feature sequences, including:

[0025] S311: The input to the model is a sequence of visual features extracted by CNN. Where N is the batch size, C feat Where T is the number of input feature channels and T is the sequence length;

[0026] S312: Design a residual block structure containing two dilated convolutional layers. The stride of the first dilated convolutional layer is typically set to 1 to maintain the sequence length. After convolution, a causal pruning operation is performed to ensure the causality of the convolution. This is usually followed by a LeakyReLU activation function and a Dropout layer to prevent overfitting. The second dilated convolutional layer has the same structure. To promote gradient propagation and information flow, a residual connection is introduced to add the input and output of this module, and then it is again activated by the LeakyReLU activation function. The output of this temporal convolutional module... It is then passed to the attention mechanism described in S320;

[0027] S320: The spatiotemporal attention mechanism is applied to optimize spatiotemporal features. The specific implementation method is as follows:

[0028] S321: Spatiotemporal attention modules typically consist of two parts: temporal attention and channel attention. For temporal attention, input features can be... The dimensions are first rearranged from (N,C,T) to (T,N,C), and then a multi-head self-attention mechanism is applied to learn the dependencies and importance weights between different time steps in the sequence, resulting in temporally attention-weighted features. For channel attention, Global average pooling is performed on the features along the time dimension to obtain the descriptor for each channel. This descriptor is then passed through a multilayer perceptron with one or two fully connected layers, and finally, an attention weight for each channel is generated using a sigmoid activation function. These weights are then element-wise multiplied back to the previous layer. Above, we obtain the channel attention-weighted features. The final output of the spatiotemporal attention module It is a feature after processing with time attention. Features after further channel attention processing Perform element-wise addition, and then pass the sum through a LeakyReLU activation function to obtain the result:

[0029] S322: At the end of each attention temporal module, the output of the temporal convolutional module of S310 is usually... Output of the spatiotemporal attention module of S320 The residues are then added together again via residual connections, and then passed through an activation function to form the final output of this attention-based temporal module. This output will serve as the input to the next attention temporal module. When it is the last module, its output is the final spatiotemporal feature representation of the entire spatiotemporal adaptive convolutional attention model, which will be used by the subsequent policy learning module.

[0030] As a further aspect of the present invention, S400 specifically includes:

[0031] S410: Set up a global network containing the parameters of the policy network and value network, start multiple parallel training processes, each process has a local copy of the global network parameters, and interacts with the environment in an independent environment instance to collect empirical data (s t ,a t ,r t ,s t+1 Each process calculates its policy gradient and value loss gradient locally, and then asynchronously sends the calculated gradients to the central parameter server to update the global network parameters. It then pulls the latest parameters from the global network to its local copy and continues the next round of interaction and learning.

[0032] S420: Perform inner-layer optimization. For each sampled task k, one or more processes will use the current global meta-policy parameter θ as a starting point; perform minimal interactions and collect data on this task. Calculate the task-specific self-supervised interaction loss L int Then, the policy parameters are updated using gradient descent to obtain task-specific parameters. After performing outer-layer optimization and obtaining task-specific parameters θ'k through inner-layer optimization, the navigation performance is evaluated on task k using these adapted parameters, and a meta-target loss, i.e., the navigation loss, is calculated. in The data collected on this task is used to evaluate the meta-performance; the goal of the outer optimization is to minimize the expected navigation loss on all sampled tasks. The meta-gradient is calculated with respect to the original meta-parameters θ, i.e. These meta-gradients are then used to update the global meta-policy parameters. In the asynchronous framework, different processes can process the inner optimization and outer loss calculation of different tasks in parallel and asynchronously aggregate the meta-gradients to the global parameter server.

[0033] S430: To enhance the exploratory and robust nature of the policy, the policy network is designed as a probabilistic model, introducing a latent variable z. The policy π(a|s,z) depends not only on the state s but also on the latent variable z sampled from a prior distribution p(z). Since directly optimizing the marginal log-likelihood log∫p(a|s,z)p(z)dz is difficult, a variational approximation posterior distribution q(z|s,a) is introduced, and the lower bound of evidence E is optimized. q(z|s) [logp(a|s,z)]-D KL (q(z|s)||p(z)), where q(z|s) is typically parameterized by an encoder network that outputs the mean μ of a Gaussian distribution. z (s) and standard deviation σz (s); to make the sampling process differentiable, the reparameterization technique is used when sampling z from q(z|s): z = μ z (s)+σ z (s), where The final loss function will include the policy learning objective and the KL divergence regularization term D. KL (q(z|s)||p(z)), where the hyperparameter β is used to balance these two terms:

[0034] As a further aspect of the present invention: the design of the reward function in S500 is crucial for guiding the behavior of the reinforcement learning agent, and it is defined as follows:

[0035]

[0036] Among them, R p It is a partial reward when the parent object is visible, calculated using the formula R. p =R t ×Pr(t|p)×k, where R t =5, k=0.1, R t This is the reward when the target object is visible at the end of the task. If both the parent and target objects are visible at the end of the task, the total reward R is obtained. t +R p Otherwise, a small negative reward of -0.01 is given to encourage exploration.

[0037] As a further aspect of the present invention, S600 specifically includes:

[0038] S610: Equips mobile robots with an RGB-D camera, Kinect, to acquire color images and depth information of the environment;

[0039] S620: Install and configure ROS on the robot's computer, create a ROS node to receive and process image data from the RGB-D camera, and create another ROS node to run the trained navigation strategy model; this node will subscribe to the processed sensor data and target information, and issue motion commands to the robot to control the movement of the robot's wheels;

[0040] S630: The navigation strategy model weights obtained from simulation training are loaded into the neural network deployed in the ROS node. The robot collects environmental visual data in real time through the RGB-D camera. After preprocessing, the data is input into the spatiotemporal adaptive convolutional attention model to extract the current spatiotemporal feature representation. These features, together with the target semantic description of the current navigation task, are provided as input to the policy network of the asynchronous metavariable policy algorithm. The policy network outputs an action decision based on the current state and the target. The decided action is converted into a command that can be executed by the robot's underlying controller to drive the robot to move until the navigation task is completed.

[0041] Compared with the prior art, the beneficial effects of the present invention are:

[0042] 1. The causal dilated convolution and spatiotemporal attention mechanism in the spatiotemporal adaptive convolutional attention model enable the model to capture and utilize spatiotemporal contextual information spanning a longer time window from continuous visual input more effectively, thereby making more accurate decisions in navigation tasks that require complex sequence understanding and long-range memory.

[0043] 2. The asynchronous metavariable strategy algorithm can learn a general navigation strategy, while variational inference enhances the strategy's ability to handle uncertainty.

[0044] 3. Through the high-quality spatiotemporal feature representation provided by the spatiotemporal adaptive convolutional attention model, the highly generalizable strategy optimized by the asynchronous metavariable strategy algorithm, and the intelligent guidance provided by hierarchical semantic information and context vectors, the robot can achieve higher navigation task success rate and better path efficiency in various complex scenarios. Attached Figure Description

[0045] Figure 1 This is a flowchart of a visual navigation method for mobile robots based on spatiotemporal attention and asynchronous metavariable strategies.

[0046] Figure 2 This is a schematic diagram of the structure of a mobile robot in a visual navigation method for mobile robots based on spatiotemporal attention and asynchronous metavariable strategy.

[0047] Figure 3 This is a flowchart illustrating the spatiotemporal adaptive convolutional attention model in a mobile robot visual navigation method based on spatiotemporal attention and asynchronous metavariable strategy.

[0048] Figure 4 This is a flowchart illustrating the asynchronous metavariable strategy algorithm in a mobile robot visual navigation method based on spatiotemporal attention and asynchronous metavariable strategy.

[0049] Figure 5This is a schematic diagram illustrating the execution process of the navigation strategy in a mobile robot visual navigation method based on spatiotemporal attention and asynchronous metavariable strategy.

[0050] The diagram shows: 1. Wheels; 2. Robot shell; 3. Mounting frame; 4. Onboard computing unit; 5. Kinect camera. Detailed Implementation

[0051] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0052] The core idea of ​​this invention is to enhance the robot's autonomous visual navigation capability in unknown environments through efficient spatiotemporal feature extraction, intelligent navigation strategy learning, and deep understanding of complex semantics and environment.

[0053] Please see Figures 1-5 In this embodiment of the invention, a visual navigation method for mobile robots based on spatiotemporal attention and asynchronous metavariable strategy is described, and its steps are as follows:

[0054] S100: Defines the state space and action space of the visual navigation task, acquires visual input data from the environment, and performs preprocessing.

[0055] S200: Introduces hierarchical semantic information modeling and context vectors to enhance the semantic understanding ability and decision-making efficiency of intelligent agents in navigation tasks;

[0056] S300: Employs a spatiotemporal adaptive convolutional attention model, which integrates causal dilated convolution and temporally adaptive multidimensional attention mechanism to dynamically capture the long-term spatiotemporal dependency features of the environment.

[0057] S400: Construct an asynchronous metavariable policy algorithm, combining asynchronous parallelized policy gradient update with metavariable inference method to model the uncertainty of navigation policy parameters;

[0058] S500: Design a multi-objective reward function and train the navigation strategy;

[0059] S600: Transfer the trained navigation strategy model to the actual device.

[0060] Specifically, S200 involves hierarchical semantic modeling, which enables the robot to identify semantic and spatial relationships between objects. By utilizing parent-child object relationships, the robot is guided to prioritize exploring potential target areas, thereby improving navigation efficiency. Context vectors combine the visibility, location, and semantic similarity of objects with target objects, helping the robot make more accurate navigation decisions based on this information. By integrating these two, the robot can more intelligently select target areas and reduce ineffective exploration.

[0061] The robot includes wheels 1, a robot shell 2, a mounting frame 3, an onboard computing unit 4, and a Kinect camera 5. The robot shell 2 is equipped with wheels 1 at its bottom, and the robot shell 2 is fixedly equipped with the mounting frame 3 at its top. The mounting frame 3 is fixedly equipped with the onboard computing unit 4 at its top, and the mounting frame 3 is fixedly equipped with a mounting frame 3 on one side.

[0062] S100 requires a formal definition of the robot navigation task, which is the basis for the effective operation of subsequent reinforcement learning algorithms.

[0063] Specifically, S100 includes:

[0064] S110: Define the state space. The robot uses the Kinect camera 5 to capture the current single-frame RGB image and the corresponding depth image to capture dynamic changes in the environment. These raw image data may undergo preliminary processing by the model described in S200 before being input into the policy network. In order for the robot to understand its navigation target, the target described in natural language will be converted into a fixed-dimensional numerical vector through a pre-trained word vector model. These two parts of information will be concatenated into a unified state vector, which will be used as the input to the subsequent policy network.

[0065] S120: Define the motion space and design a set of motion instructions that the robot can execute. This motion space is usually set as a discrete set, namely, moving forward in a straight line for 0.25 meters, rotating 30 degrees to the left, rotating 30 degrees to the right, tilting the camera up 30 degrees, tilting the camera down 30 degrees, and an action called Done that indicates that the robot has found the target.

[0066] The goal of S200 is to provide the robot with high-level semantic information that goes beyond low-level visual features to assist in navigation decisions.

[0067] Specifically, S200 includes:

[0068] S210: A general indoor scene knowledge graph is constructed. When the robot receives a navigation target, it can query the knowledge graph to find its possible parent objects. During navigation, if the target is not visible, but its parent object is visible or its possible location is known, the strategy can be guided to prioritize exploring the area where the parent object is located.

[0069] S220: At each time step of robot navigation, the system extracts a series of contextual information from the current visual observation, for each object detected in the scene. j A context vector C can be constructed. j Context vector C j =[b,x c ,y c bbox, CS j ], where: b indicates whether the target object is visible in the field of view, (x c ,y c ) is an object o j Normalized center coordinates in the image; bbox is the object o j The normalized area of ​​the bounding box in the image; CS j It is an object o j The semantic similarity between the target and the current navigation target is obtained by calculating the cosine similarity between the GloVe word embedding vectors: Where g(oj) and g(t) represent object o, respectively. j The GloVe word embedding vector of the target object t.

[0070] In the specific implementation of S300, such as Figure 3 As shown, the goal is to build a neural network model that can efficiently extract discriminative spatiotemporal features from continuous visual input;

[0071] Specifically, S300 includes:

[0072] S310: Applying causal dilated convolution to process visual feature sequences, including:

[0073] S311: The input to the model is a sequence of visual features extracted by CNN. Where N is the batch size. C feat Where T is the number of input feature channels and T is the sequence length;

[0074] S312: Design a residual block structure containing two dilated convolutional layers. The stride of the first dilated convolutional layer is typically set to 1 to maintain the sequence length. After convolution, a causal pruning operation is performed to ensure the causality of the convolution. This is usually followed by a LeakyReLU activation function and a Dropout layer to prevent overfitting. The second dilated convolutional layer has the same structure. To promote gradient propagation and information flow, a residual connection is introduced to add the input and output of this module, and then it is again activated by the LeakyReLU activation function. The output of this temporal convolutional module... It is then passed to the attention mechanism described in S320;

[0075] S320: The spatiotemporal attention mechanism is applied to optimize spatiotemporal features. The specific implementation method is as follows:

[0076] S321: Spatiotemporal attention modules typically consist of two parts: temporal attention and channel attention. For temporal attention, input features can be... The dimensions are first rearranged from (N,C,T) to (T,N,C), and then a multi-head self-attention mechanism is applied to learn the dependencies and importance weights between different time steps in the sequence, resulting in temporally attention-weighted features. For channel attention, Global average pooling is performed on the features along the time dimension to obtain the descriptor for each channel. This descriptor is then passed through a multilayer perceptron with one or two fully connected layers, and finally, an attention weight for each channel is generated using a sigmoid activation function. These weights are then element-wise multiplied back to the previous layer. Above, we obtain the channel attention-weighted features. The final output of the spatiotemporal attention module It is a feature after processing with time attention. Features after further channel attention processing Perform element-wise addition, and then pass the sum through a LeakyReLU activation function to obtain the result:

[0077] S322: At the end of each attention temporal module, the output of the temporal convolutional module of S310 is usually... Output of the spatiotemporal attention module of S320 The residues are then added together again via residual connections, and then passed through an activation function to form the final output of this attention-based temporal module. This output will serve as the input to the next attention temporal module. When it is the last module, its output is the final spatiotemporal feature representation of the entire spatiotemporal adaptive convolutional attention model, which will be used by the subsequent policy learning module.

[0078] In the specific implementation of S400, such as Figure 4 As shown, the goal is to build a reinforcement learning algorithm that can efficiently learn navigation strategies and has good generalization ability.

[0079] Specifically, S400 includes:

[0080] S410: Set up a global network containing the parameters of the policy network and value network, start multiple parallel training processes, each process has a local copy of the global network parameters, and interacts with the environment in an independent environment instance to collect empirical data (s t ,at ,r t ,s t+1 Each process calculates its policy gradient and value loss gradient locally, and then asynchronously sends the calculated gradients to the central parameter server to update the global network parameters. It then pulls the latest parameters from the global network to its local copy and continues the next round of interaction and learning.

[0081] S420: Perform inner-layer optimization. For each sampled task k, one or more processes will use the current global meta-policy parameter θ as a starting point; perform minimal interactions and collect data on this task. Calculate the task-specific self-supervised interaction loss L int Then, the policy parameters are updated using gradient descent to obtain task-specific parameters. After performing outer-layer optimization and obtaining task-specific parameters θ'k through inner-layer optimization, the navigation performance is evaluated on task k using these adapted parameters, and a meta-target loss, i.e., the navigation loss, is calculated. in The data collected on this task is used to evaluate the meta-performance; the goal of the outer optimization is to minimize the expected navigation loss on all sampled tasks. The meta-gradient is calculated with respect to the original meta-parameters θ, i.e. These meta-gradients are then used to update the global meta-policy parameters. In the asynchronous framework, different processes can process the inner optimization and outer loss calculation of different tasks in parallel and asynchronously aggregate the meta-gradients to the global parameter server.

[0082] S430: To enhance the exploratory and robust nature of the policy, the policy network is designed as a probabilistic model, introducing a latent variable z. The policy π(a|s,z) depends not only on the state s but also on the latent variable z sampled from a prior distribution p(z). Since directly optimizing the marginal log-likelihood log∫p(a|s,z)p(z)dz is difficult, a variational approximation posterior distribution q(z|s,a) is introduced, and the lower bound of evidence E is optimized. q(z|s) [logp(a|s,z)]-D KL (q(z|s)||p(z)), where q(z|s) is typically parameterized by an encoder network that outputs the mean μ of a Gaussian distribution. z (s) and standard deviation σ z (s); to make the sampling process differentiable, the reparameterization technique is used when sampling z from q(z|s): z = μ z (s)+σ z (s), where The final loss function will include the policy learning objective and the KL divergence regularization term D. KL(q(z|s)||p(z)), where the hyperparameter β is used to balance these two terms:

[0083] In the specific implementation of S500, the design of the reward function is crucial for guiding the behavior of the reinforcement learning agent, and it is defined as follows:

[0084]

[0085] Among them, R p It is a partial reward when the parent object is visible, calculated using the formula R. p =R t ×Pr(t|p)×k, where R t =5, k=0.1, R t This is the reward when the target object is visible at the end of the task. If both the parent and target objects are visible at the end of the task, the total reward R is obtained. t +R p Otherwise, a small negative reward of -0.01 is given to encourage exploration;

[0086] When implementing S600, such as Figure 5 As shown, the goal is to deploy navigation models trained and validated in a simulation environment onto real physical robots and enable them to work effectively.

[0087] The S600 specifically includes:

[0088] S610: Equips mobile robots with an RGB-D camera, Kinect, to acquire color images and depth information of the environment;

[0089] S620: Install and configure ROS on the robot's computer, create a ROS node to receive and process image data from the RGB-D camera, and create another ROS node to run the trained navigation strategy model; this node will subscribe to the processed sensor data and target information, and issue motion commands to the robot to control the movement of the robot's wheels;

[0090] S630: The navigation strategy model weights obtained from simulation training are loaded into the neural network deployed in the ROS node. The robot collects environmental visual data in real time through the RGB-D camera. After preprocessing, the data is input into the spatiotemporal adaptive convolutional attention model to extract the current spatiotemporal feature representation. These features, together with the target semantic description of the current navigation task, are provided as input to the policy network of the asynchronous metavariable policy algorithm. The policy network outputs an action decision based on the current state and the target. The decided action is converted into a command that can be executed by the robot's underlying controller to drive the robot to move until the navigation task is completed.

[0091] Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A visual navigation method for mobile robots based on spatiotemporal attention and asynchronous metavariable strategy, characterized in that: The method and steps are as follows: S100: Defines the state space and action space of the visual navigation task, acquires visual input data from the environment, and performs preprocessing. S200: Introduces hierarchical semantic information modeling and context vectors to enhance the semantic understanding ability and decision-making efficiency of intelligent agents in navigation tasks; S300: Employs a spatiotemporal adaptive convolutional attention model, which integrates causal dilated convolution and temporally adaptive multidimensional attention mechanism to dynamically capture the long-term spatiotemporal dependency features of the environment. S400: Construct an asynchronous metavariable policy algorithm, combining asynchronous parallelized policy gradient update with metavariable inference method to model the uncertainty of navigation policy parameters; S500: Design a multi-objective reward function and train the navigation strategy; S600: Transfer the trained navigation strategy model to the actual device.

2. The mobile robot visual navigation method based on spatiotemporal attention and asynchronous metavariable strategy according to claim 1, characterized in that: The state space includes the word embeddings of the current RGB image and the target object, and the action space includes a discrete set of navigation actions.

3. The mobile robot visual navigation method based on spatiotemporal attention and asynchronous metavariable strategy according to claim 1, characterized in that: Specifically, S500 involves training an agent in a simulation environment using the spatiotemporal adaptive convolutional attention model and asynchronous metavariable policy algorithm, optimizing policy parameters through a multi-objective reward function, and ultimately achieving efficient autonomous navigation in complex environments.

4. The mobile robot visual navigation method based on spatiotemporal attention and asynchronous metavariable strategy according to claim 1, characterized in that: Specifically, S200 involves hierarchical semantic modeling, which enables the robot to identify semantic and spatial relationships between objects. By utilizing parent-child object relationships, the robot is guided to prioritize exploring potential target areas, thereby improving navigation efficiency. Context vectors combine the visibility, location, and semantic similarity of objects with target objects, helping the robot make more accurate navigation decisions based on this information. By integrating these two, the robot can more intelligently select target areas and reduce ineffective exploration.

5. The mobile robot visual navigation method based on spatiotemporal attention and asynchronous metavariable strategy according to claim 1, characterized in that: Specifically, S100 includes: S110: Define the state space. The robot uses the Kinect camera 5 to capture the current single-frame RGB image and the corresponding depth image to capture dynamic changes in the environment. These raw image data may undergo preliminary processing by the model described in S200 before being input into the policy network. In order for the robot to understand its navigation target, the target described in natural language will be converted into a fixed-dimensional numerical vector through a pre-trained word vector model. These two parts of information will be concatenated into a unified state vector, which will be used as the input to the subsequent policy network. S120: Define the motion space and design a set of motion instructions that the robot can execute. This motion space is usually set as a discrete set, namely, moving forward in a straight line for 0.25 meters, rotating 30 degrees to the left, rotating 30 degrees to the right, tilting the camera up 30 degrees, tilting the camera down 30 degrees, and an action called Done that indicates that the robot has found the target.

6. The mobile robot visual navigation method based on spatiotemporal attention and asynchronous metavariable strategy according to claim 1, characterized in that: Specifically, S200 includes: S210: A general indoor scene knowledge graph is constructed. When the robot receives a navigation target, it can query the knowledge graph to find its possible parent objects. During navigation, if the target is not visible, but its parent object is visible or its possible location is known, the strategy can be guided to prioritize exploring the area where the parent object is located. S220: At each time step of robot navigation, the system extracts a series of contextual information from the current visual observation, for each object detected in the scene. j A context vector C can be constructed. j Context vector C j =[b,x c ,y c bbox, CS j ], where: b indicates whether the target object is visible in the field of view, (x c ,y c ) is an object o j Normalized center coordinates in the image; bbox is the object o j The normalized area of ​​the bounding box in the image; CS j It is an object o j The semantic similarity between the target and the current navigation target is obtained by calculating the cosine similarity between the GloVe word embedding vectors: Among them, g(o j g(t) and g(t) represent the objects o j The GloVe word embedding vector of the target object t.

7. The mobile robot visual navigation method based on spatiotemporal attention and asynchronous metavariable strategy according to claim 1, characterized in that: Specifically, S300 includes: S310: Applying causal dilated convolution to process visual feature sequences, including: S311: The input to the model is a sequence of visual features extracted by CNN. Where N is the batch size, C feat Where T is the number of input feature channels and T is the sequence length; S312: Design a residual block structure containing two dilated convolutional layers. The stride of the first dilated convolutional layer is typically set to 1 to maintain the sequence length. After convolution, a causal pruning operation is performed to ensure the causality of the convolution. This is usually followed by a LeakyReLU activation function and a Dropout layer to prevent overfitting. The second dilated convolutional layer has the same structure. To promote gradient propagation and information flow, a residual connection is introduced to add the input and output of this module, and then it is again activated by the LeakyReLU activation function. The output of this temporal convolutional module... It is then passed to the attention mechanism described in S320; S320: The spatiotemporal attention mechanism is applied to optimize spatiotemporal features. The specific implementation method is as follows: S321: Spatiotemporal attention modules typically consist of two parts: temporal attention and channel attention. For temporal attention, input features can be... The dimensions are first rearranged from (N,C,T) to (T,N,C), and then a multi-head self-attention mechanism is applied to learn the dependencies and importance weights between different time steps in the sequence, resulting in temporally attention-weighted features. For channel attention, Global average pooling is performed on the features along the time dimension to obtain the descriptor for each channel. This descriptor is then passed through a multilayer perceptron with one or two fully connected layers, and finally, an attention weight for each channel is generated using a sigmoid activation function. These weights are then element-wise multiplied back to the previous layer. Above, we obtain the channel attention-weighted features. The final output of the spatiotemporal attention module It is a feature after processing with time attention. Features after further channel attention processing Perform element-wise addition, and then pass the sum through a LeakyReLU activation function to obtain the result: S322: At the end of each attention temporal module, the output of the temporal convolutional module of S310 is usually... Output of the S320's spatiotemporal attention module The residues are then added together again via residual connections, and then passed through an activation function to form the final output of this attention-based temporal module. This output will serve as the input to the next attention temporal module. When it is the last module, its output is the final spatiotemporal feature representation of the entire spatiotemporal adaptive convolutional attention model, which will be used by the subsequent policy learning module.

8. The mobile robot visual navigation method based on spatiotemporal attention and asynchronous metavariable strategy according to claim 1, characterized in that: Specifically, S400 includes: S410: Set up a global network containing the parameters of the policy network and value network, start multiple parallel training processes, each process has a local copy of the global network parameters, and interacts with the environment in an independent environment instance to collect empirical data (s t ,a t ,r t ,s t+1 Each process calculates its policy gradient and value loss gradient locally, and then asynchronously sends the calculated gradients to the central parameter server to update the global network parameters. It then pulls the latest parameters from the global network to its local copy and continues the next round of interaction and learning. S420: Perform inner-layer optimization. For each sampled task k, one or more processes will use the current global meta-policy parameter θ as a starting point; perform minimal interactions and collect data on this task. Calculate the task-specific self-supervised interaction loss L int Then, the policy parameters are updated using gradient descent to obtain task-specific parameters. After performing outer-layer optimization and obtaining task-specific parameters θ'k through inner-layer optimization, the navigation performance is evaluated on task k using these adapted parameters, and a meta-target loss, i.e., the navigation loss, is calculated. in The data collected on this task is used to evaluate the meta-performance; the goal of the outer optimization is to minimize the expected navigation loss on all sampled tasks. The meta-gradient is calculated with respect to the original meta-parameters θ, i.e. These meta-gradients are then used to update the global meta-policy parameters. In the asynchronous framework, different processes can process the inner optimization and outer loss calculation of different tasks in parallel and asynchronously aggregate the meta-gradients to the global parameter server. S430: To enhance the exploratory and robust nature of the policy, the policy network is designed as a probabilistic model, introducing a latent variable z. The policy π(a|s,z) depends not only on the state s but also on the latent variable z sampled from a prior distribution p(z). Since directly optimizing the marginal log-likelihood log∫p(a|s,z)p(z)dz is difficult, a variational approximation posterior distribution q(z|s,a) is introduced, and the lower bound of evidence E is optimized. q(z|s) [logp)a|s,z)]-D KL (q(z|s)||p(z)), where q(z|s) is typically parameterized by an encoder network that outputs the mean μ of a Gaussian distribution. z (s) and standard deviation σ z (s); to make the sampling process differentiable, the reparameterization technique is used when sampling z from q(z|s): z = μ z (s)+σ z (s), where The final loss function will include the policy learning objective and the KL divergence regularization term D. KL (q(z|s)||p(z)), where the hyperparameter β is used to balance these two terms:

9. The mobile robot visual navigation method based on spatiotemporal attention and asynchronous metavariable strategy according to claim 1, characterized in that: The design of the reward function in S500 is crucial for guiding the behavior of the reinforcement learning agent, and it is defined as follows: Among them, R p It is a partial reward when the parent object is visible, calculated using the formula R. p =R t ×Pr(t|p)×k, where R t =5, k=0.1, R t This is the reward when the target object is visible at the end of the task. If both the parent and target objects are visible at the end of the task, the total reward R is obtained. t +R p Otherwise, a small negative reward of -0.01 is given to encourage exploration.

10. The mobile robot visual navigation method based on spatiotemporal attention and asynchronous metavariable strategy according to claim 1, characterized in that: Specifically, S600 includes: S610: Equips mobile robots with an RGB-D camera, Kinect, to acquire color images and depth information of the environment; S620: Install and configure ROS on the robot's computer, create a ROS node to receive and process image data from the RGB-D camera, and create another ROS node to run the trained navigation strategy model; this node will subscribe to the processed sensor data and target information, and issue motion commands to the robot to control the movement of the robot's wheels; S630: The navigation strategy model weights obtained from simulation training are loaded into the neural network deployed in the ROS node. The robot collects environmental visual data in real time through the RGB-D camera. After preprocessing, the data is input into the spatiotemporal adaptive convolutional attention model to extract the current spatiotemporal feature representation. These features, together with the target semantic description of the current navigation task, are provided as input to the policy network of the asynchronous metavariable policy algorithm. The policy network outputs an action decision based on the current state and the target. The decided action is converted into a command that can be executed by the robot's underlying controller to drive the robot to move until the navigation task is completed.

Images