Multi-modal robot navigation method and system based on deep reinforcement learning

Abstract

The invention discloses a multi-modal robot navigation method and system based on deep reinforcement learning, and the method comprises the steps: constructing a deep reinforcement learning model, designing a reward function, and calculating a reward value of a robot after each execution of an action for model training; taking the image, the point cloud data and the sub-target point position collected at each moment as state data, inputting the state data into a deep reinforcement learning model, and outputting the linear velocity, the angular velocity and the state value at each moment by the deep reinforcement learning model; according to interaction data of the robot and the environment, a near-end strategy optimization algorithm is adopted to optimize the deep reinforcement learning model, and finally an optimal strategy model is obtained; and the optimal strategy model outputs an optimal linear velocity and angular velocity combination by using the multi-modal data and the sub-target point position information, and is used for controlling the robot to execute a navigation task.

Description

Technical Field

[0001] The present invention relates to the field of robot navigation technology, and in particular to a multimodal robot navigation method and system based on deep reinforcement learning. Background Art

[0002] Robot navigation technology is an important branch of robotics research, which aims to enable robots to efficiently navigate and avoid obstacles in complex environments, and complete tasks such as logistics transportation and emergency rescue. Reinforcement learning, as a learning method, enables robots to continuously optimize navigation strategies through interaction with the environment, thereby achieving better performance. Robots with multimodal perception capabilities can overcome the sensitivity of a single sensor to environmental changes and enhance their perception and adaptability to the environment. Multimodal navigation methods based on deep reinforcement learning are expected to significantly improve the navigation capabilities of robots.

[0003] The navigation methods of autonomous mobile robots are mainly divided into two categories: rule-based and learning-based. Rule-based methods include the dynamic window algorithm (DWA) and the artificial potential field method (APF), which are widely used in simple navigation scenarios due to their stability. However, in complex environments, these methods often face local difficulties and rely on parameter adjustment of environmental characteristics, making it difficult for robots to achieve continuous generalization in different environments. In contrast, learning-based methods usually adopt an end-to-end navigation mode, which directly maps perception information into speed commands by decoupling perception and control. This method has greater flexibility and adaptability, and can continuously learn and improve, so that the robot performs well in different environments. For example, Cimurs et al. proposed an autonomous navigation system based on reinforcement learning, which uses lidar for target-driven exploration of unknown environments.

[0004] Single-modal sensors are susceptible to environmental changes. For example, image quality may deteriorate in dim light conditions. For example, Mezghani et al. used only image information to complete indoor navigation tasks, but faced certain difficulties when generalizing to outdoor environments. In contrast, robots based on multimodal perception can better adapt to environmental changes. Multimodal perception complements each other and can make up for the shortcomings of a single modality. For example, Zu et al. proposed a multimodal navigation framework that uses reinforcement learning to process the received information to complete multiple tasks. This method is not only stable, but can also be effectively generalized to unseen environments.

[0005] Due to the advantages of deep reinforcement learning and multimodal perception, multimodal robot navigation based on deep reinforcement learning has become a very promising solution. However, designing a reliable autonomous navigation system still faces challenges, especially in complex real-world environments where robots need to navigate from the starting point to the end point and avoid numerous static and dynamic obstacles.

[0006] When autonomous mobile robots perform tasks in scenes containing static and dynamic obstacles, they often face the problem of difficulty in extracting environmental structural features, which places higher demands on navigation performance. Current navigation methods are usually suitable for simpler scenes, but when migrating to more complex environments, performance often deteriorates, making it difficult for robots to successfully complete navigation tasks.

[0007] At present, the commonly used navigation methods for mobile robots are traditional methods, such as the dynamic window algorithm (DWA) and the artificial potential field method (APF). These methods are stable and safe, but they often face local dilemmas and rely on parameter adjustment of environmental characteristics. The navigation method based on deep reinforcement learning decouples perception and control and directly maps the perception information into speed commands. This method overcomes the characteristics of traditional methods that are easy to get into trouble and require frequent parameter adjustment. On the other hand, single-modal perception is usually susceptible to environmental changes and is difficult to generalize to a wide range of scenarios. Summary of the invention

[0008] In order to solve the shortcomings of the prior art, the present invention provides a multimodal robot navigation method and system based on deep reinforcement learning. First, a deep reinforcement learning navigation framework is constructed, which can output speed data end-to-end by extracting features of images and single-line laser radars. Then, a set of reward functions is designed to guide the robot to avoid collisions and reach the target point smoothly. Based on this, the robot can effectively complete the navigation task.

[0009] On the one hand, a multimodal robot navigation method based on deep reinforcement learning is provided, including:

[0010] Acquire multimodal data collected by the robot; the multimodal data includes: images and point cloud data in front of the robot;

[0011] Construct a deep reinforcement learning model and design a reward function to calculate the reward value of the robot after each action for model training; the reward function is the sum of the reward sub-function for the robot approaching the target point, the penalty sub-function for the robot approaching and colliding with obstacles, the penalty sub-function for the robot shaking too much, and the penalty sub-function for the robot driving on the left;

[0012] The images, point cloud data and sub-target point positions collected at each moment are used as state data and input into the deep reinforcement learning model. The deep reinforcement learning model outputs the linear velocity, angular velocity and state value at each moment; based on the interaction data between the robot and the environment, the proximal policy optimization algorithm is used to optimize the deep reinforcement learning model, and finally an optimal policy model is obtained; the optimal policy model uses multimodal data and sub-target point position information to output the optimal combination of linear velocity and angular velocity, which is used to control the robot to perform navigation tasks.

[0013] On the other hand, a multimodal robot navigation system based on deep reinforcement learning is provided, including:

[0014] An acquisition module is configured to: acquire multimodal data collected by the robot; the multimodal data includes: images and point cloud data in front of the robot;

[0015] A construction module is configured to: construct a deep reinforcement learning model, design a reward function to calculate the reward value of the robot after each action is performed for model training; the reward function is the sum of a reward sub-function for the robot approaching a target point, a penalty sub-function for the robot approaching and colliding with an obstacle, a penalty sub-function for the robot shaking too much, and a penalty sub-function for the robot driving on the left;

[0016] The optimization module is configured as follows: the images, point cloud data and sub-target point positions collected at each moment are input as state data into the deep reinforcement learning model, and the deep reinforcement learning model outputs the linear velocity, angular velocity and state value at each moment; according to the interaction data between the robot and the environment, the proximal strategy optimization algorithm is used to optimize the deep reinforcement learning model, and finally an optimal strategy model is obtained; the optimal strategy model uses multimodal data and sub-target point position information to output the optimal combination of linear velocity and angular velocity for controlling the robot to perform navigation tasks.

[0017] On the other hand, there is also provided an electronic device, comprising:

[0018] a memory for non-transitory storage of computer-readable instructions; and

[0019] a processor for executing the computer readable instructions,

[0020] When the computer-readable instructions are executed by the processor, the method described in the first aspect is executed.

[0021] On the other hand, a storage medium is provided, which non-temporarily stores computer-readable instructions, wherein when the non-temporary computer-readable instructions are executed by a computer, the method described in the first aspect is executed.

[0022] On the other hand, a computer program product is provided, comprising a computer program, wherein the computer program is used to implement the method described in the first aspect when running on one or more processors.

[0023] The above technical solution has the following advantages or beneficial effects:

[0024] A deep reinforcement learning navigation framework is constructed, which can output velocity data end-to-end by extracting features from images and single-line lidar. Then, a set of reward functions is designed to guide the robot to avoid collisions and reach the target point smoothly. Based on this, the robot can effectively complete the navigation task. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0026] Figure 1 This is a deep reinforcement learning navigation framework for implementing Example 1. DETAILED DESCRIPTION

[0027] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0028] Embodiment 1: This embodiment provides a multimodal robot navigation method based on deep reinforcement learning, including:

[0029] S101: Acquire multimodal data collected by the robot; the multimodal data includes: images and point cloud data in front of the robot;

[0030] S102: construct a deep reinforcement learning model and design a reward function to calculate the reward value of the robot after each action for model training; the reward function is the sum of a reward sub-function for the robot approaching the target point, a penalty sub-function for the robot approaching and colliding with an obstacle, a penalty sub-function for the robot shaking too much, and a penalty sub-function for the robot driving on the left;

[0031] S103: The images, point cloud data and sub-target point positions collected at each moment are used as state data and input into the deep reinforcement learning model. The deep reinforcement learning model outputs the linear velocity, angular velocity and state value at each moment. According to the interaction data between the robot and the environment, the proximal policy optimization algorithm (PPO) is used to optimize the deep reinforcement learning model, and finally an optimal policy model is obtained. The optimal policy model uses multimodal data and sub-target point position information to output the optimal combination of linear velocity and angular velocity for controlling the robot to perform navigation tasks.

[0032] The interaction data includes: state data, state value, action and reward value; the action includes: linear velocity and angular velocity; the multimodal data includes: images and point cloud data collected at each moment.

[0033] The PPO algorithm comes from the paper Proximal Policy Optimization Algorithms.

[0034] Further, the S101: obtains multimodal data collected by the robot; the multimodal data includes: images and point cloud data in front of the robot, wherein the image is an RGB image in front of the robot taken by the robot's camera during the robot's forward movement; wherein the point cloud data is point cloud data in front of the robot collected by the robot's laser radar during the robot's forward movement.

[0035] Furthermore, the S102: constructing a deep reinforcement learning model, the deep reinforcement learning model includes:

[0036] A feature extraction module and a strategy generation module connected in sequence;

[0037] The feature extraction module includes a first branch and a second branch arranged in parallel;

[0038] The first branch includes: an image preprocessing unit, a Resnet34 network and a Flatten layer connected in sequence; the first branch extracts features of the input RGB image to obtain image features;

[0039] The image preprocessing unit is used to crop the image into sub-images of a set size; for example, cropping the image into a dimension of 224×224;

[0040] The Resnet34 network is used to extract features from each sub-image;

[0041] The Flatten layer is used to flatten the features extracted by the Resnet34 network; the flattening operation is to convert the multi-dimensional input into one dimension;

[0042] The second branch includes: a sampling layer, a data normalization layer and a first fully connected layer connected in sequence; the second branch extracts features from the point cloud data to obtain point cloud data features;

[0043] The sampling layer is used to perform sampling processing on the point cloud data;

[0044] The data normalization layer is used to normalize the sampling results;

[0045] The first fully connected layer is used to extract features from the first half of the point cloud data to obtain point cloud data features; the first half refers to the data of 0° to 180° scanned by the laser radar;

[0046] The strategy generation module comprises:

[0047] A second fully connected layer, wherein the input end of the second fully connected layer is respectively connected to the output end of the Flatten layer and the first fully connected layer; the input end of the second fully connected layer is also used to input the position of the sub-target point; the sub-target point is a plurality of position points set between the starting point and the final target point, and a sub-target point is set at each set distance;

[0048] The output end of the second fully connected layer is connected to the input end of the third fully connected layer, and the output end of the third fully connected layer is connected to the input end of the fourth fully connected layer; the output end of the fourth fully connected layer is connected to the input end of the first activation function layer, and the output end of the first activation function layer is used to output the linear velocity of the robot;

[0049] The output end of the fourth fully connected layer is connected to the input end of the second activation function layer, and the output end of the second activation function layer outputs the angular velocity of the robot;

[0050] The output end of the second fully connected layer is also connected to the input end of the fifth fully connected layer, the output end of the fifth fully connected layer is connected to the input end of the sixth fully connected layer, and the output end of the sixth fully connected layer outputs the state value.

[0051] It should be understood that the beneficial effects of the above-mentioned deep reinforcement learning model are: it can simultaneously extract multimodal data features and easily generalize to different environments; it is conducive to the conversion from simulation to real environment.

[0052] Furthermore, a reward function is designed to calculate the reward value of the robot after each action is performed for model training; the reward function is the sum of the reward sub-function of the robot approaching the target point, the penalty sub-function of the robot approaching and colliding with obstacles, the penalty sub-function of the robot shaking too much, and the penalty sub-function of the robot driving on the left, wherein the reward sub-function r of the robot approaching the target point is the sum of the reward sub-function r of the robot approaching the target point g ,include:

[0053] Reward the robot for approaching and reaching the target point:

[0054]

[0055] in, Indicates the distance between the robot and the sub-target point at the current moment, Indicates the distance between the robot and the sub-target point at the previous moment.

[0056] Furthermore, the penalty subfunction r for the robot approaching and colliding with obstacles c ,include:

[0057] Penalize the robot for approaching and colliding with obstacles:

[0058]

[0059] in, Indicates the distance between the robot and the obstacle.

[0060] Furthermore, the penalty sub-function r for excessive robot jitter w ,include:

[0061] Penalize the robot for excessive angular velocity:

[0062]

[0063] Where w represents the angular velocity of the robot.

[0064] Furthermore, the penalty subfunction r for the robot driving to the left l ,include:

[0065] In order to make the robot follow the right-hand driving rule, when the robot drives on the left side of the road, it will be punished:

[0066]

[0067] The goal of the present invention is to avoid collision in an environment containing static and dynamic obstacles and move toward a target point. In order to guide the agent to achieve this goal, the present invention designs the above-mentioned reward function.

[0068] This paper designs a deep reinforcement learning navigation framework that can output speed data end-to-end by extracting features of image and single-line laser radar multimodal data and sub-target point location information. Then, a set of reward functions is designed to guide the robot to avoid collisions and reach the target point smoothly.

[0069] like Figure 1 As shown in the figure, the deep reinforcement learning navigation framework is demonstrated. The present invention first extracts the features of the image and lidar data respectively, then fuses the extracted features with the sub-target point position information, and finally outputs the speed command.

[0070] The present invention first designs a deep reinforcement learning navigation framework, which can output speed data end-to-end by extracting features of image and single-line lidar multimodal data and sub-target point location information. At the same time, a set of reward functions is designed for deep reinforcement learning training to guide the robot to avoid collision and reach the target point smoothly.

[0071] The present invention designs an end-to-end navigation framework, which includes a feature extraction module and a strategy module. In the feature extraction module, the RGB image is first processed using a pre-trained ResNet34 to extract useful features. The extracted feature tensor is flattened for integration with subsequent tensors. At the same time, the LiDAR point cloud data is also included in the feature extraction module. Considering that robot navigation mainly focuses on forward movement, the present invention only uses the first half of the point cloud, which contains 512 feature points spanning 180 degrees, each point represents coordinate information, and is directly input into the fully connected network. The sub-target point provides an intermediate target position for the robot to track. The present invention sets a sub-target point every 5 meters. When the robot reaches a sub-target point, it will transition to the next sub-target point.

[0072] In the policy module, the extracted features are used to generate speed commands and evaluation values. These features are combined and processed by multiple fully connected layers, and the ReLU activation function is applied after each layer, except for the last layer. The final output of the policy module is linear velocity and angular velocity on the one hand, where the linear velocity range is [0,2] and the angular velocity range is [-1,1], so the last layer uses sigmoid and tanh activation functions for these outputs respectively. The linear velocity is doubled to meet the required speed range. On the other hand, it is an evaluation value, which is used to evaluate the state value of the current strategy. The present invention uses the PPO algorithm to train the network, and the network is updated according to the evaluation value and reward signal. The evaluation value guides the network to improve the strategy to maximize the cumulative return, while the reward signal provides immediate feedback on the effect of the action.

[0073] Embodiment 2

[0074] This embodiment provides a multimodal robot navigation system based on deep reinforcement learning, including:

[0075] An acquisition module is configured to: acquire multimodal data collected by the robot; the multimodal data includes: images and point cloud data in front of the robot;

[0076] A construction module is configured to: construct a deep reinforcement learning model, design a reward function to calculate the reward value of the robot after each action is performed for model training; the reward function is the sum of a reward sub-function for the robot approaching a target point, a penalty sub-function for the robot approaching and colliding with an obstacle, a penalty sub-function for the robot shaking too much, and a penalty sub-function for the robot driving on the left;

[0077] The optimization module is configured as follows: the images, point cloud data and sub-target point positions collected at each moment are input as state data into the deep reinforcement learning model, and the deep reinforcement learning model outputs the linear velocity, angular velocity and state value at each moment; according to the interaction data between the robot and the environment, the proximal strategy optimization algorithm is used to optimize the deep reinforcement learning model, and finally an optimal strategy model is obtained; the optimal strategy model uses multimodal data and sub-target point position information to output the optimal combination of linear velocity and angular velocity for controlling the robot to perform navigation tasks.

[0078] It should be noted that the acquisition module, construction module and optimization module described above correspond to steps S101 to S103 in Embodiment 1, and the examples and application scenarios implemented by the modules and the corresponding steps are the same, but are not limited to the contents disclosed in Embodiment 1. It should be noted that the modules described above as part of the system can be executed in a computer system such as a set of computer executable instructions.

[0079] The description of each embodiment in the above embodiments has different emphases. For parts not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0080] The proposed system can be implemented in other ways. For example, the system embodiment described above is only illustrative, and the division of the modules is only a logical function division. In actual implementation, there may be other division methods, such as multiple modules can be combined or integrated into another system, or some features can be ignored or not executed.

[0081] Embodiment 3

[0082] This embodiment also provides an electronic device, including: one or more processors, one or more memories, and one or more computer programs; wherein the processor is connected to the memory, and the one or more computer programs are stored in the memory. When the electronic device is running, the processor executes the one or more computer programs stored in the memory so that the electronic device executes the method described in the above embodiment one.

[0083] It should be understood that in this embodiment, the processor may be a central processing unit CPU, and the processor may also be other general-purpose processors, digital signal processors DSP, application-specific integrated circuits ASIC, off-the-shelf programmable gate arrays FPGA or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0084] The memory may include a read-only memory and a random access memory, and provide instructions and data to the processor. A portion of the memory may also include a non-volatile random access memory. For example, the memory may also store information about the device type.

[0085] In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in a processor or an instruction in the form of software.

[0086] The method in the first embodiment can be directly embodied as a hardware processor, or a combination of hardware and software modules in the processor. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware. To avoid repetition, it will not be described in detail here.

[0087] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with this embodiment can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0088] Embodiment 4

[0089] This embodiment further provides a computer-readable storage medium for storing computer instructions. When the computer instructions are executed by a processor, the method described in the first embodiment is completed.

[0090] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A multimodal robot navigation method based on deep reinforcement learning, characterized by: include: Obtain multimodal data collected by the robot; The multimodal data includes: images and point cloud data in front of the robot; Construct a deep reinforcement learning model and design a reward function to calculate the reward value of the robot after each action for model training; the reward function is the sum of the reward sub-function for the robot approaching the target point, the penalty sub-function for the robot approaching and colliding with obstacles, the penalty sub-function for the robot shaking too much, and the penalty sub-function for the robot driving on the left; The images, point cloud data and sub-target point positions collected at each moment are used as state data and input into the deep reinforcement learning model. The deep reinforcement learning model outputs the linear velocity, angular velocity and state value at each moment; based on the interaction data between the robot and the environment, the proximal policy optimization algorithm is used to optimize the deep reinforcement learning model, and finally an optimal policy model is obtained; the optimal policy model uses multimodal data and sub-target point position information to output the optimal combination of linear velocity and angular velocity, which is used to control the robot to perform navigation tasks.

2. The multimodal robot navigation method based on deep reinforcement learning as claimed in claim 1, characterized in that: Construct a deep reinforcement learning model, the deep reinforcement learning model comprising: A feature extraction module and a strategy generation module connected in sequence; The feature extraction module includes a first branch and a second branch arranged in parallel; The first branch includes: an image preprocessing unit, a Resnet34 network and a Flatten layer connected in sequence; the first branch extracts features of the input RGB image to obtain image features; The image preprocessing unit is used to crop the image into sub-images of a set size; the Resnet34 network is used to extract features from each sub-image; The Flatten layer is used to flatten the features extracted by the Resnet34 network; the flattening operation is to convert the multi-dimensional input into one dimension; The second branch includes: a sampling layer, a data normalization layer and a first fully connected layer connected in sequence; the second branch extracts features from the point cloud data to obtain point cloud data features; The sampling layer is used to perform sampling processing on the point cloud data; The data normalization layer is used to normalize the sampling results; The first fully connected layer is used to extract features from the first half of the point cloud data to obtain point cloud data features; the first half refers to the data of 0° to 180° scanned by the laser radar.

3. The multimodal robot navigation method based on deep reinforcement learning as claimed in claim 2, characterized in that: The strategy generation module comprises: A second fully connected layer, wherein the input end of the second fully connected layer is respectively connected to the output end of the Flatten layer and the first fully connected layer; the input end of the second fully connected layer is also used to input the position of the sub-target point; the sub-target point is a plurality of position points set between the starting point and the final target point, and a sub-target point is set at each set distance; The output end of the second fully connected layer is connected to the input end of the third fully connected layer, and the output end of the third fully connected layer is connected to the input end of the fourth fully connected layer; the output end of the fourth fully connected layer is connected to the input end of the first activation function layer, and the output end of the first activation function layer is used to output the linear velocity of the robot; The output end of the fourth fully connected layer is connected to the input end of the second activation function layer, and the output end of the second activation function layer outputs the angular velocity of the robot; The output end of the second fully connected layer is also connected to the input end of the fifth fully connected layer, the output end of the fifth fully connected layer is connected to the input end of the sixth fully connected layer, and the output end of the sixth fully connected layer outputs the state value.

4. The multimodal robot navigation method based on deep reinforcement learning as claimed in claim 1, characterized in that: A reward function is designed to calculate the reward value of the robot after each action for model training; the reward function is the sum of the reward sub-function for the robot approaching the target point, the penalty sub-function for the robot approaching and colliding with obstacles, the penalty sub-function for the robot shaking too much, and the penalty sub-function for the robot driving on the left. g ,include: Reward the robot for approaching and reaching the target point: in, Indicates the distance between the robot and the sub-target point at the current moment, Indicates the distance between the robot and the sub-target point at the previous moment.

5. The multimodal robot navigation method based on deep reinforcement learning as claimed in claim 4, characterized in that the robot The penalty subfunction r for approaching and colliding with obstacles c ,include: Penalize the robot for approaching and colliding with obstacles: in, Indicates the distance between the robot and the obstacle.

6. The multimodal robot navigation method based on deep reinforcement learning as claimed in claim 4, characterized in that: The penalty sub-function r for excessive robot jitter w ,include: Penalize the robot for excessive angular velocity: Where w represents the angular velocity of the robot; The penalty subfunction r for the robot driving on the left l ,include: In order to make the robot follow the right-hand driving rule, when the robot drives on the left side of the road, it will be punished:

7. A multimodal robot navigation system based on deep reinforcement learning, characterized by: include: An acquisition module is configured to: acquire multimodal data collected by the robot; The multimodal data includes: images and point cloud data in front of the robot; A construction module is configured to: construct a deep reinforcement learning model, design a reward function to calculate the reward value of the robot after each action is performed for model training; the reward function is the sum of a reward sub-function for the robot approaching a target point, a penalty sub-function for the robot approaching and colliding with an obstacle, a penalty sub-function for the robot shaking too much, and a penalty sub-function for the robot driving on the left; The optimization module is configured as follows: the images, point cloud data and sub-target point positions collected at each moment are input as state data into the deep reinforcement learning model, and the deep reinforcement learning model outputs the linear velocity, angular velocity and state value at each moment; according to the interaction data between the robot and the environment, the proximal strategy optimization algorithm is used to optimize the deep reinforcement learning model, and finally an optimal strategy model is obtained; the optimal strategy model uses multimodal data and sub-target point position information to output the optimal combination of linear velocity and angular velocity for controlling the robot to perform navigation tasks.

8. An electronic device, characterized in that: include: A memory for non-temporarily storing computer-readable instructions; and a processor for executing the computer-readable instructions, wherein when the computer-readable instructions are executed by the processor, the method according to any one of claims 1 to 6 is executed.

9. A storage medium, characterized in that: Non-transitory storage of computer-readable instructions, wherein when the non-transitory computer-readable instructions are executed by a computer, the method according to any one of claims 1 to 6 is performed.

10. A computer program product, characterized in that The invention comprises a computer program, which is used to implement the method according to any one of claims 1 to 6 when running on one or more processors.

Images