Robot target navigation method and device
By extracting the visual features and semantic embedding information of the robot in the preset time period, and biasing the actor's policy network weight, the problem of inaccurate navigation of the robot for the targets of the category is solved, and the navigation strategy transfer under zero sample conditions is realized Improved accuracy.
Patent Information
- Application Number
- CN202311280776.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-09-28
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2043-09-28
AI Technical Summary
The robot is inaccurate in navigation of unseen categories in real environments, and the existing technology is difficult to effectively solve this problem.
By extracting visual features and semantic embedding information from the visual observation data of the robot to be navigated within the preset time period, biasing the actor's policy network weights, aligning the visual features and semantic knowledge of the seen categories in the training stage, and using the unseen categories The semantic embedding of the process of handling the missing categories during the test phase, improving navigation accuracy.
The shift of the seen category navigation strategy to the unseen category under zero sample conditions is achieved, improving the accuracy of the robot's navigation of the unseen category targets.
Smart Images

Figure CN117516530B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of robot technology, and in particular to a robot target navigation method and device. Background Art
[0002] Goal navigation is an important skill for robots to perform real-world tasks, guiding them to reach their goals through visual observation. Learning effective navigation strategies is a complex problem that involves many areas of robotics, such as visual perception, scene understanding, and motion planning.
[0003] Current research on robot target navigation mainly focuses on target categories pre-defined in the training phase, called seen categories. However, in real environments, there may be target categories that are not included in the training phase, called unseen categories, which leads to inaccurate robot navigation for targets of unseen categories. Summary of the invention
[0004] The present invention provides a robot target navigation method and device, which are used to solve the defect of inaccurate robot navigation to targets of unseen categories in the prior art, and improve the accuracy of robot navigation to targets of unseen categories.
[0005] The present invention provides a robot target navigation method, comprising:
[0006] Extracting visual features of the robot to be navigated from visual observation data of the robot to be navigated at each moment within a preset time period before the current moment, and determining semantic embedding information of target categories received by the robot to be navigated at each moment within the preset time period, wherein the preset time period includes the current moment;
[0007] biasing the trained weights of the actor strategy network in the actor-critic algorithm according to the semantic embedding information corresponding to the current moment to obtain the final weights of the actor strategy network;
[0008] According to the visual features and semantic embedding information corresponding to each moment in the preset time period and the actions of the robot to be navigated at each moment in the preset time period except the current moment, the action of the robot to be navigated at the current moment is obtained based on the actor strategy network corresponding to the final weight;
[0009] The actor-critic algorithm is obtained by training by using the visual observation data and target categories of the robot samples at each moment in the preset time period as samples and the actions corresponding to the robot samples as labels.
[0010] According to a robot target navigation method provided by the present invention, the weights of the actor strategy network trained in the actor-critic algorithm are biased according to the semantic embedding information corresponding to the current moment to obtain the final weights of the actor strategy network, including:
[0011] Inputting the semantic embedding information corresponding to the current moment into the fully connected layer and the Tanh function in sequence to obtain the weight bias matrix of the actor strategy network;
[0012] The final weights of the actor strategy network are obtained according to the trained weights of the actor strategy network and the weight bias matrix.
[0013] According to a robot target navigation method provided by the present invention, the action of the robot to be navigated at the current moment is obtained based on the actor strategy network corresponding to the final weight according to the visual features and semantic embedding information corresponding to each moment in the preset time period and the action of the robot to be navigated at each moment in the preset time period except the current moment, including:
[0014] The visual features and semantic embedding information corresponding to each moment in the preset time period and the action corresponding to the previous moment of each moment are spliced to obtain the navigation information corresponding to each moment;
[0015] According to the navigation information corresponding to each moment in the preset time period, the action of the robot to be navigated at the current moment is obtained based on the actor strategy network corresponding to the final weight.
[0016] According to a robot target navigation method provided by the present invention, the action of the robot to be navigated at the current moment is obtained based on the navigation information corresponding to each moment in the preset time period and the actor strategy network corresponding to the final weight, including:
[0017] The navigation information corresponding to all moments within the preset time period is input into a time series convolutional neural network to obtain the action of the robot to be navigated at the current moment output by the time series convolutional neural network.
[0018] According to a robot target navigation method provided by the present invention, the visual features of the robot to be navigated are extracted from the visual observation data of the robot to be navigated at each moment within a preset time period before the current moment, including:
[0019] Based on the ResNet-50 model, visual features of the robot to be navigated are extracted from the visual observation data of the robot to be navigated at each moment within the preset time period.
[0020] According to a robot target navigation method provided by the present invention, the step of determining the semantic embedding information of the target category received by the robot to be navigated at each moment within the preset time period includes:
[0021] The semantic embedding information of the target category received by the robot to be navigated at each moment within the preset time period is determined based on the GloVe model.
[0022] The present invention also provides a robot target navigation device, comprising:
[0023] an extraction module, configured to extract visual features of the robot to be navigated from visual observation data of the robot to be navigated at each moment within a preset time period before a current moment, and determine semantic embedding information of target categories received by the robot to be navigated at each moment within the preset time period, wherein the preset time period includes the current moment;
[0024] A bias module, configured to bias the weights of the actor strategy network trained in the actor-critic algorithm according to the semantic embedding information corresponding to the current moment, so as to obtain the final weights of the actor strategy network;
[0025] A navigation module, configured to obtain the action of the robot to be navigated at the current moment based on the actor strategy network corresponding to the final weight according to the visual features and semantic embedding information corresponding to each moment within the preset time period and the action of the robot to be navigated at each moment within the preset time period except the current moment;
[0026] The actor-critic algorithm is obtained by training by using the visual observation data and target categories of the robot samples at each moment in the preset time period as samples and the actions corresponding to the robot samples as labels.
[0027] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the robot target navigation method described above is implemented.
[0028] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the robot target navigation method as described in any one of the above is implemented.
[0029] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the robot target navigation method as described above is implemented.
[0030] The robot target navigation method and device provided by the present invention align the visual features and semantic knowledge of seen categories into a shared space during the training phase, and can use the semantic embedding of unseen categories and the aligned shared space to process unseen categories during the testing phase. In addition, the weights of the actor strategy network after training are biased by using the semantic embedding information corresponding to the current moment, so as to achieve better utilization of the semantic embedding information and improve the accuracy of the robot's target navigation for unseen categories. BRIEF DESCRIPTION OF THE DRAWINGS
[0031] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0032] Figure 1 It is a flow chart of the robot target navigation method provided by the present invention;
[0033] Figure 2 It is a structural schematic diagram of a robot target navigation model in the robot target navigation method provided by the present invention;
[0034] Figure 3 It is a schematic diagram of the structure of the robot target navigation device provided by the present invention;
[0035] Figure 4 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION
[0036] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0037] Combine the following Figure 1 A robot target navigation method of the present invention is described, comprising:
[0038] Step 101, extracting visual features of the robot to be navigated from visual observation data of the robot to be navigated at each moment in a preset time period before the current moment, and determining semantic embedding information of target categories received by the robot to be navigated at each moment in the preset time period, wherein the preset time period includes the current moment;
[0039] The robot to be navigated is the robot that needs to perform target navigation. At each current moment t, the robot to be navigated receives the visual observation data o from its own perspective. t and target category l. The target category can be a seen category or an unseen category.
[0040] Visual observation data t It can be an RGB-D image captured by a camera of the robot to be navigated, such as an RGB-D image with a size of 224 × 224. The target category l can be the category of the target object, such as clothes, counters, and tables.
[0041] The preset time period includes multiple moments. Visual observation data and target categories of the robot to be navigated at each moment in the preset time period are obtained from the historical navigation path of the robot to be navigated. Feature extraction is performed on the visual observation data at each moment, and semantic embedding information of the target category is obtained, thereby obtaining visual features and semantic embedding information of the robot to be navigated at different moments. This embodiment does not limit the feature extraction method and the method for obtaining the semantic embedding information.
[0042] For example, when the preset time period includes 8 moments, the visual observation data and target categories of the robot to be navigated from moment t-7 to moment t are obtained. Feature extraction is performed on the visual observation data at each moment from moment t-7 to moment t, and the semantic embedding information of the target category at each moment from moment t-7 to moment t is obtained.
[0043] Step 102, biasing the trained weights of the actor strategy network in the actor-critic algorithm according to the semantic embedding information corresponding to the current moment, to obtain the final weights of the actor strategy network;
[0044] The semantic embedding information corresponding to the navigation robot at the current time t is s l . Used to bias the weights of the actor-strategy network in the Actor-Critic algorithm.
[0045] The weight of the actor strategy network is the weight of the actor strategy network after training the Actor-Critic algorithm. The semantic embedding information corresponding to the current time t is s l The weights of the actor strategy network are biased, and this embodiment is not limited to a specific method of biasing.
[0046] Step 103, according to the visual features and semantic embedding information corresponding to each moment in the preset time period and the actions of the robot to be navigated at each moment in the preset time period except the current moment, the action of the robot to be navigated at the current moment is obtained based on the actor strategy network corresponding to the final weight;
[0047] The actor-critic algorithm is obtained by training by using the visual observation data and target categories of the robot samples at each moment in the preset time period as samples and the actions corresponding to the robot samples as labels.
[0048] The visual features, semantic embedding information corresponding to each moment of the robot to be navigated within a preset time period and the actions of each moment except the current moment within the preset time period are used as the historical navigation information of the robot to be navigated.
[0049] Input the navigation information of the robot to be navigated into the weighted biased actor strategy network to obtain the action a at the current moment output by the actor strategy network t .
[0050] The output of the actor strategy network may include the probabilities corresponding to the actions "forward", "left", "right" and "stop", from which the action with the highest probability is selected as the action a at the current moment t .
[0051] Figure 2 This is a schematic diagram of the structure of the robot target navigation model in the robot target navigation method provided by the present invention. Figure 2 As shown in Figure 1, the robot target navigation model includes a semantic actor strategy module and a semantic trajectory module. The semantic actor strategy module is constructed using the Actor-Critic algorithm.
[0052] The semantic actor strategy module uses the semantic embedding information of different target categories to bias the weights in the actor strategy network. This module enhances the influence of semantic embedding information on navigation strategy and better transfers the navigation strategy learned on seen categories to unseen categories under zero-shot conditions.
[0053] The semantic trajectory module is used to store and extract global semantic information in spatial and temporal dimensions. The semantic trajectory module obtains the visual observation data and target category of the robot to be navigated at each moment in a preset time period from the historical navigation path of the robot to be navigated, as well as the action at the previous moment of each moment, and extracts global semantic information from this information.
[0054] The complete process of the robot target navigation method in this embodiment may include:
[0055] 1. Build the Habitat platform and download the corresponding MP3D and HM3D indoor simulation data sets.
[0056] 2. Use the deep learning weight initialization method to initialize the robot target navigation model provided in this embodiment.
[0057] 3. During the training phase, the robot target navigation model is trained using the deep reinforcement learning algorithm DDPPO based on visual observation data and seen categories to learn navigation strategies.
[0058] 4. During the testing phase, the robot’s target navigation model can be tested using visual observation data, as well as seen or unseen categories, to evaluate the learned strategy.
[0059] A large number of experiments are used to prove the effectiveness of the robot target navigation method provided by this embodiment. The experimental results show that the semantic actor strategy module and the semantic trajectory module in the robot target navigation model effectively transfer the learned navigation strategy from the seen category to the unseen category in a zero-shot manner.
[0060] This embodiment aligns the visual features and semantic knowledge of seen categories into a shared space during the training phase, and can use the semantic embeddings of unseen categories and the aligned shared space to process unseen categories during the testing phase. It also biases the trained weights of the actor strategy network using the semantic embedding information corresponding to the current moment, thereby making better use of the semantic embedding information and improving the robot's target navigation accuracy for unseen categories.
[0061] On the basis of the above embodiment, the weights of the actor strategy network trained in the actor-critic algorithm are biased according to the semantic embedding information corresponding to the current moment to obtain the final weights of the actor strategy network, including:
[0062] Inputting the semantic embedding information corresponding to the current moment into the fully connected layer and the Tanh function in sequence to obtain the weight bias matrix of the actor strategy network;
[0063] The final weights of the actor strategy network are obtained according to the trained weights of the actor strategy network and the weight bias matrix.
[0064] The semantic actor strategy module uses a fully connected layer and Tanh function to embed the semantic information corresponding to the current moment into the l Projected into the policy space of the actor-policy network, a weight bias matrix is generated for the actor-policy network. The size of the weight bias matrix is the same as the weight size of the actor-policy network.
[0065] The original weights of the trained actor-strategy network can be added to the weight bias matrix to obtain the final weights of the actor-strategy network.
[0066] The semantic actor strategy module in this embodiment can generate diversified navigation strategies based on different semantic embeddings. When the target category is an unseen category, it can accept the semantic embedding information of the unseen category. By adjusting the weights of the actor strategy network, the navigation strategies learned from the seen categories are transferred to the navigation measurements of the unseen categories. Different navigation strategies can be generated for different unseen categories without training samples.
[0067] On the basis of the above embodiment, the method of obtaining the action of the robot to be navigated at the current moment based on the actor strategy network corresponding to the final weight according to the visual features and semantic embedding information corresponding to each moment in the preset time period and the action of the robot to be navigated at each moment in the preset time period except the current moment, includes:
[0068] The visual features and semantic embedding information corresponding to each moment in the preset time period and the action corresponding to the previous moment of each moment are spliced to obtain the navigation information corresponding to each moment;
[0069] According to the navigation information corresponding to each moment in the preset time period, the action of the robot to be navigated at the current moment is obtained based on the actor strategy network corresponding to the final weight.
[0070] like Figure 2 As shown, for example, the visual feature v corresponding to time t t , semantic embedding information l and action a corresponding to time t-1 t-1 splicing to obtain the navigation information at time t, and taking the navigation information corresponding to time t as a semantic node sn t . The same method can be used to obtain sn t-1 、sn t-2 、sn t-3 Etc. These semantic nodes, such as sn t-7 to sn t Merge into a semantic track to store the information acquired by the robot. For each new moment, add the latest semantic node and delete the semantic node with the longest storage time.
[0071] In each step of the robot to be navigated, the semantic trajectory module connects the action of the previous moment, the visual features of the current moment, and the semantic embedding information of the current moment into a semantic node, and extracts global information from these nodes to obtain better semantic transfer capabilities.
[0072] Based on the above embodiment, in this embodiment, according to the navigation information corresponding to each moment in the preset time period, the action of the robot to be navigated at the current moment is obtained based on the actor strategy network corresponding to the final weight, including:
[0073] The navigation information corresponding to all moments within the preset time period is input into a time series convolutional neural network to obtain the action of the robot to be navigated at the current moment output by the time series convolutional neural network.
[0074] Inspired by the Temporal Convolutional Network (TCN), multiple semantic nodes form a semantic track. The TCN may include three causal convolutions, and the TCN is used to extract global information from these nodes to obtain better semantic transfer capabilities.
[0075] On the basis of the above embodiments, in this embodiment, the visual features of the robot to be navigated are extracted from the visual observation data of the robot to be navigated at each moment within a preset time period before the current moment, including:
[0076] Based on the ResNet (Residual Network)-50 model, the visual features of the robot to be navigated are extracted from the visual observation data of the robot to be navigated at each moment within the preset time period.
[0077] The ResNet-50 model is a deep learning model that contains 50 layers. The ResNet-50 model is pre-trained on the MP3D and HM3D datasets.
[0078] like Figure 2 As shown in Figure 2, the ResNet-50 model is used to treat the visual observation data of the navigation robot at each moment in the preset time period. t Processing is performed to obtain the visual features v of the robot to be navigated at each moment t .
[0079] On the basis of the above embodiments, the method of determining the semantic embedding information of the target category received by the robot to be navigated at each moment within the preset time period in this embodiment includes:
[0080] The semantic embedding information of the target category received by the robot to be navigated at each moment within the preset time period is determined based on the GloVe (Global Vectors for Word Representation, word embedding with global vectors) model.
[0081] The GloVe model is an unsupervised learning algorithm used to obtain vector representations of words. Figure 2 As shown, the GloVe model is used to obtain the semantic embedding information s of each target category l received at each moment in the preset time period. l =GloVe(l).
[0082] The robot target navigation device provided by the present invention is described below. The robot target navigation device described below and the robot target navigation method described above can be referenced to each other.
[0083] Figure 3 Schematic diagram of the structure of the robot target navigation device provided by the present invention. Figure 3 As shown, the device includes an extraction module 301, a bias module 302 and a navigation module 303, wherein:
[0084] The extraction module 301 is used to extract the visual features of the robot to be navigated from the visual observation data of the robot to be navigated at each moment in a preset time period before the current moment, and determine the semantic embedding information of the target category received by the robot to be navigated at each moment in the preset time period, wherein the preset time period includes the current moment;
[0085] The bias module 302 is used to bias the weights of the actor strategy network trained in the actor-critic algorithm according to the semantic embedding information corresponding to the current moment to obtain the final weights of the actor strategy network;
[0086] The navigation module 303 is used to obtain the action of the robot to be navigated at the current moment based on the actor strategy network corresponding to the final weight according to the visual features and semantic embedding information corresponding to each moment in the preset time period and the action of the robot to be navigated at each moment in the preset time period except the current moment;
[0087] The actor-critic algorithm is obtained by training by using the visual observation data and target categories of the robot samples at each moment in the preset time period as samples and the actions corresponding to the robot samples as labels.
[0088] This embodiment aligns the visual features and semantic knowledge of seen categories into a shared space during the training phase, and can use the semantic embeddings of unseen categories and the aligned shared space to process unseen categories during the testing phase. It also biases the trained weights of the actor strategy network using the semantic embedding information corresponding to the current moment, thereby making better use of the semantic embedding information and improving the robot's target navigation accuracy for unseen categories.
[0089] Figure 4 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 4As shown, the electronic device may include: a processor 410, a communication interface 420, a memory 430 and a communication bus 440, wherein the processor 410, the communication interface 420 and the memory 430 communicate with each other through the communication bus 440. The processor 410 may call the logic instructions in the memory 430 to execute the robot target navigation method, which includes: extracting the visual features of the robot to be navigated from the visual observation data of each moment in a preset time period before the current moment of the robot to be navigated, and determining the semantic embedding information of the target category received by the robot to be navigated at each moment in the preset time period, wherein the preset time period includes the current moment; biasing the weight of the actor strategy network trained in the actor-commentator algorithm according to the semantic embedding information corresponding to the current moment to obtain the final weight of the actor strategy network; according to the visual features corresponding to each moment in the preset time period, the semantic embedding information and the actions of the robot to be navigated at each moment in the preset time period except the current moment, obtaining the action of the robot to be navigated at the current moment based on the actor strategy network corresponding to the final weight.
[0090] In addition, the logic instructions in the above-mentioned memory 430 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.
[0091] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the robot target navigation method provided by the above-mentioned methods, which method includes: extracting visual features of the robot to be navigated from visual observation data of the robot to be navigated at each moment within a preset time period before the current moment, and determining semantic embedding information of target categories received by the robot to be navigated at each moment within the preset time period, and the preset time period includes the current moment; biasing the weights of the actor strategy network after training in the actor-commentator algorithm according to the semantic embedding information corresponding to the current moment to obtain the final weights of the actor strategy network; according to the visual features and semantic embedding information corresponding to each moment within the preset time period and the actions of the robot to be navigated at each moment within the preset time period except the current moment, obtaining the action of the robot to be navigated at the current moment based on the actor strategy network corresponding to the final weight.
[0092] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the robot target navigation method provided by the above-mentioned methods, the method comprising: extracting visual features of the robot to be navigated from visual observation data of the robot to be navigated at each moment within a preset time period before the current moment, and determining semantic embedding information of target categories received by the robot to be navigated at each moment within the preset time period, the preset time period including the current moment; biasing the weights of the actor strategy network trained in the actor-commentator algorithm according to the semantic embedding information corresponding to the current moment to obtain the final weights of the actor strategy network; obtaining the action of the robot to be navigated at the current moment based on the actor strategy network corresponding to the final weights according to the visual features and semantic embedding information corresponding to each moment within the preset time period and the action of the robot to be navigated at each moment within the preset time period except the current moment.
[0093] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.
[0094] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.
[0095] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A robot target navigation method, characterized in that: include: Extracting visual features of the robot to be navigated from visual observation data of the robot to be navigated at each moment within a preset time period before the current moment, and determining semantic embedding information of target categories received by the robot to be navigated at each moment within the preset time period, wherein the preset time period includes the current moment; biasing the trained weights of the actor strategy network in the actor-critic algorithm according to the semantic embedding information corresponding to the current moment to obtain the final weights of the actor strategy network; According to the visual features and semantic embedding information corresponding to each moment in the preset time period and the actions of the robot to be navigated at each moment in the preset time period except the current moment, the action of the robot to be navigated at the current moment is obtained based on the actor strategy network corresponding to the final weight; The actor-critic algorithm is obtained by training the visual observation data and target categories of the robot samples at each moment in the preset time period as samples and the actions corresponding to the robot samples as labels; Wherein, the method of obtaining the action of the robot to be navigated at the current moment based on the actor strategy network corresponding to the final weight according to the visual features and semantic embedding information corresponding to each moment in the preset time period and the action of the robot to be navigated at each moment in the preset time period except the current moment comprises: splicing the visual features and semantic embedding information corresponding to each moment in the preset time period and the action corresponding to the previous moment of each moment to obtain the navigation information corresponding to each moment; obtaining the action of the robot to be navigated at the current moment based on the actor strategy network corresponding to the final weight according to the navigation information corresponding to each moment in the preset time period; Among them, according to the navigation information corresponding to each moment in the preset time period, the action of the robot to be navigated at the current moment is obtained based on the actor strategy network corresponding to the final weight, including: inputting the navigation information corresponding to all moments in the preset time period into the time series convolutional neural network, and obtaining the action of the robot to be navigated at the current moment output by the time series convolutional neural network.
2. The robot target navigation method according to claim 1, characterized in that: The biasing of the weights of the actor strategy network trained in the actor-critic algorithm according to the semantic embedding information corresponding to the current moment to obtain the final weights of the actor strategy network includes: Inputting the semantic embedding information corresponding to the current moment into the fully connected layer and the Tanh function in sequence to obtain the weight bias matrix of the actor strategy network; The final weights of the actor strategy network are obtained according to the trained weights of the actor strategy network and the weight bias matrix.
3. The robot target navigation method according to any one of claims 1-2, characterized in that: The step of extracting visual features of the robot to be navigated from visual observation data of the robot to be navigated at each moment within a preset time period before the current moment comprises: Based on the ResNet-50 model, visual features of the robot to be navigated are extracted from the visual observation data of the robot to be navigated at each moment within the preset time period.
4. The robot target navigation method according to claim 1, characterized in that: The determining of semantic embedding information of target categories received by the robot to be navigated at each moment within the preset time period includes: The semantic embedding information of the target category received by the robot to be navigated at each moment within the preset time period is determined based on the GloVe model.
5. A robot target navigation device, characterized in that: include: an extraction module, configured to extract visual features of the robot to be navigated from visual observation data of the robot to be navigated at each moment within a preset time period before a current moment, and determine semantic embedding information of target categories received by the robot to be navigated at each moment within the preset time period, wherein the preset time period includes the current moment; A bias module, configured to bias the weights of the actor strategy network trained in the actor-critic algorithm according to the semantic embedding information corresponding to the current moment, so as to obtain the final weights of the actor strategy network; A navigation module, configured to obtain the action of the robot to be navigated at the current moment based on the actor strategy network corresponding to the final weight according to the visual features and semantic embedding information corresponding to each moment within the preset time period and the action of the robot to be navigated at each moment within the preset time period except the current moment; The actor-critic algorithm is obtained by training the visual observation data and target categories of the robot samples at each moment in the preset time period as samples and the actions corresponding to the robot samples as labels; The navigation module is further used to splice the visual features and semantic embedding information corresponding to each moment in the preset time period and the action corresponding to the previous moment of each moment to obtain the navigation information corresponding to each moment; according to the navigation information corresponding to each moment in the preset time period, the action of the robot to be navigated at the current moment is obtained based on the actor strategy network corresponding to the final weight; Among them, the navigation module is also used to input the navigation information corresponding to all moments in the preset time period into a time series convolutional neural network, and obtain the action of the robot to be navigated at the current moment output by the time series convolutional neural network.
6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the robot target navigation method according to any one of claims 1 to 4 is implemented.
7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the robot target navigation method according to any one of claims 1 to 4 is implemented.
8. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the robot target navigation method according to any one of claims 1 to 4 is implemented.
