Navigation method and device implemented by map-free navigation model based on knowledge guidance, robot and storage medium

Abstract

The invention provides a navigation method and device for realizing a map-free navigation model based on knowledge guidance, a robot and a storage medium, the map-free navigation model is obtained by training a pre-constructed DDPG algorithm model based on a plurality of action rules in a knowledge system, and the generalization is stronger compared with navigation by singly adopting a DDPG algorithm. In the training process, the motion selection randomness of a DDPG algorithm model can be reduced by fusing a guide motion instruction and a strategy motion instruction, so that valuable data can be quickly obtained, the learning efficiency is improved, in an environment with sparse rewards, the mobile robot can interact with the environment under the guidance of knowledge instead of random interaction, and the training efficiency is improved. Local optimum is avoided, and it is guaranteed that convergence is easy in the reward sparse environment; and inputting the strategy action instruction and the integrated action instruction obtained by fusion into the preset loss function, thereby reducing the uncertainty of the corresponding loss function when a DDPG algorithm is singly adopted, and enabling the learning process to be more stable.

Description

Technical Field

[0001] The present invention relates to the field of robot navigation, and in particular to a navigation method, device, robot and storage medium implemented based on a knowledge-guided map-free navigation model. Background Art

[0002] With the continuous development of mobile robot technology, navigation technology has become the key to realize autonomous robot movement. Traditional robot navigation mostly uses SLAM (Simultaneous Localization and Mapping) technology. SLAM technology creates an environmental map in real time through sensor data (such as lidar, camera, etc.) and simultaneously determines the position of the robot in the map, thereby achieving precise navigation. However, in practical applications, this method faces the following challenges: (1) In complex and ever-changing scenarios, SLAM technology has difficulty updating environmental maps in real time, resulting in reduced navigation accuracy; (2) In a dynamic environment, SLAM technology has difficulty handling moving objects and is prone to positioning errors; (3) In unknown environments where maps cannot be built in advance, SLAM technology has difficulty achieving efficient navigation.

[0003] As a result, researchers began to explore navigation strategies that do not rely on map information, and proposed a map-free navigation method based on reinforcement learning (RL). For example, the DDPG (Deep Deterministic Policy Gradient) algorithm is used to enable mobile robots to learn optimal action strategies through interaction with the environment to maximize cumulative rewards. The advantages of this method are: it does not require prior construction or reliance on environmental maps, and is suitable for unknown environments; it makes decisions directly based on sensor input (such as distance, speed, obstacle information, etc.) and immediate feedback to achieve end-to-end navigation. This navigation model obtained by using the DDPG algorithm alone performs well in simple navigation scenarios similar to the virtual environment during training, but is difficult to adapt to complex dynamic navigation scenarios. Summary of the invention

[0004] The purpose of the present invention is to provide a navigation method, device, robot and storage medium based on a knowledge-guided map-free navigation model to improve the problems existing in the prior art.

[0005] The embodiments of the present invention can be implemented as follows: In a first aspect, the present invention provides a navigation method implemented by a knowledge-guided map-free navigation model, comprising: Obtain the environmental state data of the mobile robot at the current moment, the relative position data between the current position and the target point, and the action instructions at the previous moment; Converting the environmental state data and relative position data at the current moment and the action instruction at the previous moment into a current state vector; Inputting the current state vector into a trained mapless navigation model to obtain the action instruction at the current moment; the mapless navigation model is obtained by training a pre-built deep deterministic policy gradient algorithm model based on multiple action rules in a preset knowledge system; The mobile robot is controlled to move toward the target point using the action instruction at the current moment.

[0006] In an optional implementation manner, the map-free navigation model is trained in the following manner: Based on the knowledge system, the deep deterministic policy gradient algorithm model and the generalizer, the intelligent agent is controlled to continuously interact with the preset virtual environment, and each piece of interaction data generated by the interaction is added to an initially empty experience database; When the amount of interaction data in the experience database reaches When Interaction data; is a positive integer; Based on the The deep deterministic policy gradient algorithm model and the generalizer are updated with the interaction data and the multiple action rules in the knowledge system; Based on the knowledge system, the updated deep deterministic policy gradient algorithm model and the updated generalizer, control the agent to continuously interact with the virtual environment, and add each piece of interaction data generated by the interaction to the experience database; Determining whether the number of interaction rounds between the agent and the virtual environment reaches a preset value; If not, then return the sampled The step of collecting pieces of interaction data until the number of interaction rounds between the agent and the virtual environment reaches the preset value; If so, the agent is controlled to stop interacting with the virtual environment, and the current deep deterministic policy gradient algorithm model is used as the map-free navigation model.

[0007] In an optional embodiment, the The step of updating parameters of the deep deterministic policy gradient algorithm model and the generalizer based on the interaction data and the multiple action rules in the knowledge system comprises: For the Interaction data, from the The state vector is obtained from the interaction data, and the state vector is matched with the multiple action rules in the knowledge system to obtain the guiding action instruction; wherein, ; Inputting the state vector into the deep deterministic policy gradient algorithm model to obtain a policy action instruction and a state value; Inputting the guidance action instruction and the strategy action instruction into the generalizer for fusion to obtain a comprehensive action instruction; The current fusion parameter of the generalizer, the state value, the strategy action instruction and the comprehensive action instruction are input into the preset loss function to obtain the first The comprehensive loss corresponding to the interaction data; based on The expected loss is calculated based on the comprehensive loss corresponding to the interaction data, and the model parameters of the deep deterministic policy gradient algorithm model and the fusion parameters of the generalizer are updated based on the expected loss.

[0008] In an optional embodiment, the guiding motion instruction includes a guiding linear velocity and a guiding angular velocity; The plurality of action rules of the knowledge system include at least one precise action rule, at least one fuzzy action rule and at least one mixed action rule, and each of the action rules has associated position data; The precise action rule includes at least one precise condition and conclusion data, the fuzzy action rule includes at least one fuzzy condition and conclusion data, and the mixed action rule includes at least one precise condition, at least one mixed condition and conclusion data; the conclusion data includes a preset linear velocity and a preset angular velocity; The step of matching the state vector with a plurality of action rules in the knowledge system to obtain guiding action instructions comprises: For each of the precise action rules, at least one vector value is obtained from the state vector based on the position data associated with the precise action rule; if the at least one vector value makes each precise condition of the precise action rule true, the preset linear velocity and the preset angular velocity in the precise action rule are used as the target linear velocity and the target angular velocity respectively; For each of the fuzzy action rules, at least one vector value is obtained from the state vector based on the position data associated with the fuzzy action rule; the membership degree between the at least one vector value and the fuzzy sets associated with each fuzzy condition of the fuzzy action rule is calculated, and the obtained membership degrees are multiplied to obtain activation strength, and the obtained activation strength is multiplied by the preset linear speed and the preset angular speed in the fuzzy action rule respectively to obtain the target linear speed and the target angular speed; For each of the mixed action rules, at least one vector value is obtained from the state vector based on the position data associated with the mixed action rule; if the at least one vector value makes each precise condition of the mixed action rule true, the membership degree between the at least one vector value and the fuzzy sets associated with each fuzzy condition of the mixed action rule is calculated, and the obtained membership degrees are multiplied to obtain activation strength, and the obtained activation strength is multiplied with the preset linear speed and the preset angular speed in the mixed action rule respectively to obtain the target linear speed and the target angular speed; The guide linear velocity is obtained by adding all the target linear velocities, and the guide angular velocity is obtained by intersecting all the target angular velocities.

[0009] In an optional embodiment, the guidance action instruction includes a guidance linear velocity and a guidance angular velocity; the strategy action instruction includes a predicted linear velocity and a predicted angular velocity; the comprehensive action instruction includes a comprehensive linear velocity and a comprehensive angular velocity; The step of inputting the guiding action instruction and the strategy action instruction into the generalizer for fusion to obtain a comprehensive action instruction includes: inputting the guide linear velocity, the guide angular velocity, the predicted linear velocity and the predicted angular velocity into the generalizer; Using the current fusion parameter, weighted processing is performed on the predicted linear velocity and the guided linear velocity to obtain a pending linear velocity, and Gaussian noise is added to the pending linear velocity to obtain the comprehensive linear velocity; The predicted angular velocity and the guided angular velocity are weighted by using the current fusion parameter to obtain a pending angular velocity, and the Gaussian noise is added to the pending angular velocity to obtain the comprehensive angular velocity.

[0010] In an optional embodiment, the strategic action instruction includes a predicted linear velocity and a predicted angular velocity; The calculation formula of the preset loss function is:

[0011]

[0012]

[0013] in, represents said comprehensive loss, represents the line speed loss, represents the angular velocity loss; is a fixed parameter, indicating the credibility of the knowledge system; represents the value of the state, is the current fusion parameter of the generalizer, represents the predicted linear velocity, represents the integrated linear velocity, represents the predicted angular velocity, represents the integrated angular velocity.

[0014] In a second aspect, the present invention provides a navigation device implemented based on a knowledge-guided map-free navigation model, comprising: The state acquisition module is used to obtain the environmental state data of the mobile robot at the current moment, the relative position data between the current position and the target point, and the action instructions at the previous moment; The state acquisition module is further used to convert the environmental state data and relative position data at the current moment and the action instruction at the previous moment into a current state vector; A prediction module, used for inputting the current state vector into a trained mapless navigation model to obtain the action instruction at the current moment; the mapless navigation model is obtained by training a pre-built deep deterministic policy gradient algorithm model based on multiple action rules in a preset knowledge system; The control module is used to control the mobile robot to move toward the target point by using the linear velocity and angular velocity in the action instruction at the current moment.

[0015] In a third aspect, the present invention provides a mobile robot, comprising: a memory and a processor, wherein the memory stores a software program, and when the mobile robot is running, the processor executes the software program to implement the navigation method as described in the first aspect above.

[0016] In a fourth aspect, the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the navigation method described in the first aspect is implemented.

[0017] Compared with the prior art, the embodiment of the present invention provides a navigation method, device, robot and storage medium implemented by a mapless navigation model guided by knowledge. The navigation method is: first, the environmental state data of the mobile robot at the current moment, the relative position data between the current position and the target point, and the action instruction at the previous moment are obtained; then the environmental state data and relative position data at the current moment and the action instruction at the previous moment are converted into a current state vector; then the current state vector is input into the trained mapless navigation model to obtain the action instruction at the current moment; finally, the action instruction at the current moment is used to control the mobile robot to move toward the target point. The mapless navigation model of the present invention is obtained by training a pre-constructed deep deterministic policy gradient algorithm model based on multiple action rules in a preset knowledge system, rather than using a single DDPG algorithm. This makes the mapless navigation model not only adaptable to navigation scenarios similar to the training environment, but also adaptable to complex dynamic navigation scenarios, ensuring the accuracy of navigation. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0019] Figure 1 A flowchart of a navigation method provided by an embodiment of the present invention.

[0020] Figure 2 A schematic diagram of the training process of a map-free navigation model provided in an embodiment of the present invention.

[0021] Figure 3 A schematic diagram of a scenario in which an intelligent agent interacts with a virtual environment provided in an embodiment of the present invention.

[0022] Figure 4 A structural schematic diagram of a navigation device provided by an embodiment of the present invention.

[0023] Figure 5 A schematic structural diagram of a mobile robot provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0025] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0026] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0027] It should be noted that, in the absence of conflict, the features in the embodiments of the present invention may be combined with each other.

[0028] In the prior art, researchers have proposed a map-free navigation method based on reinforcement learning. For example, the DDPG algorithm is used to enable a mobile robot to learn the optimal action strategy through interaction with the environment to maximize the cumulative reward. The advantages of this method are: no need to build or rely on environmental maps in advance, suitable for unknown environments; decisions are made directly based on sensor input (such as distance, speed, obstacle information, etc.) and instant feedback to achieve end-to-end navigation. During the training process, this navigation model obtained by using only the DDPG algorithm has the following problems: (1) Low learning efficiency, especially in sparse reward environments, where it is difficult for the algorithm to quickly converge to an effective strategy; (2) The learning process has poor stability and is easily affected by noise and interference.

[0029] Moreover, this navigation model obtained by using only the DDPG algorithm performs well in simple navigation scenarios similar to the virtual environment during training, but is difficult to adapt to complex dynamic navigation scenarios.

[0030] Based on the discovery of the above technical problems, the inventors have proposed the following technical solutions to solve or improve the above problems through creative work. It should be noted that the defects in the solutions in the above prior art are the results obtained by the inventors after practice and careful research. Therefore, the discovery process of the above problems and the solutions proposed in the embodiments of the present application for the above problems below should all be the contributions made by the inventors to the present application in the process of invention and creation, and should not be understood as technical contents known to those skilled in the art.

[0031] In view of the shortcomings of the prior art, the inventors proposed a map-free navigation model, which introduces a knowledge system as a supervisor during the training process, so that the model can converge quickly during training and improve learning efficiency. The following is a detailed description through embodiments and the accompanying drawings.

[0032] Please refer to Figure 1 , Figure 1 The flowchart of a navigation method provided by an embodiment of the present invention is a flowchart of a navigation method provided by an embodiment of the present invention, wherein the navigation method is implemented based on a knowledge-guided mapless navigation model, and the execution subject of the navigation method is a mobile robot. The navigation method includes the following steps S201 to S204.

[0033] S201, obtaining the environmental state data of the mobile robot at the current moment, the relative position data between the current position and the target point, and the action instruction at the previous moment.

[0034] In this embodiment, the environmental state data may include data collected and processed by various sensors carried by the mobile robot, such as obstacle point cloud data, acceleration data, image data, etc. The relative position data between the current position and the target point may include a straight-line distance and a deflection angle, where the deflection angle is the angle between the line connecting the current position and the target point and the forward direction of the mobile robot. The action instruction at the previous moment includes a linear velocity and an angular velocity, where the linear velocity is the traveling velocity of the mobile robot and the angular velocity is the rotation velocity of the mobile robot.

[0035] S202: Convert the current environmental state data and relative position data and the action instruction at the previous moment into a current state vector.

[0036] Optionally, in the environmental state data, the distance data between the mobile robot and the obstacle can be determined by using the obstacle point cloud data, and the image data can be subjected to image recognition to obtain obstacle determination data, which indicates whether there are obstacles around the mobile robot. Therefore, the distance data, acceleration data, obstacle determination data, relative position data, and the linear velocity and angular velocity at the previous moment can be converted into the current state vector.

[0037] S203: Input the current state vector into the trained map-free navigation model to obtain the action instruction at the current moment.

[0038] In this embodiment, the map-free navigation model is obtained by training a pre-built deep deterministic policy gradient algorithm model based on multiple action rules in a preset knowledge system. An action rule reflects the linear velocity and angular velocity that the robot should adopt under specific conditions.

[0039] S204. Use the current action command to control the mobile robot to move toward the target point.

[0040] In this embodiment, the mobile robot can be controlled to move toward the target point by using the linear velocity and angular velocity in the motion instruction at the current moment.

[0041] That is, the mobile robot starts from the starting point and repeatedly executes the above steps S201 to S204 until it reaches the target point (i.e., the end point), thus completing a map-free navigation from the starting point to the end point.

[0042] The navigation method provided by the embodiment of the present invention first converts the environmental state data of the mobile robot at the current moment, the relative position data between the current position and the target point, and the action instructions at the previous moment into the current state vector; then the current state vector is input into the trained mapless navigation model to obtain the action instructions at the current moment; finally, the action instructions at the current moment are used to control the mobile robot to move toward the target point. The mapless navigation model of the present invention is obtained by training a pre-constructed deep deterministic policy gradient algorithm model based on multiple action rules in a preset knowledge system, rather than using a single DDPG algorithm. This makes the mapless navigation model not only adaptable to navigation scenarios similar to the training environment, but also adaptable to complex dynamic navigation scenarios, ensuring the accuracy of navigation.

[0043] The following is an introduction to the training process of the mapless navigation model. The training process can be implemented by a computing device, such as a laptop, a tablet computer, a desktop computer, a server, etc.

[0044] See also Figure 2 , Figure 2 A schematic diagram of a training process of a map-free navigation model provided by an embodiment of the present invention. The training process may include the following steps S101 to S107.

[0045] S101. Based on the knowledge system, deep deterministic policy gradient algorithm model and generalizer, the intelligent agent is controlled to continuously interact with the preset virtual environment, and each interaction data generated by the interaction is added to the initially empty experience database.

[0046] In this embodiment, the experience database is initially empty, and the agent is a virtual robot that simulates a mobile robot by a computing device. In the process of the agent continuously interacting with the preset virtual environment, an interaction round is: in the virtual environment, the agent moves from the starting point to the target point until it reaches the target point or collides with an obstacle in the virtual environment. In each interaction round, multiple interaction data are generated and added to the experience database. An interaction data in the experience database is represented as:

[0047] in, Indicates The state vector at the moment, Indicates The mixed action command output at each moment, Indicates A moment of reward, Representative The state data includes the environment state data (such as obstacle point cloud data, acceleration data, image data, etc.), the relative position data between the current position and the target point (straight-line distance and deflection angle), and the mixed action instructions at the previous moment.

[0048] S102: When the number of interaction data in the experience database reaches When Interaction data.

[0049] In this embodiment, is a positive integer and The size can be set by yourself. For example, It can be 128 or 256. This example is only an example and is not limited here.

[0050] by For example, when the number of interaction data in the experience database reaches 256, the interaction data can be sampled to train and update the deep deterministic policy gradient algorithm model and the generalizer through the following step S103. Moreover, during the training and updating, the intelligent agent continues to interact with the virtual environment.

[0051] S103, based on It updates the parameters of the deep deterministic policy gradient algorithm model and generalizer based on multiple action rules in the interactive data and knowledge system.

[0052] In this embodiment, the sampled The deep deterministic policy gradient algorithm model and the generalizer can be trained and updated once with the interaction data to obtain the updated deep deterministic policy gradient algorithm model and the updated generalizer.

[0053] S104. Based on the knowledge system, the updated deep deterministic policy gradient algorithm model and the updated generalizer, the intelligent agent is controlled to continuously interact with the virtual environment, and each piece of interaction data generated by the interaction is added to the experience database.

[0054] In this embodiment, after the training is updated once, it is necessary to continue to control the intelligent agent to interact with the virtual environment based on the knowledge system, the updated deep deterministic policy gradient algorithm model and the updated generalizer, and add each interaction data generated by the interaction to the experience database.

[0055] It is assumed that the environmental state data only includes obstacle point cloud data. Figure 3 In the above steps S101 and S104, the process of each interaction between the agent and the virtual environment is as follows: obtaining the obstacle point cloud data of the agent at the current moment, the relative position data between the current position and the target point, and the action instruction at the previous moment, and then converting the obstacle point cloud data at the current moment, the relative position data between the current position and the target point, and the action instruction at the previous moment into a state vector, inputting the state vector into the deep deterministic policy gradient algorithm model to obtain the policy action instruction, and matching the state vector with multiple action rules in the knowledge system to obtain the guiding action instruction; then inputting the guiding action instruction and the policy action instruction into the generalizer for fusion to obtain the comprehensive action instruction, and using the comprehensive action instruction to control the agent to move toward the target point, and at the same time, an interaction data is generated during this interaction. Join the experience database.

[0056] S105: Determine whether the number of interaction rounds between the intelligent agent and the virtual environment reaches a preset value.

[0057] In this embodiment, if the number of interaction rounds between the agent and the virtual environment does not reach the preset value, the process directly returns to the step S102 of sampling from the experience database. The process of collecting pieces of interaction data is continued until the number of interaction rounds between the agent and the virtual environment reaches a preset value. If the number of interaction rounds between the agent and the virtual environment reaches the preset value, the following step S106 is executed.

[0058] Optionally, the preset value can be set based on actual conditions. For example, the preset value can be 10,000 or 15,000. This example is only an example and is not limited here.

[0059] S106. Control the intelligent agent to stop interacting with the virtual environment and use the current deep deterministic policy gradient algorithm model as a map-free navigation model.

[0060] Through steps S101 to S106, a map-free navigation model based on knowledge guidance is obtained, and the map-free navigation model can be used to implement the map-free navigation task of the mobile robot.

[0061] In an optional implementation, in the above step S103, "based on The process of "updating parameters of a deep deterministic policy gradient algorithm model and a generalizer based on multiple action rules in the interactive data and knowledge system" may include the following sub-steps S1031~S1035.

[0062] S1031, for Interaction data, from The state vector is obtained from the interaction data, and the state vector is matched with multiple action rules in the knowledge system to obtain the guiding action instructions; wherein, .

[0063] In this embodiment, from The state vector obtained from the interaction data is The guidance motion instruction includes the guidance linear velocity and the guidance angular velocity.

[0064] Optionally, the construction of the knowledge system involves representing knowledge with fuzzy semantics and precise semantics. The multiple action rules of the knowledge system include at least one precise action rule, at least one fuzzy action rule, and at least one mixed action rule. Each action rule has associated position data, which represents the position of at least one vector value associated with the action rule in the state vector. In the knowledge system, the conditions and conclusions contained in the three types of action rules are as follows: 1. Precise rules of action: An exact action rule can include at least one exact condition and conclusion data, so an exact action rule can be expressed as:

[0065] in, Indicates Precise rules of action, Indicates The first of the precise rules of action A precise condition; Indicates The conclusion data of the precise action rules, Respectively represent Preset linear speed and preset angular speed for precise action rules; 2. Fuzzy rules of action: A fuzzy action rule can include at least one fuzzy condition and conclusion data, so a fuzzy action rule can be expressed as:

[0066] in, Indicates Fuzzy rules of action, Indicates The first of the fuzzy rules of action A fuzzy condition; Indicates The conclusion data of the fuzzy action rules, Respectively represent The preset linear velocity and the preset angular velocity of the fuzzy action rules; 3. Rules of hybrid operations: A mixed action rule may include at least one precise condition, at least one mixed condition and conclusion data, so a mixed action rule can be expressed as:

[0067] in, Indicates hybrid action rules, Indicates Article 1 of the hybrid operation rules Fuzzy conditions, No. Article 1 of the hybrid operation rules A precise condition; Respectively represent The preset linear velocity and the preset angular velocity of the mixed action rule.

[0068] Therefore, in step S1031, the process of "matching the state vector with multiple action rules in the knowledge system to obtain guiding action instructions" may include the following sub-steps S10311~S10314.

[0069] S10311. For each precise action rule, obtain at least one vector value from the state vector based on the position data associated with the precise action rule; if at least one vector value makes each precise condition of the precise action rule true, then the preset linear velocity and preset angular velocity in the precise action rule are used as the target linear velocity and target angular velocity, respectively.

[0070] In this embodiment, for a precise action rule An exact condition , because of its binary nature, the exact condition The activation strength It can be expressed as:

[0071] So, precise rules of action The activation strength for: .

[0072] Therefore, for each precise action rule, after at least one vector value is obtained from the state vector based on the position data associated with the precise action rule; only when the at least one vector value makes each precise condition of the precise action rule meet (that is, each precise condition is true), the precise action rule will be deemed to be activated, and the preset linear velocity and preset angular velocity in the precise action rule will be used as the target linear velocity and target angular velocity, respectively.

[0073] For example, "if there is an obstacle on the left, turn right" is a precise rule in the human cognitive system. If a precise action rule of a mobile robot is determined based on this precise rule and added to the knowledge system, it needs to be expressed as: "if there is an obstacle on the left, then and "This example is only an example and is not intended to be limiting.

[0074] S10312. For each fuzzy action rule, obtain at least one vector value from the state vector based on the position data associated with the fuzzy action rule; calculate the membership of at least one vector value between the fuzzy sets associated with each fuzzy condition of the fuzzy action rule, and multiply the obtained memberships to obtain activation strength, and multiply the obtained activation strength with the preset linear velocity and preset angular velocity in the fuzzy action rule to obtain the target linear velocity and target angular velocity.

[0075] In this embodiment, for a fuzzy action rule An exact condition , due to its fuzziness, the membership degree is used as the precise condition The activation strength , expressed as:

[0076] in, It is an exact condition The vector value corresponding to the subject variable in , It is an exact condition The associated fuzzy set, express Relative to the exact conditions Related fuzzy sets The degree of membership between .

[0077] Therefore, fuzzy action rules The activation strength for: .

[0078] Therefore, for each fuzzy action rule, after determining the activation strength of the fuzzy action rule, a set of target linear velocity and target angular velocity can be obtained by multiplying the activation strength with the preset linear velocity and preset angular velocity of the fuzzy action rule.

[0079] For example, "if the obstacle is very close, stop" is a fuzzy rule in the human cognitive system. If a fuzzy action rule of a mobile robot is determined based on this fuzzy rule and added to the knowledge system, it is necessary to first define how close the distance is: assuming that the domain of the fuzzy set is 0~5m, and the distance between 0~0.2m is considered very close, then determine the center point and standard deviation , the Gaussian membership function used is defined as:

[0080] In this way, at a distance The closer it is to 0, the higher the membership The closer it is to 1. 0.2m, is about 0.98, so the fuzzy action rule can be expressed as: “if , then and "This example is only an example and is not intended to be limiting.

[0081] S10313. For each mixed action rule, obtain at least one vector value from the state vector based on the position data associated with the mixed action rule; if at least one vector value makes each precise condition of the mixed action rule true, calculate the membership of at least one vector value between the fuzzy sets associated with each fuzzy condition of the mixed action rule, multiply the obtained memberships to obtain the activation strength, and multiply the obtained activation strength with the preset linear velocity and preset angular velocity in the mixed action rule to obtain the target linear velocity and target angular velocity.

[0082] In this embodiment, for a mixed action rule, the premise for the mixed action rule to be activated is that the activation intensity of each precise condition therein is 1 (that is, each precise condition is established), and then the membership degrees of each fuzzy condition are multiplied to obtain the activation intensity of the mixed action rule, and finally the activation intensity is multiplied with the preset linear velocity and preset angular velocity in the mixed action rule respectively to obtain a set of target linear velocity and target angular velocity.

[0083] The method for calculating the membership degree of each fuzzy condition in the mixed action rule is consistent with the method for calculating the membership degree of each fuzzy condition in the above fuzzy action rule, and will not be elaborated here.

[0084] For example, "if the obstacle is on the left and you are close to the obstacle, turn right" is a mixed rule in the human cognitive system, in which "the obstacle is on the left" is a precise condition and "close to the obstacle" is a fuzzy condition. Then, combined with the above example, a mixed action rule determined based on this can be expressed as: "if there is an obstacle on the left and , then and "This example is only an example and is not intended to be limiting.

[0085] S10314. Add all target linear velocities to obtain a guiding linear velocity, and intersect all target angular velocities to obtain a guiding angular velocity.

[0086] In this embodiment, for Interaction data, the The state vector corresponding to the interaction data is input into the knowledge system. Through the above steps S10311 to S10314, multiple target linear velocities and multiple target angular velocities can be determined. The guide linear velocity is obtained by adding all the target linear velocities, and the guide angular velocity is obtained by intersecting all the target angular velocities. In this way, the first The guidance action instructions corresponding to the interaction data.

[0087] S1032. Input the state vector into the deep deterministic policy gradient algorithm model to obtain the policy action instruction and state value.

[0088] In this embodiment, the deep deterministic policy gradient algorithm model includes an Actor network and a Critic network. The state vector is first input into the Actor network to obtain a policy action instruction, which includes a predicted linear velocity and a predicted angular velocity. The policy action instruction is input into the Critic network to obtain the state value. The detailed processing process in the state vector deep deterministic policy gradient algorithm model is prior art and will not be described in detail here.

[0089] S1033: Input the guidance action instruction and the strategy action instruction into the generalizer for fusion to obtain a comprehensive action instruction.

[0090] In this embodiment, the comprehensive motion instruction output by the generalizer includes a comprehensive linear velocity and a comprehensive angular velocity.

[0091] Optionally, the sub-steps of step S1033 may include: (1) Input the guided linear velocity, guided angular velocity, predicted linear velocity and predicted angular velocity into the generalizer; (2) Using the current fusion parameters, the predicted linear velocity and the guided linear velocity are weighted to obtain the undetermined linear velocity, and Gaussian noise is added to the undetermined linear velocity to obtain the comprehensive linear velocity; (3) The predicted angular velocity and the guided angular velocity are weighted by using the current fusion parameters to obtain the undetermined angular velocity. Gaussian noise is added to the undetermined angular velocity to obtain the comprehensive angular velocity.

[0092] Among them, the calculation formula of comprehensive linear speed is as follows:

[0093] in, is the comprehensive linear speed, is the fusion parameter of the generalizer, is the guide line speed, To predict the linear velocity; represents Gaussian noise, is the mean of the Gaussian noise, is the variance of the Gaussian noise, is the input to the Gaussian noise function.

[0094] The calculation formula of the comprehensive angular velocity is as follows:

[0095] in, is the integrated angular velocity, is the guide angular velocity, To predict the angular velocity.

[0096] S1034, input the current fusion parameters, state value, strategy action instructions and comprehensive action instructions of the generalizer into the preset loss function to obtain the The comprehensive loss corresponding to the interaction data.

[0097] In this embodiment, for For each piece of interaction data, execute steps S1031 to S1034 to obtain A comprehensive loss. Among them, the calculation formula of the preset loss function is:

[0098]

[0099]

[0100] in, represents comprehensive loss, represents the line speed loss, represents the angular velocity loss; is a fixed parameter, indicating the credibility of the knowledge system; Represents the state value.

[0101] S1035, based on The expected loss is calculated based on the comprehensive loss corresponding to the interaction data, and the model parameters of the deep deterministic policy gradient algorithm model and the fusion parameters of the generalizer are updated based on the expected loss.

[0102] In this embodiment, based on The corresponding interaction data The comprehensive loss can be used to calculate the expected loss (i.e., the average loss). The expected loss can be used to update the model parameters of the deep deterministic policy gradient algorithm model and the fusion parameters of the generalizer. The specific update process is an existing technology and will not be described in detail here.

[0103] From the calculation formula of the above preset loss function, it can be seen that if the fixed parameters Not 0, which means that even if the fusion parameter of the generalizer When the model parameter update of the deep deterministic policy gradient algorithm model always has a certain proportion of supervision. Therefore, for knowledge systems with extremely high reliability, fixed parameters can be set. , so that the model parameter update of the deep deterministic policy gradient algorithm model has a certain proportion of supervision.

[0104] It should be noted that the execution order of each step in the above method embodiment is not limited to that shown in the drawings, and the execution order of each step shall be based on the actual application situation.

[0105] Compared with the prior art, the embodiments of the present invention have the following beneficial effects: The mapless navigation model of the present invention is obtained by training a pre-built deep deterministic policy gradient algorithm model based on multiple action rules in a preset knowledge system, rather than using only the DDPG algorithm. This makes the mapless navigation model not only adaptable to navigation scenarios similar to the training environment, but also adaptable to complex dynamic navigation scenarios, and the navigation generalization is stronger; In the training process of the map-free navigation model, the present invention inputs the state vector into the deep deterministic policy gradient algorithm model to obtain the policy action instruction, and also inputs the state vector into the knowledge system for rule matching to obtain the guidance action instruction. Finally, the guidance action instruction and the policy action instruction are integrated to obtain the comprehensive action instruction, thereby reducing the randomness of the deep deterministic policy gradient algorithm model for action selection, thereby being able to quickly obtain valuable data and improve learning efficiency. In the training process of the map-free navigation model, the present invention inputs both the strategy action instructions and the integrated action instructions obtained by fusion into a preset loss function, thereby reducing the uncertainty of the corresponding loss function when the DDPG algorithm is used alone, thereby making the learning process more stable; Since the present invention integrates the guidance action instructions with the strategy action instructions to obtain the comprehensive action instructions, in an environment with sparse rewards, the mobile robot can interact with the environment under the guidance of knowledge rather than randomly interacting, thus avoiding falling into the local optimum and ensuring easy convergence in an environment with sparse rewards.

[0106] In order to execute the corresponding steps in the above method embodiment and various possible implementations, an implementation of a navigation device is provided below.

[0107] See also Figure 4 , Figure 4 The schematic diagram of the structure of the navigation device provided by the embodiment of the present invention is shown. The navigation device 200 comprises: a state acquisition module 210 , a prediction module 220 and a control module 230 .

[0108] The state acquisition module 210 is used to acquire the environmental state data of the mobile robot at the current moment, the relative position data between the current position and the target point, and the action instruction at the previous moment; The state acquisition module 210 is also used to convert the current environmental state data and relative position data and the action instruction of the previous moment into the current state vector; The prediction module 220 is used to input the current state vector into the trained map-free navigation model to obtain the action instruction at the current moment; the map-free navigation model is obtained by training a pre-built deep deterministic policy gradient algorithm model based on multiple action rules in a preset knowledge system; The control module 230 is used to control the mobile robot to move toward the target point by using the linear velocity and angular velocity in the action instruction at the current moment.

[0109] Optionally, the map-free navigation model is trained by: based on the knowledge system, the pre-built deep deterministic policy gradient algorithm model and the generalizer, the agent is controlled to interact with the preset virtual environment to build an experience database; the experience database is initially empty; when the number of interaction data in the experience database is greater than When Interaction data; based on The system updates the parameters of the deep deterministic policy gradient algorithm model and the generalizer based on the interaction data and multiple action rules in the knowledge system; based on the knowledge system, the updated deep deterministic policy gradient algorithm model and the generalizer, controls the agent to continue to interact with the virtual environment to expand the experience database; determines whether the number of interaction rounds between the agent and the virtual environment reaches the preset value; if not, returns the sampled value from the experience database. The step of performing the above steps is to perform the above steps until the number of interaction rounds between the agent and the virtual environment reaches the preset value; if so, the agent is controlled to stop interacting with the virtual environment and the current deep deterministic policy gradient algorithm model is used as the map-free navigation model.

[0110] Optional, based on The steps of updating the parameters of the deep deterministic policy gradient algorithm model and the generalizer based on the interaction data and multiple action rules in the knowledge system include: Interaction data, from The state vector is obtained from the interaction data, and the state vector is matched with multiple action rules in the knowledge system to obtain the guiding action instructions; wherein, ; Input the state vector into the deep deterministic policy gradient algorithm model to obtain the policy action instruction and state value; Input the guidance action instruction and the policy action instruction into the generalizer for fusion to obtain the comprehensive action instruction; Input the current fusion parameters, state value, policy action instruction and comprehensive action instruction of the generalizer into the preset loss function to obtain the first The comprehensive loss corresponding to the interaction data; based on The expected loss is calculated based on the comprehensive loss corresponding to the interaction data, and the model parameters of the deep deterministic policy gradient algorithm model and the fusion parameters of the generalizer are updated based on the expected loss.

[0111] Optionally, the guiding action instruction includes guiding linear velocity and guiding angular velocity. The multiple action rules of the knowledge system include at least one precise action rule, at least one fuzzy action rule and at least one mixed action rule, and each action rule has associated position data; the precise action rule includes at least one precise condition and conclusion data, the fuzzy action rule includes at least one fuzzy condition and conclusion data, and the mixed action rule includes at least one precise condition, at least one mixed condition and conclusion data; the conclusion data includes a preset linear velocity and a preset angular velocity. The step of matching the state vector with the multiple action rules in the knowledge system to obtain the guiding action instruction includes: for each precise action rule, obtaining at least one vector value from the state vector based on the position data associated with the precise action rule; if at least one vector value makes each precise condition of the precise action rule true, then the preset linear velocity and preset angular velocity in the precise action rule are used as the target linear velocity and target angular velocity respectively; for each fuzzy action rule, obtaining at least one vector value from the state vector based on the position data associated with the fuzzy action rule; calculating the membership of at least one vector value relative to the fuzzy sets associated with each fuzzy condition of the fuzzy action rule, and multiplying the obtained memberships to obtain the activation strength, and the obtained activation The intensity is multiplied with the preset linear speed and preset angular speed in the fuzzy action rule to obtain the target linear speed and target angular speed; for each mixed action rule, at least one vector value is obtained from the state vector based on the position data associated with the mixed action rule; if at least one vector value makes each precise condition of the mixed action rule true, then the membership of at least one vector value relative to the fuzzy sets associated with each fuzzy condition of the mixed action rule is calculated, and the obtained memberships are multiplied to obtain the activation intensity, and the obtained activation intensity is multiplied with the preset linear speed and preset angular speed in the mixed action rule to obtain the target linear speed and target angular speed; all target linear velocities are added to obtain the guide linear velocity, and all target angular velocities are intersected to obtain the guide angular velocity.

[0112] Optionally, the guidance action instruction includes the guidance linear velocity and the guidance angular velocity; the strategy action instruction includes the predicted linear velocity and the predicted angular velocity; the comprehensive action instruction includes the comprehensive linear velocity and the comprehensive angular velocity. The guidance action instruction and the strategy action instruction are input into the generalizer for fusion to obtain the comprehensive action instruction, including: inputting the guidance linear velocity, the guidance angular velocity, the predicted linear velocity and the predicted angular velocity into the generalizer; performing weighted processing on the predicted linear velocity and the guidance linear velocity using the current fusion parameters to obtain the pending linear velocity, adding Gaussian noise to the pending linear velocity to obtain the comprehensive linear velocity; performing weighted processing on the predicted angular velocity and the guidance angular velocity using the current fusion parameters to obtain the pending angular velocity, adding Gaussian noise to the pending angular velocity to obtain the comprehensive angular velocity.

[0113] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the navigation device 200 described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0114] See also Figure 5 , Figure 5 The mobile robot 300 includes a processor 310 , a memory 320 , and a bus 330 , wherein the processor 310 is connected to the memory 320 via the bus 330 .

[0115] The memory 320 may be used to store software programs, for example, software programs corresponding to the navigation device 200 provided in the embodiment of the present invention. The processor 310 executes various functional applications and data processing to implement the navigation method provided in the embodiment of the present invention by running the software programs stored in the memory 320.

[0116] Among them, the memory 320 can be but is not limited to: RAM (Random Access Memory), ROM (Read Only Memory), FLASH (Flash Memory), PROM (Programmable Read-Only Memory), EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electric Erasable Programmable Read-Only Memory), etc.

[0117] The processor 310 may be an integrated circuit chip with signal processing capability. The processor 310 may be a general-purpose processor, including: CPU (Central Processing Unit), NP (Network Processor), SoC (System on Chip), etc.; it may also be: DSP (Digital Signal Processing), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0118] Understandably, Figure 5The structure shown is for illustration only. The mobile robot 300 may also include Figure 5 More or fewer components as shown, or with Figure 5 Different configurations are shown. Figure 5 Each component shown in the figure can be implemented by hardware, software or a combination thereof.

[0119] The embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the navigation method disclosed in the above embodiment is implemented. The computer-readable storage medium can be, but is not limited to, various media that can store program codes, such as a USB flash drive, a mobile hard disk, a ROM, a RAM, a PROM, an EPROM, an EEPROM, a FLASH disk, or an optical disk.

[0120] In summary, the embodiments of the present invention provide a navigation method, device, robot and storage medium implemented based on a knowledge-guided mapless navigation model. The navigation method is: first, obtain the environmental state data of the mobile robot at the current moment, the relative position data between the current position and the target point, and the action instructions at the previous moment; then convert the environmental state data and relative position data at the current moment and the action instructions at the previous moment into the current state vector; then input the current state vector into the trained mapless navigation model to obtain the action instructions at the current moment; finally, use the action instructions at the current moment to control the mobile robot to move toward the target point. The mapless navigation model of the present invention is obtained by training a pre-constructed deep deterministic policy gradient algorithm model based on multiple action rules in a preset knowledge system, rather than using a single DDPG algorithm. This makes the mapless navigation model not only adaptable to navigation scenarios similar to the training environment, but also adaptable to complex dynamic navigation scenarios, ensuring the accuracy of navigation.

[0121] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A navigation method based on a knowledge-guided map-free navigation model, characterized in that: include: Obtain the environmental state data of the mobile robot at the current moment, the relative position data between the current position and the target point, and the action instructions at the previous moment; Converting the environmental state data and relative position data at the current moment and the action instruction at the previous moment into a current state vector; Inputting the current state vector into a trained map-free navigation model to obtain an action instruction at the current moment; The map-free navigation model is obtained by training a pre-built deep deterministic policy gradient algorithm model based on multiple action rules in a preset knowledge system; The mobile robot is controlled to move toward the target point using the action instruction at the current moment.

2. The method according to claim 1, characterized in that The map-free navigation model is trained in the following way: Based on the knowledge system, the deep deterministic policy gradient algorithm model and the generalizer, the intelligent agent is controlled to continuously interact with the preset virtual environment, and each piece of interaction data generated by the interaction is added to an initially empty experience database; When the amount of interaction data in the experience database reaches When Interaction data; is a positive integer; Based on the The deep deterministic policy gradient algorithm model and the generalizer are updated with the interaction data and the multiple action rules in the knowledge system; Based on the knowledge system, the updated deep deterministic policy gradient algorithm model and the updated generalizer, control the agent to continuously interact with the virtual environment, and add each piece of interaction data generated by the interaction to the experience database; Determining whether the number of interaction rounds between the agent and the virtual environment reaches a preset value; If not, then return the sampled The step of collecting pieces of interaction data until the number of interaction rounds between the agent and the virtual environment reaches the preset value; If so, the agent is controlled to stop interacting with the virtual environment, and the current deep deterministic policy gradient algorithm model is used as the map-free navigation model.

3. The method according to claim 2, characterized in that Based on the The step of updating parameters of the deep deterministic policy gradient algorithm model and the generalizer based on the interaction data and the multiple action rules in the knowledge system comprises: For the Interaction data, from the The state vector is obtained from the interaction data, and the state vector is matched with the multiple action rules in the knowledge system to obtain the guiding action instruction; wherein, ; Inputting the state vector into the deep deterministic policy gradient algorithm model to obtain a policy action instruction and a state value; Inputting the guidance action instruction and the strategy action instruction into the generalizer for fusion to obtain a comprehensive action instruction; The current fusion parameter of the generalizer, the state value, the strategy action instruction and the comprehensive action instruction are input into the preset loss function to obtain the first The comprehensive loss corresponding to the interaction data; based on The expected loss is calculated based on the comprehensive loss corresponding to the interaction data, and the model parameters of the deep deterministic policy gradient algorithm model and the fusion parameters of the generalizer are updated based on the expected loss.

4. The method according to claim 3, characterized in that: The guiding motion instruction includes a guiding linear velocity and a guiding angular velocity; The plurality of action rules of the knowledge system include at least one precise action rule, at least one fuzzy action rule and at least one mixed action rule, and each of the action rules has associated position data; The precise action rule includes at least one precise condition and conclusion data, the fuzzy action rule includes at least one fuzzy condition and conclusion data, and the mixed action rule includes at least one precise condition, at least one mixed condition and conclusion data; the conclusion data includes a preset linear velocity and a preset angular velocity; The step of matching the state vector with a plurality of action rules in the knowledge system to obtain guiding action instructions comprises: For each of the precise action rules, at least one vector value is obtained from the state vector based on the position data associated with the precise action rule; if the at least one vector value makes each precise condition of the precise action rule true, the preset linear velocity and the preset angular velocity in the precise action rule are used as the target linear velocity and the target angular velocity respectively; For each of the fuzzy action rules, at least one vector value is obtained from the state vector based on the position data associated with the fuzzy action rule; the membership degree between the at least one vector value and the fuzzy sets associated with each fuzzy condition of the fuzzy action rule is calculated, and the obtained membership degrees are multiplied to obtain activation strength, and the obtained activation strength is multiplied by the preset linear speed and the preset angular speed in the fuzzy action rule respectively to obtain the target linear speed and the target angular speed; For each of the mixed action rules, at least one vector value is obtained from the state vector based on the position data associated with the mixed action rule; if the at least one vector value makes each precise condition of the mixed action rule true, the membership degree between the at least one vector value and the fuzzy sets associated with each fuzzy condition of the mixed action rule is calculated, and the obtained membership degrees are multiplied to obtain activation strength, and the obtained activation strength is multiplied with the preset linear speed and the preset angular speed in the mixed action rule respectively to obtain the target linear speed and the target angular speed; The guide linear velocity is obtained by adding all the target linear velocities, and the guide angular velocity is obtained by intersecting all the target angular velocities.

5. The method according to claim 3, characterized in that: The guidance action instruction includes the guidance linear velocity and the guidance angular velocity; the strategy action instruction includes the predicted linear velocity and the predicted angular velocity; the comprehensive action instruction includes the comprehensive linear velocity and the comprehensive angular velocity; The step of inputting the guiding action instruction and the strategy action instruction into the generalizer for fusion to obtain a comprehensive action instruction includes: inputting the guide linear velocity, the guide angular velocity, the predicted linear velocity and the predicted angular velocity into the generalizer; Using the current fusion parameter, weighted processing is performed on the predicted linear velocity and the guided linear velocity to obtain a pending linear velocity, and Gaussian noise is added to the pending linear velocity to obtain the comprehensive linear velocity; The predicted angular velocity and the guided angular velocity are weighted by using the current fusion parameter to obtain a pending angular velocity, and the Gaussian noise is added to the pending angular velocity to obtain the comprehensive angular velocity.

6. The method according to claim 3, characterized in that: The strategic action instruction includes a predicted linear velocity and a predicted angular velocity; The calculation formula of the preset loss function is: in, represents said comprehensive loss, represents the line speed loss, represents the angular velocity loss; is a fixed parameter, indicating the credibility of the knowledge system; represents the value of the state, is the current fusion parameter of the generalizer, represents the predicted linear velocity, represents the integrated linear velocity, represents the predicted angular velocity, represents the integrated angular velocity.

7. A navigation device implemented based on a knowledge-guided map-free navigation model, characterized in that: include: The state acquisition module is used to obtain the environmental state data of the mobile robot at the current moment, the relative position data between the current position and the target point, and the action instructions at the previous moment; The state acquisition module is further used to convert the environmental state data and relative position data at the current moment and the action instruction at the previous moment into a current state vector; A prediction module, used for inputting the current state vector into a trained map-free navigation model to obtain an action instruction at the current moment; The map-free navigation model is obtained by training a pre-built deep deterministic policy gradient algorithm model based on multiple action rules in a preset knowledge system; The control module is used to control the mobile robot to move toward the target point by using the linear velocity and angular velocity in the action instruction at the current moment.

8. The device according to claim 7, characterized in that The map-free navigation model is trained in the following way: Based on the knowledge system, the pre-built deep deterministic policy gradient algorithm model and the generalizer, the intelligent agent is controlled to interact with a preset virtual environment to build an experience database; The experience database is initially empty; When the amount of interaction data in the experience database is greater than When Interaction data; Based on the The interaction data and the multiple action rules in the knowledge system are used to update the parameters of the deep deterministic policy gradient algorithm model and the generalizer; Based on the knowledge system, the updated deep deterministic policy gradient algorithm model and the generalizer, controlling the agent to continue interacting with the virtual environment to expand the experience database; Determining whether the number of interaction rounds between the agent and the virtual environment reaches a preset value; If not, if the sampled data from the experience database is returned, The step of collecting pieces of interaction data until the number of interaction rounds between the agent and the virtual environment reaches the preset value; If so, the agent is controlled to stop interacting with the virtual environment, and the current deep deterministic policy gradient algorithm model is used as the map-free navigation model.

9. A mobile robot, characterized in that: include: A memory and a processor, wherein the memory stores a software program, and when the mobile robot is running, the processor executes the software program to implement the navigation method as described in any one of claims 1-6.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the navigation method according to any one of claims 1 to 6 is implemented.

Images