Intelligent navigation method, device, computer equipment and storage medium

Abstract

The present application proposes an intelligent agent navigation method, including: constructing a path planning model, the output layer of the path planning model includes a macro layer, a decision layer and a micro layer; obtaining observation variables of a first intelligent agent; inputting the observation variables into the path planning model, and the path planning model outputs first navigation information through the macro layer; the first navigation information includes a first target position of the first intelligent agent; inputting the first navigation information into the decision layer, and the decision layer outputs second navigation information based on the first navigation information, and the second navigation information includes the second target position of the first intelligent agent; inputting the second navigation information into the micro layer, and the micro layer outputs third navigation information based on the second navigation information; the third navigation information includes the third target position of the first intelligent agent; navigating the first intelligent agent to run towards the second target position according to the third target position until the first intelligent agent reaches the first target position.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and in particular to an intelligent agent navigation method, apparatus, computer equipment and storage medium. Background Art

[0002] Navigation systems have long been a key research area in artificial intelligence (AI). Intelligent navigation by agents has been applied across various industrial sectors, including autonomous driving, delivery, traffic scheduling, and AI for gaming. The intelligence of navigation systems not only reduces industrial costs but also improves overall operational efficiency. However, as virtual environments become increasingly complex, existing agent-based navigation methods are unable to adapt to the navigation requirements of these complex scenarios.

[0003] Application Contents

[0004] This application proposes an intelligent body navigation method, device, computer equipment and storage medium to solve the technical problem that existing intelligent body navigation technology cannot adapt to complex scenarios.

[0005] In a first aspect, a method for intelligent agent navigation is provided, the method comprising:

[0006] Constructing a path planning model, wherein the output layer of the path planning model includes a macro layer, a decision layer, and a micro layer;

[0007] Get the observation variables of the first agent;

[0008] The observation variables are input into the path planning model, and the path planning model outputs first navigation information through the macro layer; the first navigation information includes a first target location of the first agent; the first target location is used to indicate a first target location of the first agent;

[0009] Inputting the first navigation information into the decision layer, the decision layer outputting second navigation information based on the first navigation information, wherein the second navigation information includes a second target location of the first agent; the second target location is used to indicate a location that the first agent will pass when arriving at the first target location;

[0010] Inputting the second navigation information into the micro layer, the micro layer outputting third navigation information based on the second navigation information; the third navigation information includes a third target location of the first agent; the third target location is used to indicate the direction of the first agent when it departs to the first target location;

[0011] The first agent is navigated toward the second target location according to the third target location until the first agent reaches the first target location.

[0012] In combination with the first aspect, in a possible implementation, the method also includes: obtaining first intervention information input by the user; inputting the first intervention information into the macro layer, the macro layer intervening in the first navigation information based on the first intervention information, and outputting fourth navigation information, the fourth navigation information including the fourth target position of the first intelligent agent; inputting the fourth navigation information into the decision layer, the decision layer outputting fifth navigation information based on the fourth navigation information, the fifth navigation information including the fifth target position of the first intelligent agent; the micro layer outputting sixth navigation information based on the fifth navigation information; the sixth navigation information including the sixth target position of the first intelligent agent; navigating the first intelligent agent to the fifth target position according to the sixth target position until the first intelligent agent reaches the fourth target position.

[0013] In combination with the first aspect, in a possible implementation, the method also includes: obtaining second intervention information input by the user; inputting the second intervention information into the decision layer, the decision layer intervening in the second navigation information based on the second intervention information, and outputting seventh navigation information, the seventh navigation information including the seventh target position of the first intelligent agent; the micro layer outputting eighth navigation information based on the seventh navigation information; the eighth navigation information including the eighth target position of the first intelligent agent; navigating the first intelligent agent to run towards the seventh target position according to the eighth target position until the first intelligent agent reaches the first target position.

[0014] In combination with the first aspect, in a possible implementation, the method also includes: obtaining third intervention information input by the user; inputting the third intervention information into the micro layer, and the micro layer intervening in the third navigation information based on the third intervention information, and outputting ninth navigation information; the ninth navigation information includes the ninth target position of the first intelligent body; navigating the first intelligent body to the second target position according to the ninth target position until the first intelligent body reaches the first target position.

[0015] In combination with the first aspect, in a possible implementation, the path planning model also includes a feature extraction layer and a coding mapping layer; the feature extraction layer and the output layer are connected through the coding mapping layer, and the observation variables are input into the path planning model, and the path planning model outputs the first navigation information through the macro layer, including: inputting the observation variables into the path planning model, performing feature extraction on the observation variables through the feature extraction layer to obtain feature data corresponding to the observation variables; performing coding mapping processing on the feature data through the coding mapping layer to obtain a feature vector corresponding to the observation variable; and the macro layer outputs the first navigation information based on the feature vector.

[0016] In combination with the first aspect, in a possible implementation, the macro layer is not time-sensitive, and the decision layer and the micro layer are time-sensitive.

[0017] In combination with the first aspect, in a possible implementation method, the construction of the path planning model includes: obtaining sample observation variables of the sample agent and offline navigation trajectory data of the sample agent; determining the first label, second label and third label of the sample agent based on the offline navigation trajectory data; the first label is used to indicate the sample target location of the sample agent, the second label is used to indicate the passing location of the sample agent when arriving at the sample target location, and the third label is used to indicate the direction of the sample agent when arriving at the sample target location; the sample agent, the first label, the second label and the third label are input into a deep learning model for training to obtain a path planning model.

[0018] In a second aspect, an intelligent navigation device is provided, the device comprising:

[0019] A model building module is used to build a path planning model, wherein the output layer of the path planning model includes a macro layer, a decision layer, and a micro layer;

[0020] A variable acquisition module, used to obtain the observation variables of the first agent;

[0021] a first navigation module, configured to input the observed variables into the path planning model, wherein the path planning model outputs first navigation information via the macro layer; the first navigation information includes a first target location of the first agent; the first target location is used to indicate a first target location of the first agent;

[0022] a second navigation module, configured to input the first navigation information into the decision layer, wherein the decision layer outputs second navigation information based on the first navigation information, wherein the second navigation information includes a second target location of the first agent; the second target location indicates a location that the first agent will pass when arriving at the first target location;

[0023] a third navigation module configured to output third navigation information at the micro-level based on the second navigation information; the third navigation information including a third target location of the first agent; the third target location indicating the direction of the first agent when departing to the first target location;

[0024] A control module is used to navigate the first intelligent body to the second target position according to the third target position until the first intelligent body reaches the first target position.

[0025] In combination with the second aspect, in a possible implementation, the device also includes a fourth navigation module, which is used to obtain first intervention information input by the user; input the first intervention information into the macro layer, and the macro layer intervenes in the first navigation information based on the first intervention information, and outputs fourth navigation information, and the fourth navigation information includes the fourth target position of the first intelligent agent; input the fourth navigation information into the decision layer, and the decision layer outputs fifth navigation information based on the fourth navigation information, and the fifth navigation information includes the fifth target position of the first intelligent agent; the micro layer outputs sixth navigation information based on the fifth navigation information; the sixth navigation information includes the sixth target position of the first intelligent agent; navigate the first intelligent agent to the fifth target position according to the sixth target position until the first intelligent agent reaches the fourth target position.

[0026] In combination with the second aspect, in a possible implementation, the device also includes a fifth navigation module, which is used to obtain second intervention information input by the user; input the second intervention information into the decision layer, and the decision layer intervenes in the second navigation information based on the second intervention information, and outputs seventh navigation information, wherein the seventh navigation information includes the seventh target position of the first intelligent agent; the micro layer outputs eighth navigation information based on the seventh navigation information; the eighth navigation information includes the eighth target position of the first intelligent agent; and navigates the first intelligent agent to run towards the seventh target position according to the eighth target position until the first intelligent agent reaches the first target position.

[0027] In combination with the second aspect, in a possible implementation, the device also includes a sixth navigation module, which is used to obtain third intervention information input by the user; input the third intervention information into the micro layer, and the micro layer intervenes in the third navigation information based on the third intervention information, and outputs ninth navigation information; the ninth navigation information includes the ninth target position of the first intelligent body; and navigate the first intelligent body to the second target position according to the ninth target position until the first intelligent body reaches the first target position.

[0028] In combination with the second aspect, in a possible implementation, the path planning model also includes a feature extraction layer and a coding mapping layer; the feature extraction layer and the output layer are connected through the coding mapping layer, and the first navigation module is specifically used to input the observation variable into the path planning model, perform feature extraction on the observation variable through the feature extraction layer, and obtain feature data corresponding to the observation variable; perform coding mapping processing on the feature data through the coding mapping layer to obtain a feature vector corresponding to the observation variable; the macro layer outputs the first navigation information based on the feature vector.

[0029] In combination with the second aspect, in a possible implementation, the macro layer is not time-sensitive, while the decision layer and the micro layer are time-sensitive.

[0030] In combination with the second aspect, in one possible implementation, the model building module is specifically used to obtain sample observation variables of the sample agent and offline navigation trajectory data of the sample agent; determine the first label, second label and third label of the sample agent based on the offline navigation trajectory data; the first label is used to indicate the sample target location of the sample agent, the second label is used to indicate the passing location of the sample agent when it arrives at the sample target location, and the third label is used to indicate the direction of the sample agent when it arrives at the sample target location; the sample agent, the first label, the second label and the third label are input into the deep learning model for training to obtain a path planning model.

[0031] The present application can achieve the following beneficial effects: By constructing a path planning model including a macro layer, a decision layer and a micro layer, and using the output of the macro layer as the input of the decision layer and the output of the decision layer as the input of the micro layer, the present application can adapt to the navigation needs of complex scenes, solve the problem of complex scene decision-making in navigation problems, and improve the accuracy of intelligent body navigation. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 A flowchart of an intelligent agent navigation method provided in an embodiment of the present application;

[0033] Figure 2 A schematic diagram of the structure of a path planning model provided in an embodiment of the present application;

[0034] Figure 3 A schematic diagram of a process for constructing a path planning model provided in an embodiment of the present application;

[0035] Figure 4 A flowchart of an intelligent agent navigation method provided in an embodiment of the present application;

[0036] Figure 5 A flowchart of an intelligent agent navigation method provided in an embodiment of the present application;

[0037] Figure 6 A flowchart of an intelligent agent navigation method provided in an embodiment of the present application;

[0038] Figure 7 A schematic diagram of the structure of an intelligent navigation device provided in an embodiment of the present application;

[0039] Figure 8 It is a structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0040] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0041] The technical solution of this application can be applied to various scenarios of intelligent agent navigation. Specifically, the technical solution of this application can be used to navigate an intelligent agent based on its observed variables in intelligent agent navigation scenarios. In intelligent agent navigation scenarios, a pre-trained path planning model is used to identify the observed variables of the intelligent agent in the scenario, thereby determining the target location of the intelligent agent's navigation, the locations that will be passed during the navigation process, and the navigation direction, thereby navigating the intelligent agent.

[0042] Among them, the intelligent agent refers to a virtual object in a virtual environment, and a virtual object is a digital object that can be operated by a computer device. The virtual object can be a three-dimensional object or a two-dimensional object, etc. It can be a virtual character, a virtual animal, etc. Specifically, the virtual environment can be the game environment of a computer game, and the intelligent agent can be a game character in the game environment. For example, the virtual environment can be the game environment in the computer game Game for Peace, and the intelligent agent can be various game characters in Game for Peace. It should be noted that the intelligent agent in this application is not limited to game characters, but can also be a virtual object in other virtual environments; similarly, the virtual environment in this application is not limited to the game environment, but can also be other virtual environments.

[0043] Among them, observation variables are used to express the virtual object's perception of the virtual environment, other virtual objects, etc.

[0044] In one embodiment, the present application proposes an agent navigation method, such as Figure 1 As shown, Figure 1 A flowchart of an intelligent agent navigation method provided in an embodiment of the present application, the method comprising:

[0045] Step 101: construct a path planning model, wherein the output layer of the path planning model includes a macro layer, a decision layer, and a micro layer.

[0046] Among them, when navigating the intelligent agent, it is necessary to first build a path planning model for navigation. After the path planning model is built, the intelligent agent can be navigated.

[0047] In one embodiment, Figure 2 As shown, Figure 2 This is a schematic diagram of the structure of a path planning model provided in an embodiment of the present application. The path planning model includes a feature extraction layer, a coding and mapping layer, and an output layer. The feature extraction layer is used to extract features from the input model data to obtain feature parameters. The coding and mapping layer is used to code and map the feature parameters into feature vectors of preset dimensions. The output layer is used to output navigation information based on the feature vectors.

[0048] The output layer of the path planning model includes a macro layer, a decision layer and a micro layer. The macro layer is not time-sensitive, while the decision layer and the micro layer are time-sensitive.

[0049] Specifically, the macro layer is used to output the destination or end point of this navigation, that is, our navigation goal, so that we can finally reach the place. In a complete navigation, the position of this point should be relatively unchanged, and this macro layer does not need to consider time.

[0050] Specifically, the decision layer is used to decompose the navigation target output by the macro layer, and output the places to pass through to reach the navigation target; to complete the navigation target, where should my next small goal be to reach? This small goal has a strong timeliness, such as how long it will take to reach the place with a high probability. This is the process of decomposing the goals of the macro layer; it is understandable that as the navigation progresses, this small goal will change.

[0051] Specifically, the micro-layer decomposes the small goals output by the decision-making layer and outputs methods for achieving them, such as walking directions. The micro-layer's work can be understood as determining what needs to be done to achieve this small goal. Because there are many paths to achieving small goals, the micro-operations we perform in different states vary. However, a micro-operation only represents the current point in time. After performing one operation, we immediately perform the next micro-operation. Many micro-operations constitute a small goal, and many small goals constitute the navigation destination we need to reach. In other words, the micro-layer has a stronger sense of timeliness.

[0052] In the path planning model, we also use the output of the macro layer as the input of the decision layer, and the output of the decision layer as the input of the micro layer. This can better express and represent information at different levels, achieve the goal of dynamically adjusting the path planning model, and thus adapt to the navigation needs of complex scenarios.

[0053] In one embodiment, Figure 3 As shown, Figure 3 A schematic diagram of a process for constructing a path planning model provided in an embodiment of the present application, wherein constructing the path planning model includes:

[0054] Step 1011: Obtain sample observation variables of the sample agent and offline navigation trajectory data of the sample agent.

[0055] The sample agent refers to the agent trained as a sample memory model. The sample observation variable is used to express the sample agent's perception of the virtual environment and other virtual objects.

[0056] Among them, a sample data set is obtained; the sample data set includes multiple samples, one sample corresponds to a sample agent, and the sample includes the sample observation variables of the sample agent and the offline navigation trajectory data of the sample agent, that is, the sample observation variables of a sample agent and the offline navigation trajectory data of the sample agent constitute a sample; by training the deep learning model with the obtained sample data set, a path planning model can be obtained.

[0057] Step 1012, determine the first label, second label and third label of the sample agent based on the offline navigation trajectory data; the first label is used to indicate the sample target location of the sample agent, the second label is used to indicate the passing location of the sample agent when arriving at the sample target location, and the third label is used to indicate the direction of the sample agent when arriving at the sample target location.

[0058] Among them, in a sample, the offline navigation trajectory data in the sample is decomposed to obtain the navigation target of the sample intelligent agent corresponding to the sample, that is, the navigation destination and terminal. The decomposed navigation target is used as the sample target location, and the sample target location is used as the first label of the sample intelligent agent.

[0059] In the above sample, the offline navigation trajectory data in the sample is decomposed to obtain the places through which the sample agent navigates to the navigation target, and the places through which the sample agent reaches the navigation target (sample target location) are used as the second label of the sample agent.

[0060] In the above sample, the offline navigation trajectory data in the sample is decomposed to obtain the walking direction of the sample agent at each moment in the process from the starting point to the navigation target, that is, the walking direction of the sample agent, and the walking direction of the sample agent is used as the third label of the sample agent.

[0061] Specifically, in a sample, there are sample observation variables of the sample agent, and a first label, a second label and a third label obtained based on the offline navigation trajectory data of the sample agent; the sample observation variables have a corresponding relationship with the first label, the second label and the third label.

[0062] Step 1013: Input the sample agent, the first label, the second label, and the third label into a deep learning model for training to obtain a path planning model.

[0063] Among them, the sample composed of the sample observation variables of the sample intelligent agent, the first label, the second label and the third label of the sample intelligent agent is input into the deep learning model for training, so that the deep learning model can learn the trajectory characteristics of the offline navigation trajectory data, so that the path planning output by the trained deep learning model based on each sample can be infinitely close to the first label, the second label and the third label of the sample, that is, the deep learning model has the ability of path planning. The deep learning model with path planning ability is a path planning model.

[0064] Step 102: Obtain the observation variables of the first agent.

[0065] Among them, the observation variables are used to express the virtual object's perception of the virtual environment, other virtual objects, etc. In some embodiments, when the first agent is an agent in the game, the computer device can obtain the observation variables of the first agent from the game kernel. The observation variables of the first agent include the scene parameters of the first agent, the scene parameters of the second agent, and the scene parameters of the virtual items. The first agent refers to the agent that needs to navigate in this navigation. The second agent refers to the agent that does not need to navigate in this navigation, that is, other agents in the virtual environment except the first agent. Virtual items refer to items in the virtual environment such as doors, windows, and rivers.

[0066] Specifically, the scene parameters of the second intelligent agent include the operating status data of the second intelligent agent and the virtual character data of the second intelligent agent; the operating status data of the second intelligent agent may include data such as the operating speed, operating direction, and operating destination of the second intelligent agent, and the virtual character data of the second intelligent agent may include data such as the relationship data between the second intelligent agent and the first intelligent agent. For example, the relationship data may be like, dislike, or an enemy camp or an ally camp.

[0067] Specifically, the scene parameters of virtual objects include the connectivity of virtual objects, for example: ladders allow the agent to climb, rivers do not allow the agent to pass through, and the agent needs to use certain tools to pass through, etc.

[0068] Step 103: Input the observed variables into the path planning model, and the path planning model outputs first navigation information through the macro layer; the first navigation information includes the first target position of the first agent; the first target position is used to indicate the first target location of the first agent.

[0069] The first navigation information is used to indicate navigation information from the starting point to the destination of this navigation, including the location information of the destination or end point of this navigation, the method of reaching the destination, etc. For example, the method of reaching the destination may be walking, cycling, driving, etc.

[0070] Among them, the first target position is used to indicate the destination or end point of this navigation, and the first target location is used to indicate the destination or end point of the first intelligent agent in this navigation.

[0071] In one embodiment, the feature extraction layer and the output layer are connected through the coding mapping layer, the observation variables are input into the path planning model, and the path planning model outputs the first navigation information through the macro layer, including: inputting the observation variables into the path planning model, performing feature extraction on the observation variables through the feature extraction layer to obtain feature data corresponding to the observation variables; performing coding mapping processing on the feature data through the coding mapping layer to obtain a feature vector corresponding to the observation variable; and the macro layer outputs the first navigation information based on the feature vector.

[0072] Among them, the path planning model includes a feature extraction layer, a coding mapping layer and an output layer; after the observation variables of the first intelligent agent are input into the path planning model, the feature extraction layer of the path planning model extracts features of the observation variables to obtain feature parameters, and then the coding mapping layer of the path planning model encodes and maps the feature parameters, and maps the feature parameters into feature vectors of preset dimensions; finally, the output layer of the path planning model outputs navigation information according to the feature vector. Specifically, the macro layer of the path planning model outputs the first navigation information based on the feature vector.

[0073] Specifically, the preset dimension is determined by the number of observed variables. For example, the preset dimension may be the same as the number of observed variables.

[0074] Step 104: input the first navigation information to the decision layer, and the decision layer outputs second navigation information based on the first navigation information, wherein the second navigation information includes a second target location of the first agent; the second target location is used to indicate a passing location of the first agent when arriving at the first target location.

[0075] The second navigation information is used to indicate navigation information for reaching any point between the starting point and the destination, including location information of any point between the starting point and the destination, a method of reaching the destination, etc. For example, the method of reaching a point between the starting point and the destination may be walking.

[0076] The second target location is used to indicate any location between the starting point and the destination of this navigation, that is, the location passed by this navigation. The first target location is used to indicate the location passed by the first agent in this navigation.

[0077] In one embodiment, after the observation variables of the first agent are input into the path planning model, the decision layer of the path planning model will navigate based on parameters such as the scene parameters of the first agent, the scene parameters of the second agent, and the scene parameters of the virtual object.

[0078] Specifically, the decision layer can determine whether the first and second agents will meet during navigation based on the second agent's virtual character data and the second agent's operating status data. For example, the decision layer can determine whether the first agent will meet the second agent when navigating to the destination based on the second agent's operating status data. If it is determined that the first agent will meet the second agent when navigating to the destination, the relationship between the first and second agents can be determined based on the second agent's virtual character data. If it is determined that the relationship between the first and second agents is a favorable or allied relationship, the second navigation information will not be updated. If it is determined that the relationship between the first and second agents is a unfavorable or hostile relationship, the second navigation information will be updated to obtain updated second navigation information, and the updated second navigation information will be output.

[0079] In one embodiment, after the observation variables of the first agent are input into the path planning model, the decision layer of the path planning model will navigate according to the scene parameters of the virtual object.

[0080] Specifically, the connectivity of the virtual item is determined based on the scene parameters of the virtual item. If the connectivity of the virtual item is acceptable, the second navigation information is not updated. If the connectivity of the virtual item is unacceptable, the second navigation information is updated and the updated second navigation information is output. For example, if it is determined that navigation to the destination will pass through a river, and the connectivity of the river is determined to be unacceptable, the second navigation information is updated so that the updated second navigation information does not include the river.

[0081] Navigation through the scene parameters of the second intelligent agent and the scene parameters of the virtual objects can improve the authenticity of navigation path planning, thereby solving the technical problem of poor authenticity of path planning.

[0082] Step 105: input the second navigation information into the micro layer, and the micro layer outputs third navigation information based on the second navigation information; the third navigation information includes the third target location of the first intelligent agent; the third target location is used to indicate the direction of the first intelligent agent when it departs for the first target location.

[0083] The third navigation information indicates the navigation information for this navigation from the starting point, including the location of the next destination after departure. This location information is used to determine the direction from departure to destination. For example, if the next destination is northwest of the starting point, then the first agent will depart in a northwest direction to reach the destination.

[0084] Among them, the third target position is used to indicate the next position of this navigation after starting from the starting point, that is, it is used to indicate the direction of the first intelligent agent to the destination.

[0085] It is understandable that, in the process of the first agent setting out to the destination, any point on the road can be regarded as a starting point, and navigation to the destination direction is required at each starting point.

[0086] Step 106: Navigate the first agent toward the second target location according to the third target location until the first agent reaches the first target location.

[0087] After determining the first, second, and third target locations for this navigation process—that is, the destination, the locations to be passed along the way, and the direction to the destination—the first agent can be navigated. Specifically, the agent is controlled to move in the direction of the destination, reach the locations to be passed along, and then depart from the locations to reach the destination.

[0088] The present application provides an intelligent agent navigation method, the method comprising: constructing a path planning model, the output layer of the path planning model comprising a macro layer, a decision layer, and a micro layer; obtaining observation variables of a first intelligent agent; inputting the observation variables into the path planning model, the path planning model outputting first navigation information through the macro layer; the first navigation information comprising a first target position of the first intelligent agent; the first target position being used to indicate a first target location of the first intelligent agent; inputting the first navigation information into the decision layer, the decision layer outputting second navigation information based on the first navigation information, the second navigation information comprising a second target position of the first intelligent agent; the second target position being used to indicate a location passed by the first intelligent agent when arriving at the first target location; inputting the second navigation information into the micro layer, the micro layer outputting third navigation information based on the second navigation information; the third navigation information comprising a third target position of the first intelligent agent; the third target position being used to indicate the direction of the first intelligent agent when departing for the first target location; and navigating the first intelligent agent toward the second target position according to the third target position until the first intelligent agent arrives at the first target location. This application constructs a path planning model including a macro layer, a decision layer and a micro layer, and uses the output of the macro layer as the input of the decision layer and the output of the decision layer as the input of the micro layer. It can adapt to the navigation needs of complex scenarios, solve the problem of complex scenario decision-making in navigation problems, and improve the accuracy of intelligent body navigation.

[0089] In one embodiment, Figure 4 As shown, Figure 4 A flow chart of an intelligent agent navigation method provided in an embodiment of the present application. Figure 1-Figure 3The agent navigation method adds a first manual intervention, which is made at the macro level. The method further includes:

[0090] Step 401: Acquire first intervention information input by a user.

[0091] The first intervention information is a human influence on the macro level, such as a change in destination or the method of travel to the destination. It should be noted that since the output of the decision-making layer is influenced by the macro level, and the output of the micro level is influenced by the decision-making layer, when the macro level is interfered with, the output of both the decision-making and micro levels will also be affected.

[0092] Step 402: input the first intervention information to the macro layer, and the macro layer intervenes in the first navigation information based on the first intervention information and outputs fourth navigation information, where the fourth navigation information includes a fourth target location of the first agent.

[0093] Step 403: input the fourth navigation information to the decision layer, and the decision layer outputs fifth navigation information based on the fourth navigation information, where the fifth navigation information includes a fifth target location of the first agent.

[0094] Step 404: The micro layer outputs sixth navigation information based on the fifth navigation information; the sixth navigation information includes the sixth target position of the first agent.

[0095] Step 405: Navigate the first agent toward the fifth target location according to the sixth target location until the first agent reaches the fourth target location.

[0096] Specifically, taking the example of manually changing the navigation destination, the first intervention information is the new destination. For example, the destination before the human intervention is Shanghai, and the new destination after the human intervention is Beijing. First, after Beijing is input into the macro layer, the macro layer will generate the fourth navigation information and use the fourth navigation information to replace the first navigation information as the output, that is, the navigation information for navigating to Beijing replaces the navigation information for navigating to Shanghai; the fourth navigation information refers to the navigation information for navigating to the new destination, that is, the navigation information for navigating to Beijing; the fourth target location refers to the new destination, that is, Beijing. Secondly, the fourth navigation information is input into the decision layer, that is, the navigation information for navigating to Beijing is input into the decision layer, the decision layer will generate the fifth navigation information and use the fifth navigation information to replace the second navigation information as the output; the fifth navigation information refers to the navigation information for navigating to any point between the starting point and Beijing, and the fifth target location refers to any point between the starting point and Beijing. Next, the fifth navigation information is input into the micro-layer. This information, which indicates navigation to any point between the starting point and Beijing, is then fed into the micro-layer. The micro-layer generates the sixth navigation information and uses it as output, replacing the third navigation information. The sixth navigation information refers to the navigation information from the starting point, with the sixth target location indicating the direction to Beijing. Finally, the navigation agent follows the direction to Beijing to reach any point between the starting point and Beijing, and then reaches Beijing from any point between the starting point and Beijing.

[0097] In this embodiment, by inputting manual intervention information, the path planning model is no longer completely black, but can be changed according to actual needs, thereby improving the convenience of navigation.

[0098] In one embodiment, Figure 5 As shown, Figure 5 A flow chart of an intelligent agent navigation method provided in an embodiment of the present application. Figure 1-Figure 3 The agent navigation method adds a second manual intervention, which is made to the decision-making layer. The method also includes:

[0099] Step 501: Acquire second intervention information input by a user.

[0100] The second intervention information refers to human influence on the decision-making layer, such as changes to route locations. It should be noted that since the output of the decision-making layer does not serve as input to the macro layer, intervention at the decision-making layer will not affect the output of the macro layer. However, the output of the decision-making layer serves as input to the micro layer, so intervention at the decision-making layer will also affect the output of the micro layer.

[0101] Step 502: input the second intervention information to the decision layer, and the decision layer intervenes in the second navigation information based on the second intervention information, and outputs seventh navigation information, where the seventh navigation information includes the seventh target position of the first agent.

[0102] Step 503: The micro layer outputs eighth navigation information based on the seventh navigation information; the eighth navigation information includes the eighth target position of the first agent.

[0103] Step 504: Navigate the first agent toward the seventh target location according to the eighth target location until the first agent reaches the first target location.

[0104] Specifically, let's take the example of manually changing the navigation route. For example, the route before the intervention was Suzhou, and the route after the intervention was Yangzhou. First, after Suzhou is input into the decision layer, the decision layer generates the seventh navigation information and uses it to replace the second navigation information as output. That is, the navigation information for navigating to Yangzhou replaces the navigation information for navigating to Suzhou. The seventh navigation information refers to the navigation information for navigating to the new route, that is, the navigation information for navigating to Yangzhou. The seventh target location refers to the new route, that is, Yangzhou. Second, the seventh navigation information is input into the micro layer. The micro layer generates the eighth navigation information and uses it to replace the third navigation information as output. The eighth navigation information indicates the navigation information when departing from the starting point, and the eighth target location indicates the direction to Yangzhou, that is, the direction to the navigation destination. Finally, the navigation agent arrives in Yangzhou according to the direction to the destination, and then reaches the destination from Yangzhou.

[0105] In this embodiment, by inputting manual intervention information, the path planning model is no longer completely black, but can be changed according to actual needs, thereby improving the convenience of navigation.

[0106] In one embodiment, Figure 6 As shown, Figure 6 A flow chart of an intelligent agent navigation method provided in an embodiment of the present application. Figure 1-Figure 3 The agent navigation method adds a third manual intervention, which is made at the micro level. The method also includes:

[0107] Step 601: Acquire third intervention information input by the user.

[0108] The third intervention information is the human influence on the micro layer, such as a change in navigation direction. It should be noted that since the output of the micro layer does not serve as input to the macro layer and the decision layer, the output of the macro layer and the decision layer will not be affected when the micro layer is interfered with.

[0109] Step 602: input the third intervention information into the micro layer, and the micro layer intervenes in the third navigation information based on the third intervention information, and outputs ninth navigation information; the ninth navigation information includes the ninth target position of the first agent.

[0110] Step 603: Navigate the first agent toward the second target position according to the ninth target position until the first agent reaches the first target position.

[0111] Specifically, let's take the example of manually changing the navigation direction. For example, the navigation direction before the intervention was east, and the navigation direction after the intervention was south. First, after the south direction is input into the micro layer, the micro layer will generate the ninth navigation information and use the ninth navigation information to replace the third navigation information as the output; the ninth navigation information is the navigation information when starting from the starting point, and the ninth target position is used to indicate going south. It can be understood that the direction is indicated by the geographical relationship between the ninth target position and the starting point. Finally, the navigation agent presses the south button to reach the destination.

[0112] This application constructs a path planning model that includes a macro layer, a decision layer, and a micro layer. The output of the macro layer is used as the input of the decision layer, and the output of the decision layer is used as the input of the micro layer. This can adapt to the navigation needs of complex scenarios, solve the problem of complex scenario decision-making in navigation problems, and improve the accuracy of intelligent navigation. By inputting human intervention information, the path planning model is no longer completely black and can be changed according to actual needs, improving the convenience of navigation.

[0113] The above describes the method of the present application. In order to better implement the method of the present application, the device of the present application will be described below.

[0114] In one embodiment, Figure 7 As shown, Figure 7 This is a schematic diagram of the structure of an intelligent navigation device provided in an embodiment of the present application, the device comprising:

[0115] A model building module 701 is used to build a path planning model, wherein the output layer of the path planning model includes a macro layer, a decision layer, and a micro layer;

[0116] The variable acquisition module 702 is used to obtain the observation variables of the first agent;

[0117] A first navigation module 703 is configured to input the observed variables into the path planning model, and the path planning model outputs first navigation information through the macro layer; the first navigation information includes a first target location of the first agent; the first target location is used to indicate a first target location of the first agent;

[0118] A second navigation module 704 is configured to input the first navigation information into the decision layer, and the decision layer outputs second navigation information based on the first navigation information, wherein the second navigation information includes a second target location of the first agent; the second target location indicates a location that the first agent will pass when arriving at the first target location;

[0119] A third navigation module 705 is configured to output third navigation information based on the second navigation information at the micro-level; the third navigation information includes a third target location of the first agent; the third target location indicates the direction of the first agent when it departs for the first target location;

[0120] The control module 706 is configured to navigate the first agent toward the second target location according to the third target location until the first agent reaches the first target location.

[0121] like Figure 8 As shown in FIG. 1 , in one embodiment, it is an internal structure diagram of a computer device. The computer device may be an intelligent body navigation device, or a terminal or server connected to an intelligent body navigation device. Figure 8 As shown, the computer device includes a processor, a memory, and a network interface connected via a system bus. The memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium of the computer device stores an operating system and may also store a computer program. When the computer program is executed by the processor, the processor may implement an intelligent body navigation method. The internal memory may also store a computer program. When the computer program is executed by the processor, the processor may implement an intelligent body navigation method. The network interface is used to communicate with an external device. Those skilled in the art will understand that Figure 8 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0122] In one embodiment, the agent navigation method provided by the present application can be implemented in the form of a computer program. The computer program can be used in Figure 8The computer device is shown as running. The memory of the computer device can store various program templates that constitute the intelligent navigation device, such as the model construction module 701, the variable acquisition module 702, the first navigation module 703, the second navigation module 704, the third navigation module 705, and the control module 706.

[0123] A computer device includes a memory and a processor, wherein the memory stores a computer program. When the computer program is executed by the processor, the processor performs the following steps: constructing a path planning model, wherein the output layer of the path planning model includes a macro layer, a decision layer, and a micro layer; obtaining observation variables of a first agent; inputting the observation variables into the path planning model, wherein the path planning model outputs first navigation information through the macro layer; wherein the first navigation information includes a first target location of the first agent; wherein the first target location indicates a first target location of the first agent; inputting the first navigation information into the decision layer, wherein the decision layer outputs second navigation information based on the first navigation information; wherein the second navigation information includes a second target location of the first agent; wherein the second target location indicates a location passed by the first agent when arriving at the first target location; inputting the second navigation information into the micro layer, wherein the micro layer outputs third navigation information based on the second navigation information; wherein the third navigation information includes a third target location of the first agent; wherein the third target location indicates a direction of the first agent when departing for the first target location; and navigating the first agent toward the second target location according to the third target location until the first agent arrives at the first target location.

[0124] In one embodiment, when the computer program is executed by the processor, the processor further performs the following steps: obtaining first intervention information input by the user; inputting the first intervention information into the macro layer, the macro layer intervenes in the first navigation information based on the first intervention information, and outputs fourth navigation information, the fourth navigation information including the fourth target position of the first agent; inputting the fourth navigation information into the decision layer, the decision layer outputs fifth navigation information based on the fourth navigation information, the fifth navigation information including the fifth target position of the first agent; the micro layer outputs sixth navigation information based on the fifth navigation information; the sixth navigation information includes the sixth target position of the first agent; navigating the first agent to the fifth target position according to the sixth target position until the first agent reaches the fourth target position.

[0125] In one embodiment, when the computer program is executed by the processor, the processor further performs the following steps: obtaining second intervention information input by the user; inputting the second intervention information into the decision layer, the decision layer intervening in the second navigation information based on the second intervention information, and outputting seventh navigation information, wherein the seventh navigation information includes the seventh target position of the first agent; the micro layer outputs eighth navigation information based on the seventh navigation information; the eighth navigation information includes the eighth target position of the first agent; and navigating the first agent to run towards the seventh target position according to the eighth target position until the first agent reaches the first target position.

[0126] In one embodiment, when the computer program is executed by the processor, the processor further performs the following steps: obtaining third intervention information input by the user; inputting the third intervention information into the micro layer, and the micro layer intervenes in the third navigation information based on the third intervention information, and outputs ninth navigation information; the ninth navigation information includes the ninth target position of the first intelligent agent; and navigating the first intelligent agent to the second target position according to the ninth target position until the first intelligent agent reaches the first target position.

[0127] In combination with the first aspect, in a possible implementation, the path planning model also includes a feature extraction layer and a coding mapping layer; the feature extraction layer and the output layer are connected through the coding mapping layer, and the observation variables are input into the path planning model, and the path planning model outputs the first navigation information through the macro layer, including: inputting the observation variables into the path planning model, performing feature extraction on the observation variables through the feature extraction layer to obtain feature data corresponding to the observation variables; performing coding mapping processing on the feature data through the coding mapping layer to obtain a feature vector corresponding to the observation variable; and the macro layer outputs the first navigation information based on the feature vector.

[0128] In combination with the first aspect, in a possible implementation, the macro layer is not time-sensitive, and the decision layer and the micro layer are time-sensitive.

[0129] In combination with the first aspect, in a possible implementation method, the construction of the path planning model includes: obtaining sample observation variables of the sample agent and offline navigation trajectory data of the sample agent; determining the first label, second label and third label of the sample agent based on the offline navigation trajectory data; the first label is used to indicate the sample target location of the sample agent, the second label is used to indicate the passing location of the sample agent when arriving at the sample target location, and the third label is used to indicate the direction of the sample agent when arriving at the sample target location; the sample agent, the first label, the second label and the third label are input into a deep learning model for training to obtain a path planning model.

[0130] A computer-readable storage medium stores a computer program. When executed by a processor, the computer program causes the processor to perform the following steps: constructing a path planning model, wherein the output layer of the path planning model includes a macro layer, a decision layer, and a micro layer; obtaining observation variables of a first agent; inputting the observation variables into the path planning model, wherein the path planning model outputs first navigation information through the macro layer; the first navigation information includes a first target location of the first agent; the first target location is used to indicate a first target location of the first agent; inputting the first navigation information into the decision layer, wherein the decision layer outputs second navigation information based on the first navigation information, wherein the second navigation information includes a second target location of the first agent; the second target location is used to indicate a location passed by the first agent when arriving at the first target location; inputting the second navigation information into the micro layer, wherein the micro layer outputs third navigation information based on the second navigation information; the third navigation information includes a third target location of the first agent; the third target location is used to indicate the direction of the first agent when departing for the first target location; and navigating the first agent toward the second target location based on the third target location until the first agent arrives at the first target location.

[0131] In one embodiment, when the computer program is executed by the processor, the processor further performs the following steps: obtaining first intervention information input by the user; inputting the first intervention information into the macro layer, the macro layer intervenes in the first navigation information based on the first intervention information, and outputs fourth navigation information, the fourth navigation information including the fourth target position of the first agent; inputting the fourth navigation information into the decision layer, the decision layer outputs fifth navigation information based on the fourth navigation information, the fifth navigation information including the fifth target position of the first agent; the micro layer outputs sixth navigation information based on the fifth navigation information; the sixth navigation information includes the sixth target position of the first agent; navigating the first agent to the fifth target position according to the sixth target position until the first agent reaches the fourth target position.

[0132] In one embodiment, when the computer program is executed by the processor, the processor further performs the following steps: obtaining second intervention information input by the user; inputting the second intervention information into the decision layer, the decision layer intervening in the second navigation information based on the second intervention information, and outputting seventh navigation information, wherein the seventh navigation information includes the seventh target position of the first agent; the micro layer outputs eighth navigation information based on the seventh navigation information; the eighth navigation information includes the eighth target position of the first agent; and navigating the first agent to run towards the seventh target position according to the eighth target position until the first agent reaches the first target position.

[0133] In one embodiment, when the computer program is executed by the processor, the processor further performs the following steps: obtaining third intervention information input by the user; inputting the third intervention information into the micro layer, and the micro layer intervenes in the third navigation information based on the third intervention information, and outputs ninth navigation information; the ninth navigation information includes the ninth target position of the first intelligent agent; and navigating the first intelligent agent to the second target position according to the ninth target position until the first intelligent agent reaches the first target position.

[0134] In combination with the first aspect, in a possible implementation, the path planning model also includes a feature extraction layer and a coding mapping layer; the feature extraction layer and the output layer are connected through the coding mapping layer, and the observation variables are input into the path planning model, and the path planning model outputs the first navigation information through the macro layer, including: inputting the observation variables into the path planning model, performing feature extraction on the observation variables through the feature extraction layer to obtain feature data corresponding to the observation variables; performing coding mapping processing on the feature data through the coding mapping layer to obtain a feature vector corresponding to the observation variable; and the macro layer outputs the first navigation information based on the feature vector.

[0135] In combination with the first aspect, in a possible implementation, the macro layer is not time-sensitive, and the decision layer and the micro layer are time-sensitive.

[0136] In combination with the first aspect, in a possible implementation method, the construction of the path planning model includes: obtaining sample observation variables of the sample agent and offline navigation trajectory data of the sample agent; determining the first label, second label and third label of the sample agent based on the offline navigation trajectory data; the first label is used to indicate the sample target location of the sample agent, the second label is used to indicate the passing location of the sample agent when arriving at the sample target location, and the third label is used to indicate the direction of the sample agent when arriving at the sample target location; the sample agent, the first label, the second label and the third label are input into a deep learning model for training to obtain a path planning model.

[0137] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing related hardware through a computer program. The program can be stored in a computer-readable storage medium, and when executed, the program can include the processes in the above-described method embodiments. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0138] The above disclosure is only a preferred embodiment of the present application, and certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. An intelligent agent navigation method, characterized in that: The method comprises: Constructing a path planning model, wherein the output layer of the path planning model includes a macro layer, a decision layer, and a micro layer; Get the observation variables of the first agent; The observation variables are input into the path planning model, and the path planning model outputs first navigation information through the macro layer; the first navigation information includes a first target location of the first agent; the first target location is used to indicate a first target location of the first agent; Inputting the first navigation information into the decision layer, the decision layer outputting second navigation information based on the first navigation information, wherein the second navigation information includes a second target location of the first agent; the second target location is used to indicate a location that the first agent will pass when reaching the first target location, and the second navigation information changes at any time as navigation progresses; Inputting the second navigation information into the micro layer, the micro layer outputting third navigation information based on the second navigation information; the third navigation information includes a third target location of the first agent; the third target location is used to indicate the direction of the first agent when departing to the first target location, and the third navigation information changes at any time as navigation progresses; The first agent is navigated toward the second target location according to the third target location until the first agent reaches the first target location.

2. The method according to claim 1, characterized in that The method further comprises: Obtaining first intervention information input by the user; Inputting the first intervention information into the macro layer, the macro layer intervening in the first navigation information based on the first intervention information, and outputting fourth navigation information, wherein the fourth navigation information includes a fourth target location of the first agent; Inputting the fourth navigation information into the decision layer, the decision layer outputting fifth navigation information based on the fourth navigation information, the fifth navigation information including a fifth target location of the first agent; The micro layer outputs sixth navigation information based on the fifth navigation information; the sixth navigation information includes a sixth target position of the first agent; The first agent is navigated toward the fifth target location according to the sixth target location until the first agent reaches the fourth target location.

3. The method according to claim 1, characterized in that The method further comprises: Obtaining second intervention information input by the user; inputting the second intervention information into the decision layer, wherein the decision layer intervenes in the second navigation information based on the second intervention information and outputs seventh navigation information, wherein the seventh navigation information includes a seventh target position of the first agent; The micro layer outputs eighth navigation information based on the seventh navigation information; the eighth navigation information includes an eighth target position of the first agent; The first agent is navigated toward the seventh target position according to the eighth target position until the first agent reaches the first target position.

4. The method according to claim 1, wherein The method further comprises: Obtaining third intervention information input by the user; Inputting the third intervention information into the micro layer, wherein the micro layer intervenes in the third navigation information based on the third intervention information and outputs ninth navigation information; the ninth navigation information includes a ninth target position of the first agent; The first agent is navigated toward the second target position according to the ninth target position until the first agent reaches the first target position.

5. The method according to claim 1, wherein The path planning model further includes a feature extraction layer and a coding mapping layer; the feature extraction layer and the output layer are connected via the coding mapping layer, the observation variable is input into the path planning model, and the path planning model outputs the first navigation information via the macro layer, including: Inputting the observed variables into the path planning model, performing feature extraction on the observed variables through the feature extraction layer to obtain feature data corresponding to the observed variables; Performing encoding and mapping processing on the feature data through the encoding and mapping layer to obtain a feature vector corresponding to the observed variable; The macro layer outputs first navigation information based on the feature vector.

6. The method according to claim 1, wherein The macro layer is not time-sensitive, while the decision layer and the micro layer are time-sensitive.

7. The method according to claim 1, characterized in that The constructing of the path planning model includes: Obtaining sample observation variables of a sample agent and offline navigation trajectory data of the sample agent; Determine a first tag, a second tag, and a third tag of the sample agent based on the offline navigation trajectory data; the first tag is used to indicate the sample target location of the sample agent, the second tag is used to indicate the location passed by the sample agent when arriving at the sample target location, and the third tag is used to indicate the direction of the sample agent when arriving at the sample target location; The sample agent, the first label, the second label, and the third label are input into a deep learning model for training to obtain a path planning model.

8. An intelligent navigation device, characterized in that: The device comprises: A model building module is used to build a path planning model, wherein the output layer of the path planning model includes a macro layer, a decision layer, and a micro layer; A variable acquisition module, used to obtain the observation variables of the first agent; a first navigation module, configured to input the observed variables into the path planning model, wherein the path planning model outputs first navigation information via the macro layer; the first navigation information includes a first target location of the first agent; the first target location is used to indicate a first target location of the first agent; a second navigation module, configured to input the first navigation information into the decision layer, wherein the decision layer outputs second navigation information based on the first navigation information, wherein the second navigation information includes a second target location of the first agent; the second target location indicates a location that the first agent will pass when arriving at the first target location; a third navigation module configured to output third navigation information at the micro-level based on the second navigation information; the third navigation information including a third target location of the first agent; the third target location indicating the direction of the first agent when departing to the first target location; A control module is used to navigate the first intelligent body to the second target position according to the third target position until the first intelligent body reaches the first target position.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor is caused to perform the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor is caused to perform the steps of the method according to any one of claims 1 to 7.

Images