Intelligent agent automatic path-finding method and device, storage medium and electronic equipment

Abstract

The invention relates to the technical field of game control, and particularly provides an intelligent agent automatic path-finding method and device, a storage medium and electronic equipment, and the method can comprise the steps: in a process that an intelligent agent moves to a target position according to a movement feedback parameter, obtaining a current state of the intelligent agent and the target position in a current frame, the current state comprises at least two of distance, angle and execution action; the current state is scored, the state score of the intelligent agent is determined, and the state score is used for representing whether the current state is correct or not; and feeding back the state score to the intelligent agent, so that the intelligent agent correctly moves to the target position on the basis of the state score. According to some embodiments of the invention, efficient automatic path finding of the intelligent agent can be realized.

Description

Technical Field

[0001] The present application relates to the field of game control technology, and more specifically, to a method, device, storage medium and electronic device for automatic path finding of an intelligent agent. Background Art

[0002] As large-world games continue to increase, in order to meet the needs of actual test applications, intelligent agents (or virtual objects) need to automatically move and find paths according to actual scenarios.

[0003] In the current intelligent agent automatic pathfinding scheme, scripts are generally written according to certain rules to control the movement of virtual objects. However, it is difficult to write rules, and in some scenarios, the automatic pathfinding movement of virtual objects is significantly different from the movement of virtual objects controlled by normal users. Another way is to plan the route directly on the game map, and the virtual object automatically moves and finds the path according to the planned route. However, this solution is relatively fixed and cannot adjust the route in real time according to the changes in the surrounding virtual scenery, which greatly reduces the intelligence.

[0004] Therefore, how to provide a technical solution for an intelligent agent to automatically find its way has become a technical problem that needs to be solved urgently. Summary of the invention

[0005] The purpose of some embodiments of the present application is to provide a method, device, storage medium and electronic device for automatic path finding of an intelligent body. Through the technical solutions of the embodiments of the present application, the movement of the intelligent body can be adjusted in real time, the accuracy and efficiency of path finding can be improved, and the degree of intelligence is relatively high.

[0006] In a first aspect, some embodiments of the present application provide a method for automatic pathfinding of an intelligent agent, comprising: in the process of the intelligent agent moving toward a target position according to movement feedback parameters, obtaining a current state of the intelligent agent and the target position in a current frame, wherein the current state includes at least two of distance, angle, and execution action; scoring the current state to determine a state score of the intelligent agent, wherein the state score is used to characterize whether the current state is correct; and feeding back the state score to the intelligent agent so that the intelligent agent can correctly move to the target position based on the state score.

[0007] Some embodiments of the present application collect and score the current state of the current frame of the agent during its movement to obtain a state score, and finally feed the state score back to the agent, so that the agent can know whether the moving direction is correct, so as to realize the automatic path finding of the agent. In the embodiments of the present application, the agent can adjust the movement situation in real time through the state score, improve the accuracy and efficiency of path finding, and has a high degree of intelligence.

[0008] In some embodiments, before obtaining the current state of the agent and the target position in the current frame, the method further includes: obtaining the movement state parameters between the agent and the target position; inputting the movement state parameters into the objective function to obtain the movement feedback parameters.

[0009] Some embodiments of the present application input the movement feedback parameters of the agent through the movement state parameters and the objective function so that the agent can move accordingly, thereby improving the accuracy of automatic path finding.

[0010] In some embodiments, the weight values ​​of the environmental factors within the objective function are adaptively adjusted according to the environment in which the agent is located; the environmental factors include: mountains, rivers, land and air.

[0011] Some embodiments of the present application achieve precise and flexible control of the intelligent agent by adaptively adjusting the relevant weight values ​​of the objective function through different environmental factors.

[0012] In some embodiments, the current state is scored to determine the state score of the agent, including: obtaining a distance score and a distance weight for scoring the distance, and obtaining an angle score and an angle weight for scoring the angle; obtaining an action score and an action weight by confirming whether the executed action is a specified action in a specified scene; and calculating the distance score and the distance weight, the angle score and the angle weight, and the action score and the action weight to obtain the state score.

[0013] Some embodiments of the present application obtain a state score by scoring and weighting the distance, angle, and execution action in the current state, and then calculating it. This can analyze whether the current state of the current intelligent agent is correct when moving to the target position, thereby ensuring the accuracy of automatic path finding.

[0014] In some embodiments, the distance score and the distance weight are obtained by the following method: based on the distance parameter data before the current frame, the distance is scored to obtain the distance score; wherein, the distance score is a positive number, indicating that the moving direction of the agent is correct, and the distance score is a negative number, indicating that the moving direction is wrong; the distance weight is determined by comparing the distance with a preset distance threshold.

[0015] Some embodiments of the present application determine the distance score corresponding to the current distance through distance parameter data, and determine the distance weight through a preset distance threshold, thereby providing data support for improving the accuracy of automatic path finding.

[0016] In some embodiments, the angle score is obtained by the following method: if the angle is within the preset fan-shaped area corresponding to the angle value parameter, the angle score is a positive number; the positive angle score indicates that the moving direction of the intelligent body is correct; if the angle is not within the preset fan-shaped area, the angle score is a negative number; the negative angle score indicates that the moving direction of the intelligent body is incorrect.

[0017] Some embodiments of the present application determine the angle score by comparing the angle with a preset fan-shaped area, thereby providing data support for improving the accuracy of automatic path finding.

[0018] In a second aspect, some embodiments of the present application provide a device for automatic pathfinding of an intelligent agent, comprising: an acquisition module, used to acquire the current state of the intelligent agent and the target position in the current frame during the process of the intelligent agent moving to the target position according to the movement feedback parameters, the current state including at least two of distance, angle and execution action; a scoring module, used to score the current state and determine the state score of the intelligent agent, wherein the state score is used to characterize whether the current state is correct; a feedback module, used to feed back the state score to the intelligent agent, so that the intelligent agent can correctly move to the target position based on the state score.

[0019] In a third aspect, some embodiments of the present application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, can implement the method described in any embodiment of the first aspect.

[0020] In a fourth aspect, some embodiments of the present application provide an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor can implement a method as described in any embodiment of the first aspect when executing the program.

[0021] In a fifth aspect, some embodiments of the present application provide a computer program product, wherein the computer program product comprises a computer program, wherein the computer program, when executed by a processor, can implement the method described in any embodiment of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions of some embodiments of the present application, the drawings required for use in some embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0023] Figure 1A system diagram of automatic path finding for an intelligent agent provided for some embodiments of the present application;

[0024] Figure 2 One of the flow charts of the method for automatic path finding of an intelligent agent provided in some embodiments of the present application;

[0025] Figure 3 Flowchart 2 of the method for automatic path finding of an intelligent agent provided in some embodiments of the present application;

[0026] Figure 4 A block diagram of the composition of an intelligent agent automatic path finding device provided in some embodiments of the present application;

[0027] Figure 5 A schematic diagram of an electronic device is provided for some embodiments of the present application. DETAILED DESCRIPTION

[0028] The technical solutions in some embodiments of the present application will be described below in conjunction with the drawings in some embodiments of the present application.

[0029] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0030] In the virtual scene pathfinding method disclosed in the related art, the user's real movement route and movement operation in the virtual scene are simulated through the intermediate point prediction model learning, and the long-distance pathfinding is divided into multiple short-distance pathfinding segments by setting up intermediate nodes, and the route flexibility during pathfinding is strengthened, so that the automatic pathfinding in the virtual scene is more like the movement route operated by the user himself. However, by calculating the current position of the virtual object, the position information of the intermediate point is obtained through reinforcement learning, and then the intermediate point is used as the starting point of the next path to continue to use reinforcement learning to reason, and finally the virtual object reaches the final position. This method requires learning the position information of multiple intermediate points. If the path is long or the scene is more complex, the efficiency of pathfinding will be greatly reduced, and the accuracy cannot be guaranteed.

[0031] It can be seen from the above-mentioned related technologies that the automatic path-finding method of the intelligent body in the existing technology has a low degree of intelligence and its accuracy cannot be guaranteed.

[0032] In view of this, some embodiments of the present application provide a method for automatic pathfinding of an intelligent agent, which can collect the current state of the intelligent agent's movement while the intelligent agent is moving toward a target location; then score the current state of the intelligent agent to obtain a state score, and use the state score to confirm whether the current state of the intelligent agent is getting closer to the target location, rather than farther away. By feeding back the state score to the intelligent agent, the intelligent agent can automatically learn the correct path, adjust the distance and angle deviations in time, realize the intelligent pathfinding operation of the intelligent agent, and ensure the efficiency and accuracy of automatic pathfinding.

[0033] The following is combined with Figure 1 The overall composition structure of the system for automatic path finding of an intelligent agent provided by some embodiments of the present application is exemplified.

[0034] like Figure 1 As shown, some embodiments of the present application provide a system for automatic pathfinding of an intelligent agent, and the system for automatic pathfinding of an intelligent agent includes: a terminal 100 and a server 200. Among them, the terminal 100 is deployed with a virtual scene of a certain game and an intelligent agent that needs to move from a certain position to a target position in the virtual scene. In the process of the intelligent agent moving to the target position, the server 200 can obtain the state of the intelligent agent during the movement, and by scoring the current state of the intelligent agent in the current frame to obtain a state score, it can be determined whether the moving path of the intelligent agent is correct, that is, whether it is approaching the target position instead of moving away. The server 200 feeds back the state score to the intelligent agent, so that the intelligent agent can learn the correct moving state and realize automatic pathfinding of the intelligent agent.

[0035] In some embodiments of the present application, the server 200 is deployed with an image recognition model that can recognize environmental information in the virtual scene of the game of the terminal 100. The image recognition model can recognize scenery and objects such as mountains, rivers, trees, etc. in the virtual scene to adjust the state of the intelligent body. Among them, the image recognition model is obtained by training the model with a large number of pictures of mountains, rivers, trees, etc. as input samples and the corresponding scenery such as mountains, rivers, trees, etc. as output samples. At the same time, the server 200 is also deployed with a supervised pre-trained image model, which can realize the motion capture of the intelligent body. In the game, the distance between the virtual object and the target obstacle (for example, rivers, mountains, trees, etc.) can be captured by the action performance of the virtual object, so as to distinguish and design the corresponding action logic. Among them, the ResNet50 network can be used for action classification tasks; the detection task of virtual objects or other objects can be implemented using the Yolo network. In addition, the target model can also be trained by model training to realize the recognition of the image information of the target point (as a specific example of the target position) on the map corresponding to the virtual scene, so as to output the recognition information of the target point and feed it back to the intelligent agent of the terminal 100, so that the intelligent agent can know the direction information of the target point (for example, east, south, west, etc.). The trained image model can also recognize the information range of the scenery in the virtual scene (for example, the range of mountains, rivers, and the air has exceeded the screen display range), so as to give the intelligent agent corresponding acceleration or deceleration processing for operations such as climbing and swimming, so that the intelligent agent can smoothly climb mountains, swim and fly in the air.

[0036] In addition, in some embodiments of the present application, in the path-finding task, when the starting distance between the target point and the virtual object exceeds the maximum distance M on the map, the map cannot accurately return the precise location of the target outside the interval. Therefore, when calculating the distance between the agent and the target point, first obtain the parameter: distance = normalize (min (M, distance)), and then use the normalization operation to obtain the normalized distance feature distance i ', the specific formula is:

[0037]

[0038] In the formula, ε is a set minimum value, x can represent all distance sets; min(x) is the minimum distance information that the environment can provide to the image recognition model. This application can use the LeakyReLU activation function to calculate the distance i ', to avoid gradient disappearance due to environmental problems.

[0039] It should be noted that in the above-mentioned examples of implementing corresponding functions by training models, model training adopts conventional training methods, and the embodiments of the present application can also adjust the data sets of relevant model training and train other models according to the actual virtual game scene requirements. The embodiments of the present application are not specifically limited here.

[0040] The following is combined with Figure 2 The implementation process of automatic path finding of an intelligent agent performed by the server 200 provided in some embodiments of the present application is exemplified.

[0041] Please see attached Figure 2 , Figure 2 A flow chart of a method for automatic path finding of an intelligent agent is provided for some embodiments of the present application. The method for automatic path finding of an intelligent agent may include:

[0042] S210, in the process of the intelligent agent moving toward the target position according to the movement feedback parameter, obtaining the current state of the intelligent agent and the target position in the current frame, wherein the current state includes at least two of: distance, angle and execution action.

[0043] For example, in some embodiments of the present application, when the intelligent agent in the virtual scene of the game of the terminal 100 moves toward the target point, the situation between the intelligent agent and the target point is detected and identified to obtain the distance between the intelligent agent and the target point in the current frame, the angle of the relative position of the intelligent agent and the target point, and the execution action of the intelligent agent.

[0044] In some embodiments of the present application, before executing S210, the method for automatic path finding of the intelligent agent may include: obtaining the movement state parameters between the intelligent agent and the target location; inputting the movement state parameters into the objective function to obtain the movement feedback parameters.

[0045] For example, in some embodiments of the present application, the environment and action execution state of the agent are identified by computer vision methods to obtain the movement state parameters of the agent (for example, the distance and angle between the agent and the target point). Then, the movement feedback parameters are input into the designed reward function (as a specific example of the objective function), and the specific execution operation fed back to the agent is output (as a specific example of the movement feedback parameters); wherein the specific execution operation may include the directional information such as up, down, left, right, front, and back that the agent needs to execute.

[0046] In some embodiments of the present application, the weight values ​​of the environmental factors within the objective function are adaptively adjusted according to the environment in which the agent is located; the environmental factors include: mountains, rivers, land and air.

[0047] For example, in some embodiments of the present application, the environment and the action execution state are processed into multiple types for the analysis of the game virtual scene. The environment (as a specific example of environmental elements) can contain hills, rivers and plains (i.e., land), etc., and the actions corresponding to the scene are climbing, swimming and running. In different environments, the reward function can adaptively adjust the weight value inside it. For example, for operations such as climbing, swimming, and leaping, by adjusting the weight values ​​of these references in the reward function, the specific execution operation of the intelligent agent can be accelerated or decelerated accordingly, so that the intelligent agent can climb mountains, swim and fly in the air smoothly. It should be understood that the embodiments of the present application are not limited to this. In the game, the direction of the game perspective will affect the direction in which the virtual object performs the action, so in order to better fit the task, the map environment is set in the task to change with the direction of the protagonist.

[0048] S220, scoring the current state to determine a state score of the agent, wherein the state score is used to indicate whether the current state is correct.

[0049] For example, in some embodiments of the present application, the current movement state of the agent is scored to confirm whether the agent is moving correctly toward the target point, so that the agent can automatically learn and have path-finding capabilities.

[0050] In some embodiments of the present application, S220 may include:

[0051] S221, obtaining a distance score and a distance weight for scoring the distance, and obtaining an angle score and an angle weight for scoring the angle.

[0052] For example, in some embodiments of the present application, three different reward logics are set to analyze the current distance between the agent and the target point, the angle, and the action execution status, so as to obtain three scores and weights in different situations, and finally determine the final state score of the agent by combining the three results.

[0053] In some embodiments of the present application, the distance score and distance weight are obtained by the following method: based on the distance parameter data before the current frame, the distance is scored to obtain the distance score; wherein, the distance score is a positive number, indicating that the moving direction of the agent is correct, and the distance score is a negative number, indicating that the moving direction is wrong; the distance weight is determined by comparing the distance with a preset distance threshold.

[0054] For example, in some embodiments of the present application, a basic distance feedback logic is designed. The automatic path-finding task is determined based on the small map in the upper left corner of the game. When the display range of the small map is exceeded, it is still displayed at the maximum extent. Therefore, only when the agent approaches a certain distance, the relative distance parameter between the agent and the target point will change. In order to allow the agent to learn the rewards and penalties brought about by the distance change, the memory window is considered to calculate the trend change of the distance parameter (as a specific example of the distance parameter data). If the current distance between the agent and the target point is closer and closer to the target point than the previous distance, it indicates that its moving direction is correct. At this time, the agent is given a reward value (as a specific example of a positive distance score), for example, 5 reward values ​​(or basic reward values) are given each time; if the current distance between the agent and the target point is farther and farther away from the target point than the previous distance, it indicates that its moving direction is wrong. At this time, the agent is given a penalty value (as a specific example of a negative distance score), for example, -5 penalty values ​​are given each time. Moreover, when the distance value between the agent and the target point is shortened to a certain threshold (as a specific example of a preset distance threshold), the basic reward value will be considered. If the distance of the agent in the current frame exceeds the weighted distance value in the memory window (i.e., the weighted value of the distance), it indicates that the agent has deviated from the target point, which will cause the agent to obtain a preset penalty value. It should be understood that the reward and punishment scores for the agent based on the distance can also be set in stages according to the distance size. The basic distance feedback logic can be flexibly adjusted according to actual needs, and the embodiments of the present application are not limited to this.

[0055] In some embodiments of the present application, the angle score is obtained by the following method: if the angle is within the preset fan-shaped area corresponding to the angle value parameter, the angle score is a positive number; the positive angle score indicates that the moving direction of the intelligent body is correct; if the angle is not within the preset fan-shaped area, the angle score is a negative number; the negative angle score indicates that the moving direction of the intelligent body is incorrect.

[0056] For example, in some embodiments of the present application, a direction selection feedback logic is designed. Based on the change in the angle variation, during the pathfinding process, after the virtual object performs the corresponding action, the reward function will show a change in the map. Design timely feedback logic based on the angle variation (that is, direction selection feedback logic): After the display agent changes the action execution, the angle (that is, the angle) between the target point and the direction of the agent becomes smaller or larger. In the pathfinding task, a smaller angle means that the agent's forward direction is constantly approaching the target position, and an increase means that the agent's forward direction is constantly deviating from the correct direction. The approach process is continuous. Considering the importance of the reward trend in task learning, a phased angle variation feedback is designed, and the angle interval is used to allow the agent to learn to perform the correct action.

[0057] This application found that the mini-map bound to the agent's orientation can reduce the binary nature of the action. Specifically, in the unbound mini-map display, the agent's lens orientation will not affect the position of the target point in the agent, but the agent's lens will affect the direction of the agent's movement. Such problems will lead to differences in agent learning and make it difficult for the model to converge. Therefore, this application binds the agent's orientation to the mini-map, and the agent's lens orientation always faces the forward direction. Based on this, the timely feedback logic based on the angle change is proposed as follows:

[0058] The relative position angle between the agent and the target point is embedded in the range of (-180, 180). The direction of the agent when moving towards the target point is 0°, and vice versa is ±180°. Add the angle value parameter of the relative position of the target in the memory window. When the agent is bound to the mini-map, the agent can ensure that the target point is in a fan-shaped search area (as a specific example of a preset fan-shaped area) with an angle less than a certain threshold by performing actions with offsets (for example, left shift, right shift). In the fan-shaped search area, in theory, when the forward distance in the execution action of the agent is greater than the offset, it can continue to approach the target point. Therefore, the change in the angle value caused by the execution of the action allows the agent to learn the feedback brought by the direction selection.

[0059] Specifically, the environmental feedback based on the angle change designed in the embodiment of the present application can help the agent determine which actions will adjust to the target point within the threshold range more quickly. If the angle between the agent and the target point reaches the target point within a fan-shaped area (that is, a preset fan-shaped area) less than the threshold after the agent performs a certain action, a reward will be given; if the target point is away from the historical frame of the angle corresponding to the weight value of the angle in the memory window after the agent performs a certain action, a penalty will be given. The values ​​of the reward value and the penalty value can refer to the reward and punishment mechanism of the distance above, and the embodiment of the present application is not specifically limited here.

[0060] S222: Acquire an action score and an action weight by confirming whether the executed action is a specified action in a specified scene.

[0061] For example, in some embodiments of the present application, an intelligent agent action execution feedback logic is also designed. Through computer vision methods, the environment and action execution state of the intelligent agent are identified. For the analysis of the game scene, the environment and action execution state are processed into three types respectively. The environment is like a hill, a river and a plain, and the action execution corresponding to the scene is climbing, swimming and running. That is, the intelligent agent needs to perform a specific action (as a specific example of a specified action) in a specific scene (as a specific example of a specified scene). Therefore, the present application processes the environmental information identified by vision into a high-dimensional space, obtains a specific action that needs to be performed in a specific scene, and provides it to the intelligent agent for learning. Therefore, the present application designs a unique action execution feedback logic (that is, the intelligent agent action execution feedback logic), that is, the intelligent agent learns to perform a specific action in a specific scene, and will obtain a larger reward value. For example, when the intelligent agent is in a specific scene, the execution action of the intelligent agent is compared with the specific action to confirm whether the two are consistent. If they are consistent, the intelligent agent is given a reward value, otherwise a penalty value is given. For example, when the agent is in a specific scenario, if the agent's execution action is consistent with the specific action, it will be given a reward value of 10 each time, otherwise it will be given a penalty value of 10, that is, -10.

[0062] It should be noted that the above-mentioned reward and punishment scores can be set flexibly, and the embodiments of the present application are not limited to this.

[0063] In addition, the sum of the distance weight, angle weight and action weight mentioned above is a fixed value of 100%. The distance weight, angle weight and action weight can be flexibly adjusted according to preset conditions. For example, when the distance between the agent and the target point exceeds the range of the screen (i.e., the small map), the distance weight will be greater than the angle weight (i.e., the angle weight), otherwise the angle weight increases, and the action weight is adaptively adjusted. The specific weight adjustment method can be selected according to the actual application scenario, and the embodiments of the present application are not limited to this.

[0064] S223: Calculate the distance score and the distance weight, the angle score and the angle weight, and the action score and the action weight to obtain the state score.

[0065] For example, in some embodiments of the present application, the distance score and distance weight, the angle score and angle weight, and the action score and action weight are weighted and summed to obtain the final reward score for the agent (as a specific example of a state score).

[0066] S230, feeding back the state score to the agent, so that the agent can correctly move to the target position based on the state score.

[0067] For example, in some embodiments of the present application, the reward score is fed back to the agent. The reward score can be a positive number or a negative number, and the agent can learn to perform the correct action by itself through the reward score, so that it can automatically find its way to the target point. Although only three feedback logics are given in the above embodiment, considering the different game scenes, there can be multiple feedback logics in different games, and the embodiments of the present application are not limited to this. Finally, the reward score fed back to the agent is determined through comprehensive analysis, and the agent can learn the specified correct action logic and complete automatic path finding.

[0068] The following is combined with Figure 3 The specific process of automatic path finding of an intelligent agent provided by some embodiments of the present application is exemplified.

[0069] Please see attached Figure 3 , Figure 3 A flow chart of a method for automatic path finding of an intelligent agent provided for some embodiments of the present application.

[0070] The above process is explained below as an example.

[0071] S301, obtaining movement state parameters between the agent and the target position.

[0072] S302, inputting the movement state parameter into the objective function to obtain the movement feedback parameter.

[0073] S303, when the agent moves toward the target position according to the movement feedback parameters, the current state of the agent and the target position in the current frame is obtained within a preset time period; the current state includes: distance, angle and execution action.

[0074] S304: Obtain a distance score and a distance weight for scoring the distance.

[0075] S305, obtaining the angle score and angle weight for scoring the angle.

[0076] S306, obtaining the action score and action weight of the agent.

[0077] S307, performing weighted calculation on the distance score and distance weight, the angle score and angle weight, and the action score and action weight to obtain a state score.

[0078] S308, feeding back the state score to the agent.

[0079] It should be noted that the server 200 executes the above operations in a loop until the agent reaches the target position. The specific implementation process of S301 to S308 can refer to the method embodiment provided above, and the detailed description is appropriately omitted here to avoid repetition.

[0080] Through some of the above embodiments of the present application, it can be known that the present application uses a combination of reinforcement learning (i.e., designing a reward function) and image processing to automatically find the path of a virtual object. The main method is to calculate the distance between the virtual object and the target point and the change in angle in the actual map of the game, and use reinforcement learning to reason about the distance and angle between the virtual object and the target point; use image processing to identify the target point and the changes in the surrounding scenery and feed back the actual situation of the scenery to the reinforcement learning model to control the virtual object to perform corresponding operations, such as swimming, climbing, running, etc. The embodiments of the present application can realize automatic path finding of virtual objects, which is both efficient and accurate.

[0081] Please refer to Figure 4 , Figure 4 The block diagram of the composition of the intelligent agent automatic path finding device provided by some embodiments of the present application is shown. It should be understood that the intelligent agent automatic path finding device corresponds to the above method embodiment and can execute each step involved in the above method embodiment. The specific functions of the intelligent agent automatic path finding device can be found in the description above. To avoid repetition, the detailed description is appropriately omitted here.

[0082] Figure 4 The intelligent agent automatic path-finding device includes at least one software function module that can be stored in a memory in the form of software or firmware or solidified in the intelligent agent automatic path-finding device, and the intelligent agent automatic path-finding device includes: an acquisition module 410, which is used to acquire the current state of the intelligent agent and the target position in the current frame during the process of the intelligent agent moving to the target position according to the movement feedback parameter, and the current state includes at least two of: distance, angle and execution action; a scoring module 420, which is used to score the current state and determine the state score of the intelligent agent, wherein the state score is used to indicate whether the current state is correct; a feedback module 430, which is used to feed back the state score to the intelligent agent, so that the intelligent agent can correctly move to the target position based on the state score.

[0083] In some embodiments of the present application, the acquisition module 410 is used to obtain the movement state parameters between the agent and the target position; input the movement state parameters into the objective function to obtain the movement feedback parameters.

[0084] In some embodiments of the present application, the scoring module 420 is used to obtain a distance score and a distance weight for scoring the distance, and to obtain an angle score and an angle weight for scoring the angle; obtain an action score and an action weight by confirming whether the executed action is a specified action in a specified scene; and calculate the distance score and the distance weight, the angle score and the angle weight, and the action score and the action weight to obtain the state score.

[0085] In some embodiments of the present application, the scoring module 420 is used to score the distance based on the distance parameter data before the current frame to obtain the distance score; wherein, the distance score is a positive number, indicating that the moving direction of the agent is correct, and the distance score is a negative number, indicating that the moving direction is wrong; the distance weight is determined by comparing the distance with a preset distance threshold.

[0086] In some embodiments of the present application, the scoring module 420 is used to: if the angle is within the preset fan-shaped area corresponding to the angle value parameter, the angle score is a positive number; the positive angle score indicates that the moving direction of the intelligent body is correct; if the angle is not within the preset fan-shaped area, the angle score is a negative number; the negative angle score indicates that the moving direction of the intelligent body is incorrect.

[0087] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the device described above can refer to the corresponding process in the aforementioned method, and will not be described in detail here.

[0088] Some embodiments of the present application further provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, can implement the operations of the method corresponding to any of the above methods provided in the above embodiments.

[0089] Some embodiments of the present application further provide a computer program product, which includes a computer program, wherein when the computer program is executed by a processor, it can implement the operations corresponding to any of the above methods provided in the above embodiments.

[0090] like Figure 5 As shown, some embodiments of the present application provide an electronic device 500, which includes: a memory 510, a processor 520, and a computer program stored in the memory 510 and executable on the processor 520, wherein the processor 520 can implement a method as described in any of the above embodiments when reading the program from the memory 510 through a bus 530 and executing the program.

[0091] Processor 520 can process digital signals and can include various computing structures, such as complex instruction set computer structure, reduced instruction set computer structure, or a structure that implements a combination of multiple instruction sets. In some examples, processor 520 can be a microprocessor.

[0092] The memory 510 may be used to store instructions executed by the processor 520 or data related to the execution of instructions. These instructions and / or data may include codes for implementing some or all functions of one or more modules described in the embodiments of the present application. The processor 520 of the disclosed embodiment may be used to execute instructions in the memory 510 to implement the method shown above. The memory 510 includes a dynamic random access memory, a static random access memory, a flash memory, an optical memory, or other memory known to those skilled in the art.

[0093] The above description is only an embodiment of the present application and is not intended to limit the scope of protection of the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application. It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings.

[0094] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0095] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

Claims

1. A method for automatic path finding of an intelligent agent, characterized in that: include: In the process of the agent moving toward the target position according to the movement feedback parameter, obtaining the current state of the agent and the target position in the current frame, wherein the current state includes at least two of the distance, the angle, and the execution action; Scoring the current state to determine a state score of the agent, wherein the state score is used to indicate whether the current state is correct; The state score is fed back to the agent so that the agent can correctly move to the target position based on the state score.

2. The method according to claim 1, characterized in that Before obtaining the current state of the agent and the target position in the current frame, the method further includes: Acquire movement state parameters between the agent and the target position; The movement state parameter is input into the objective function to obtain the movement feedback parameter.

3. The method according to claim 2, characterized in that The weight values ​​of the environmental factors within the objective function are adaptively adjusted according to the environment in which the intelligent agent is located; the environmental factors include: mountains, rivers, land and air.

4. The method according to any one of claims 1 to 3, characterized in that Scoring the current state to determine the state score of the agent includes: Obtaining a distance score and a distance weight for scoring the distance, and obtaining an angle score and an angle weight for scoring the angle; Acquire an action score and an action weight by confirming whether the executed action is a specified action in a specified scene; The distance score and the distance weight, the angle score and the angle weight, and the action score and the action weight are calculated to obtain the state score.

5. The method according to claim 4, characterized in that The distance score and the distance weight are obtained by the following method: Based on the distance parameter data before the current frame, the distance is scored to obtain the distance score; wherein the distance score is a positive number, indicating that the moving direction of the agent is correct, and the distance score is a negative number, indicating that the moving direction is wrong; The distance weight is determined by comparing the distance with a preset distance threshold.

6. The method according to claim 4, characterized in that The angle score is obtained by the following method: If the angle is within the preset sector area corresponding to the angle value parameter, the angle score is a positive number; a positive angle score indicates that the moving direction of the intelligent agent is correct; If the angle is not within the preset fan-shaped area, the angle score is a negative number; a negative angle score indicates that the moving direction of the agent is wrong.

7. A device for automatic path finding of an intelligent agent, characterized in that: include: An acquisition module, used for acquiring a current state of the agent and the target position in a current frame during the process of the agent moving toward the target position according to the movement feedback parameter, wherein the current state includes at least two of a distance, an angle, and an execution action; A scoring module, used to score the current state and determine the state score of the agent, wherein the state score is used to indicate whether the current state is correct; The feedback module is used to feed back the state score to the intelligent agent so that the intelligent agent can correctly move to the target position based on the state score.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program executes the method according to any one of claims 1 to 6 when executed by a processor.

9. An electronic device, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the computer program executes the method according to any one of claims 1 to 6 when being run by the processor.

10. A computer program product, characterized in that The computer program product comprises a computer program, wherein the computer program executes the method according to any one of claims 1 to 6 when executed by a processor.

Images