Virtual robot action determination method, device, equipment and storage medium

Abstract

The present application provides a method, device, equipment and storage medium for determining a virtual robot action, wherein the method includes: obtaining the executable behavior of the virtual robot to be controlled in the game and the game status information at the current moment, and determining multiple candidate actions according to multiple action targets corresponding to the executable behavior, wherein the candidate action is used to represent the game action formed by applying the executable behavior to the action target, and different candidate actions correspond to different action targets, and according to the game status information at the current moment, the action to be executed of the virtual robot to be controlled at the next moment is determined from multiple candidate actions. In the present application, multiple candidate actions are determined by the executable behavior of the virtual robot to be controlled, and the action to be executed is determined from multiple candidate actions, which compresses the action space and reduces the computing resource consumption of the virtual robot.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method, device, equipment and storage medium for determining a virtual robot action. Background Art

[0002] As one of the most popular game categories in recent years, Multiplayer Online Battle Arena (MOBA) games, with their rich character types, complex skills and cooperation mechanisms, have brought lasting fun to players while also becoming a major challenge for Artificial Intelligence (AI). Among them, some of these MOBA games use Deep Reinforcement Learning (DRL) technology.

[0003] In the prior art, game state features are usually input into a reinforcement neural network model, which then outputs the actions that the game character needs to take. Reinforcement learning allows the game character to actively interact with the environment with the goal of maximizing the rewards obtained. In the process of interacting with the environment, the game character learns which behaviors are better through reward signals.

[0004] However, this scheme adopts multi-head action output, that is, the output action consists of multiple dimensions. The high-dimensional actions constitute a huge action space, which makes reinforcement learning difficult and consumes a lot of computing resources. Summary of the invention

[0005] In view of this, the embodiments of the present application provide a virtual robot action determination method, device, equipment and storage medium to solve the problems of high learning difficulty and high computing resource consumption caused by high-dimensional action space in the prior art.

[0006] In a first aspect, an embodiment of the present application provides a method for determining a virtual robot action, comprising:

[0007] Obtain the executable behaviors of the virtual robot to be controlled in the game and the current game status information;

[0008] Determine a plurality of candidate actions according to a plurality of action targets corresponding to the executable behavior, wherein the candidate actions are used to represent game actions formed by applying the executable behavior to the action targets, and different candidate actions correspond to different action targets;

[0009] According to the game state information at the current moment, the action to be performed by the virtual robot to be controlled at the next moment is determined from the multiple candidate actions.

[0010] In a second aspect, the embodiment of the present application further provides a virtual robot action determination device, comprising:

[0011] An acquisition module is used to acquire the executable behaviors of the virtual robot to be controlled in the game and the game status information at the current moment;

[0012] A determination module, used to determine a plurality of candidate actions according to a plurality of action targets corresponding to the executable behavior, wherein the candidate actions are used to represent game actions formed by applying the executable behavior to the action targets, and different candidate actions correspond to different action targets;

[0013] The determination module is further used to determine the action to be performed by the virtual robot to be controlled at the next moment from the multiple candidate actions according to the game state information at the current moment.

[0014] In a third aspect, an embodiment of the present application further provides an electronic device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the memory through the bus, and the processor executes the machine-readable instructions to execute the virtual robot action determination method described in any one of the first aspects.

[0015] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method for determining a virtual robot action described in any one of the first aspects is executed.

[0016] The present application provides a method, device, equipment and storage medium for determining a virtual robot action, wherein the method includes: obtaining the executable behavior of the virtual robot to be controlled in the game and the game status information at the current moment, and determining multiple candidate actions according to multiple action targets corresponding to the executable behavior, wherein the candidate action is used to represent the game action formed by applying the executable behavior to the action target, and different candidate actions correspond to different action targets, and according to the game status information at the current moment, the action to be executed of the virtual robot to be controlled at the next moment is determined from multiple candidate actions. In the present application, multiple candidate actions are determined by the executable behavior of the virtual robot to be controlled, and the action to be executed is determined from multiple candidate actions, which compresses the action space and reduces the computing resource consumption of the virtual robot.

[0017] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 Schematic diagram of the process of the virtual robot action determination method provided in the embodiment of the present application Figure 1 ;

[0020] Figure 2 A schematic diagram of a process for generating candidate actions provided in an embodiment of the present application;

[0021] Figure 3 Schematic diagram of the process of the virtual robot action determination method provided in the embodiment of the present application Figure 2 ;

[0022] Figure 4 A schematic diagram of a reinforced neural network model provided in an embodiment of the present application;

[0023] Figure 5 Schematic diagram of the process of the virtual robot action determination method provided in the embodiment of the present application Figure 3 ;

[0024] Figure 6 Schematic diagram of the process of the virtual robot action determination method provided in the embodiment of the present application Figure 4 ;

[0025] Figure 7 A schematic diagram of the structure of a virtual robot action determination device provided in an embodiment of the present application;

[0026] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0028] In one embodiment of the present application, the virtual robot action determination method can be run on a local terminal device or a server. When the virtual robot action determination method is run on a server, the method can be implemented and executed based on a cloud interaction system, wherein the cloud interaction system includes a server and a client device.

[0029] In an optional implementation, various cloud applications can be run under the cloud interaction system, such as cloud games. Taking cloud games as an example, cloud games refer to a game mode based on cloud computing. In the operation mode of cloud games, the operating body of the game program and the main body of the game screen presentation are separated. The storage and operation of the virtual robot action determination method are completed on the cloud game server. The role of the client device is used for receiving and sending data and presenting the game screen. For example, the client device can be a display device with data transmission function close to the user side, such as a mobile terminal, a TV, a computer, a handheld computer, etc.; but the cloud game server in the cloud is used for information processing. When playing the game, the player operates the client device to send an operation instruction to the cloud game server. The cloud game server runs the game according to the operation instruction, encodes and compresses the game screen and other data, and returns it to the client device through the network. Finally, the client device decodes and outputs the game screen.

[0030] In an optional embodiment, taking a game as an example, a local terminal device stores a game program and is used to present a game screen. The local terminal device is used to interact with the player through a graphical user interface, that is, the game program is downloaded and installed by an electronic device and run conventionally. The local terminal device may provide the graphical user interface to the player in a variety of ways, for example, it may be rendered and displayed on a display screen of the terminal, or provided to the player through a holographic projection. For example, the local terminal device may include a display screen and a processor, the display screen is used to present a graphical user interface, the graphical user interface includes a game screen, and the processor is used to run the game, generate a graphical user interface, and control the display of the graphical user interface on the display screen.

[0031] Take a typical multiplayer online battle arena (MOBA) game as an example. The behaviors that need to be completed include: moving, attacking, teleporting (returning to the city), releasing skills, and using props. Different behaviors have different usage methods. For example, moving requires specifying the direction, attacking requires specifying the object, some skills require specifying the direction, and some skills require specifying the position, etc. If the position, direction, object, etc. are all used as the required parameters for each action, multiple parameters are required to describe any action, and the action space has multiple dimensions. In fact, most behaviors only require one of the above parameters, and the other parameters do not affect their performance. However, due to the obvious differences in the nature and form of position, direction, and object, the existing reinforcement neural network model cannot use one dimension to take into account different types of parameters. Therefore, multiple dimensions are used to output each parameter in turn, and finally form a high-dimensional action.

[0032] The reinforcement neural network model outputs the high-dimensional actions that the game character needs to collect, but there is a correlation problem between the high-dimensional actions and a lag in strategy changes. For example, in a certain scenario, as the model learns, the selected skill gradually changes from damage to treatment. However, the selected object is still the enemy at this time, which will hinder the skill change. To address this problem, an autoregressive method is provided, that is, the action output of the next dimension is determined by the action outputs of the previous dimensions. This method can ensure the correlation between the skill dimension and other dimensions. For example, whether the skill object is an enemy or a friendly depends on whether the previously selected action is treatment or damage. Each dimension corresponds to a reinforcement neural network model.

[0033] However, high-dimensional actions constitute a huge action space, reinforcement learning is difficult and the training speed is slow, and the game character may only perform actions based on the output of some dimensions. Therefore, there is a lot of redundancy in the high-dimensional action space. And because the output high-dimensional actions are redundant, it is impractical to independently check the legitimacy of high-dimensional actions. For example, due to the distance limit of the skill, the availability of a skill does not mean that the skill is currently available for a certain target. In addition, although autoregression can effectively transmit information from the front output head, it will bring higher computational costs.

[0034] Based on the above problems, the present application provides a method for determining the action of a virtual robot, which determines multiple candidate actions according to multiple action targets corresponding to executable behaviors, and then determines the action to be executed by the virtual robot to be controlled at the next moment from the multiple candidate actions, thereby compressing the high-dimensional action space, improving the training speed, and reducing the learning difficulty. It can also check the legitimacy of each action in combination with the corresponding rules, eliminate irrelevant redundant parameters, maintain good accuracy and sufficient degrees of freedom, and, in addition, regards the behavior and the corresponding action target together as an action feature, which provides richer information so that the model can learn the commonalities between behaviors of the same action target without the need to use autoregressive methods, thereby reducing the consumption of computing resources.

[0035] The virtual robot action determination method provided by the present application is described below in conjunction with several specific embodiments.

[0036] Figure 1 Schematic diagram of the process of the virtual robot action determination method provided in the embodiment of the present application Figure 1 , the execution subject of this embodiment is an electronic device. Figure 1 As shown, the method includes:

[0037] S101, obtaining executable behaviors of the virtual robot to be controlled in the game and game status information at the current moment.

[0038] The virtual robot to be controlled may be a virtual character, virtual animal, or other virtual robot used for playing with or hosting in a game, and this embodiment does not specifically limit this. The game may be, for example, a Multiplayer Online Battle Arena (MOBA) game.

[0039] The number of executable behaviors of the virtual robot to be controlled in the game includes but is not limited to one, wherein the executable behaviors may be, for example, attack behaviors, moving behaviors, etc. The behavioral attribute information of the executable behaviors includes: distance, range, etc. For attack behaviors, the behavioral attribute information may include the attack range, and for moving behaviors, the behavioral attribute information may include the moving distance.

[0040] The current game status information is used to indicate the current game status, and the game status information may include: game scene status information and virtual character status information, and the virtual character status information includes: status information of the virtual robot to be controlled and status information of other virtual characters except the virtual robot to be controlled. Among them, the game scene status information includes: game progress, game score, etc., the status information of the virtual robot to be controlled includes: skill information, health information, camp information, etc. of the virtual robot to be controlled, and the status information of other virtual characters includes: skill information, health information, camp information, etc. of other virtual characters.

[0041] S102: Determine multiple candidate actions according to the action targets corresponding to the executable behaviors.

[0042] In this step, an executable behavior may correspond to multiple action targets. The action target may be the target of the executable behavior. The candidate action is used to represent the game action formed by the executable behavior acting on the action target. Different candidate actions correspond to different action targets, and each candidate action corresponds to an action target.

[0043] The action target includes at least one of: a virtual robot to be controlled, a virtual character in the game other than the virtual robot to be controlled, a virtual model, a skill release direction, and a skill release position.

[0044] The action target is the virtual robot to be controlled, which means that the executable behavior acts on the virtual robot to be controlled (oneself), and the candidate actions can be executing auxiliary skills on oneself or applying virtual props to complete functions such as adding blood and adding magic points; the action target is a virtual character other than the virtual robot to be controlled in the game, which means that the executable behavior acts on virtual characters other than the virtual robot to be controlled, and the candidate actions can be executing attack skills or applying props on virtual characters other than the virtual robot to be controlled to complete virtual attacks on the enemy or assisting in adding blood for teammates; the action target is the skill release direction, which means that the executable behavior acts on the direction, and the candidate action can be executing an attack in a certain direction; the action target is the skill release position, which means that the execution behavior acts on the scene position, and the candidate action can be executing the skill towards a certain scene position.

[0045] Among them, virtual characters other than the virtual robot to be controlled may include: enemy heroes, enemy soldiers, teammate heroes, etc., virtual models may be enemy buildings, enemy supplies, etc., skill release directions may include: direction A, direction B, direction C, direction D, direction E, and the skill release position may be a specified coordinate position in the virtual scene. In an optional embodiment, the skill release position is a coordinate position determined according to the player's operation within the maximum release range allowed by the current skill.

[0046] In this division, continuous angle values ​​can be discretized into a finite number of directions, and directional behaviors can be divided according to the class of the action target. It should be noted that the number of executable behaviors includes but is not limited to one, and the same executable behavior can have multiple action targets at the same time. For example, the flash skill can have a certain direction as the action target, and can also have an enemy hero as the action target (i.e., flashing to the enemy hero).

[0047] For example, if the executable behavior is an attack behavior, the attack behavior can be divided into three candidate actions: attacking enemy soldiers, attacking enemy heroes, and attacking enemy buildings. If the executable behavior is a move behavior, the move behavior can be divided into three candidate actions: moving in direction A, moving in direction B, and moving in direction C.

[0048] Figure 2 A schematic diagram of the generation process of candidate actions provided in an embodiment of the present application is shown in FIG. Figure 2 As shown, the original behavior is an executable behavior. According to the multiple action targets corresponding to the original behavior, three candidate actions are determined, namely action 1, action 2, and action 3, each corresponding to a tuple, namely [behavior, target A], [behavior, target B], [behavior, target C], each [behavior, target]. The behavior in each tuple is the original behavior, which respectively represents the candidate action formed by applying the original behavior to direction A, the candidate action formed by applying the original behavior to direction B, and the candidate action formed by applying the original behavior to direction C.

[0049] Through the above behavior division, multiple [behavior, target] tuples can be obtained according to the action goals. A [behavior, target] tuple serves as an optional action of the virtual robot to be controlled in the action space. The final action space is composed of tuples, where the number of tuples is about 3 times the number of executable behaviors.

[0050] S103. Determine, based on the game state information at the current moment, the action to be performed by the virtual robot to be controlled at the next moment from a plurality of candidate actions.

[0051] Among them, the next moment can be the next moment after the current moment. According to the game status information at the current moment, the action to be executed by the virtual robot to be controlled at the next moment that matches the game status information can be determined from multiple candidate actions. That is, the action to be executed is screened out from multiple candidate actions based on the game status information at the current moment. For example, the game status information at the current moment indicates that the virtual robot to be controlled is executing an attack action on the enemy soldiers, and the enemy hero is acquiring the game resources of the camp to which the virtual robot to be controlled belongs. Then the action to be executed by the virtual robot to be controlled at the next moment may be to move to the enemy hero to prevent the enemy hero from acquiring the game resources.

[0052] In an optional embodiment, step S103, based on the game status information at the current moment, determines the action to be executed by the virtual robot to be controlled at the next moment from multiple candidate actions, including: based on the game status information at the current moment, predicting the behavior of the virtual robot to be controlled, and obtaining the execution probability of the virtual robot to be controlled for multiple candidate actions at the next moment; determining the action to be executed from multiple candidate actions based on the execution probabilities of multiple candidate actions.

[0053] Among them, according to the game status information at the current moment, the behavior of the virtual robot to be controlled can be predicted to obtain the execution probability of the virtual robot to be controlled for multiple candidate actions at the next moment, and then based on the execution probabilities of the multiple candidate actions, the candidate action whose execution probability meets the preset conditions is determined from the multiple candidate actions as the action to be executed, wherein the preset condition can be, for example, the maximum execution probability.

[0054] In some embodiments, a pre-trained reinforced neural network model can be used to process the game status information at the current moment to obtain the execution probabilities of multiple candidate actions, wherein the reinforced neural network model has a single-head output and a single-head input, the input is the game status information at the current moment, and the output is the execution probabilities of multiple candidate actions. Compared with the multi-head output high-dimensional actions in the prior art, the action space dimension is reduced, the training speed is increased, and the learning difficulty is reduced.

[0055] In the virtual robot action determination method of the present embodiment, the executable behavior of the virtual robot to be controlled in the game and the game state information at the current moment are obtained, and multiple candidate actions are determined according to multiple action targets corresponding to the executable behavior, wherein the candidate action is used to represent the game action formed by applying the executable behavior to the action target, and different candidate actions correspond to different action targets. According to the game state information at the current moment, the action to be executed by the virtual robot to be controlled at the next moment is determined from multiple candidate actions. Multiple candidate actions are determined through the executable behavior of the virtual robot to be controlled, and the action to be executed is determined from the multiple candidate actions, which compresses the action space and reduces the computing resource consumption of the virtual robot.

[0056] Figure 3 Schematic diagram of the process of the virtual robot action determination method provided in the embodiment of the present application Figure 2 ,like Figure 3 As shown, according to the game state information at the current moment, the action to be performed by the virtual robot to be controlled at the next moment is determined from multiple actions, including:

[0057] S201: Perform a legitimacy check on multiple candidate actions to obtain the check results of the multiple candidate actions.

[0058] S202: Filter multiple candidate actions according to detection results of multiple candidate actions to obtain filtered actions.

[0059] S203: Determine the action to be executed from the filtered actions according to the current game status information.

[0060] Before determining the action to be executed from multiple candidate actions, the multiple candidate actions can also be subjected to a legality check to obtain the detection results of the multiple candidate actions, the detection results including legal actions or illegal actions, and then based on the detection results of the multiple candidate actions, the illegal actions in the multiple candidate actions are filtered out to obtain filtered actions, that is, the filtered actions are legal actions.

[0061] Then, based on the game status information at the current moment, the action to be executed is determined from the filtered actions. That is to say, based on the game status information at the current moment, the action to be executed can be determined from the legal actions to ensure that the action to be executed is a legal action and is related to the game status information at the current moment.

[0062] In an optional embodiment, based on the game status information at the current moment, the action to be executed by the virtual robot to be controlled at the next moment is determined from the filtered actions, including: based on the game status information at the current moment, the behavior of the virtual robot to be controlled is predicted to obtain the execution probability of the virtual robot to be controlled for the filtered actions at the next moment; based on the execution probability of the filtered actions, the action to be executed is determined from the filtered actions.

[0063] In this step, the behavior of the robot to be controlled can be predicted based on the game status information at the current moment, and the execution probability of the virtual robot to be controlled for the filtered action at the next moment can be obtained. Then, based on the execution probability of the filtered action, the action to be executed is determined from the filtered actions. The pre-trained reinforced neural network model can be used to process the game status information at the current moment to obtain the execution probability of the filtered action, and the action whose execution probability meets the preset condition is determined from the filtered actions as the action to be executed, wherein the preset condition can be, for example, the maximum execution probability.

[0064] Figure 4 A schematic diagram of a reinforced neural network model provided in an embodiment of the present application, such as Figure 4The multiple candidate actions shown are all represented by corresponding [behavior, target] tuples, and the tuples of multiple candidate actions constitute action features. The game state information at the current moment is input into the pre-trained reinforcement neural network model, and the output features of the reinforcement neural network model are used to perform dot product (dot) with the action features to obtain the execution probabilities of multiple candidate actions. Then, the probability weight of each candidate action is obtained through the action legality detection unit, and the adjusted execution probability of each candidate action is obtained based on the probability weight and the corresponding execution probability. Among them, the probability weight of the legal action can be 1, and the probability weight of the illegal action can be 0, so that irrelevant redundant parameters can be eliminated.

[0065] In this example, an embedding algorithm or one-hot encoding can be used to encode the behavior (SkillEmbedding) and action target (Target embedding) in the tuple, where different actions with the same action target have the same corresponding Target embedding.

[0066] In the virtual robot action determination method of this embodiment, multiple candidate actions are subjected to a legitimacy check to obtain the detection results of multiple candidate actions, and multiple candidate actions are filtered according to the detection results of multiple candidate actions to obtain filtered actions, and the action to be executed is determined from the filtered actions according to the game state information at the current moment. High-dimensional action spaces are difficult to statistically and visually analyze. After reducing the dimension of the action space, the execution probability of multiple candidate actions can be directly counted, and the legitimacy of multiple candidate actions can be checked, which can reduce the proportion of invalid actions and further improve the model learning speed.

[0067] Figure 5 Schematic diagram of the process of the virtual robot action determination method provided in the embodiment of the present application Figure 3 ,like Figure 5 As shown, multiple candidate actions are subjected to legality detection to obtain detection results of multiple candidate actions, including:

[0068] S301: Determine whether the executable behavior corresponding to each candidate action is available at the current moment.

[0069] It is determined whether the executable behavior corresponding to each candidate action is available at the current moment. If the executable behavior corresponding to the candidate action is available at the current moment, step S302 is executed; if the executable behavior corresponding to the candidate action is not available at the current moment, step S305 is executed.

[0070] It should be noted that the executable behavior is the behavior that the virtual robot to be controlled can execute in the game. At different times, the executable behavior may be available or unavailable, which can be determined based on the attribute information of the virtual robot to be controlled and the attribute information of the executable behavior. If the attribute information of the virtual robot to be controlled at the current moment meets the attribute information of the executable behavior, it is determined that the executable behavior is available; if the attribute information of the virtual robot to be controlled at the current moment does not meet the attribute information of the executable behavior, it is determined that the executable behavior is unavailable.

[0071] For example, the attribute information of the executable behavior is: magic value 100, and the attribute information of the virtual robot to be controlled at the current moment is: magic value 50. Since the magic value of the virtual robot to be controlled at the current moment does not reach the magic value required for the executable behavior, it means that the executable behavior is not available at the current moment. If the attribute information of the virtual robot to be controlled is: magic value 100, it means that the executable behavior is available at the current moment.

[0072] S302: If the executable behavior corresponding to the candidate action is available, determine whether the executable behavior corresponding to the candidate action can act on the action target of the candidate action at the current moment.

[0073] If the executable behavior corresponding to the candidate action is available, determine whether the executable behavior corresponding to the candidate action can act on the action target of the candidate action at the current moment; if the executable behavior corresponding to the candidate action can act on the action target of the candidate action at the current moment, execute step S303; if the executable behavior corresponding to the candidate action cannot act on the action target of the candidate action at the current moment, execute step S304.

[0074] In some embodiments, determining whether the executable behavior corresponding to the candidate action can act on the action target of the candidate action at the current moment includes: determining whether the behavior parameters of the candidate action can be generated at the current moment; if the behavior parameters of the candidate action can be generated, and the action target of the candidate action at the current moment is an actable target, then determining that the executable behavior corresponding to the candidate action can act on the action target of the candidate action at the current moment; if the behavior parameters of the candidate action cannot be generated, or if the behavior parameters of the candidate action can be generated, and the action target of the candidate action at the current moment is not an actable target, then determining that the executable behavior corresponding to the candidate action cannot act on the action target of the candidate action at the current moment.

[0075] Among them, the behavior parameters of the candidate action may be execution parameters when executing the candidate action. For example, if the candidate action is to attack at a certain angle, the corresponding behavior parameter is the angle value; for another example, if the candidate action is to attack at a certain coordinate, the corresponding behavior parameter is the coordinate value; for another example, if the candidate action is to attack at an enemy hero, the corresponding behavior parameter is the identity of the enemy hero.

[0076] Determining whether to generate the behavior parameters of the candidate action at the current moment may include: determining whether there is an action target of the candidate action within the attack range of the candidate action at the current moment, if so, the behavior parameters of the candidate action may be generated, if not, the behavior parameters of the candidate action may not be generated. Taking the candidate action of attacking an enemy hero as an example, if no enemy hero is within the attack range of the candidate action at the current moment, it is determined that the behavior parameters of the candidate action cannot be generated.

[0077] If the behavior parameters of the candidate action can be generated, and the action target of the candidate action at the current moment is an actionable target, then it is determined that the executable behavior corresponding to the candidate action can act on the action target of the candidate action at the current moment, where the actionable target can be a target on which the behavior can act. For example, moving in the 45° direction in the game scene, the action target is the 45° direction. Since the 45° direction in the game scene is an impassable obstacle, it means that the action target is not an actionable target.

[0078] If the behavior parameters of the candidate action cannot be generated, it is determined that the executable behavior corresponding to the candidate action cannot act on the action target of the candidate action at the current moment; or if the behavior parameters of the candidate action can be generated and the action target of the candidate action at the current moment is not an actable target, it is determined that the executable behavior corresponding to the candidate action cannot act on the action target of the candidate action at the current moment.

[0079] S303: If the executable behavior corresponding to the candidate action can act on the action target of the candidate action at the current moment, determine that the detection result of the candidate action is a legal action.

[0080] If the executable behavior corresponding to the candidate action can act on the action target of the candidate action at the current moment, the detection result of the candidate action is determined to be a legal action, that is, the legal action satisfies both that the executable behavior is available and the executable behavior can act on the action target.

[0081] S304: If the executable behavior corresponding to the candidate action cannot act on the action target of the candidate action at the current moment, determine that the detection result of the candidate action is an illegal action.

[0082] If the executable behavior corresponding to the candidate action cannot act on the action target of the candidate action at the current moment, the detection result of the candidate action is determined to be an illegal action, that is, when the executable behavior is available, the executable behavior cannot act on the action target, and the candidate action is an illegal action.

[0083] S305: If the executable behavior corresponding to the candidate action is not available, determine that the detection result of the candidate action is an illegal action.

[0084] If the executable behavior corresponding to the candidate action is not available, it is impossible to determine whether the executable behavior can act on the action target, and the detection result of the candidate action is directly determined to be an illegal action.

[0085] The legality of all actions is judged with a small amount of operations. Compared with the existing technology, there is no need to traverse the entire high-dimensional action space. For example, a game character has N behaviors, each behavior is split into c*N actions, and each action generates 1 parameter. The operations required are: 1 behavior splitting, c behavior parameter generation, c legality judgment, and a total of N+2*c*N operations to check the legality of all actions. The existing technology needs to traverse and search the high-dimensional action space, which is inefficient.

[0086] Moreover, providing legal actions to the reinforced neural network model can ensure that the model only selects legal behaviors, which significantly improves the efficiency of model output. In the existing technology, legality judgment can only be performed after the model outputs behavior parameters, so more redundant behaviors are output with a certain lag. In addition, legality judgment can provide some additional information to the model. The model can obtain valuable status information from whether some actions are legal. For example, the action of attacking an enemy hero is legal, which means that an enemy hero is within the attack range of the action.

[0087] Figure 6 Schematic diagram of the process of the virtual robot action determination method provided in the embodiment of the present application Figure 4 ,like Figure 6 As shown, the method also includes:

[0088] S401: Obtain behavior parameters of the action to be executed.

[0089] S402: Control the virtual robot to execute the action to be executed according to the behavior parameters.

[0090] Among them, the behavior parameters of the action to be executed are the execution parameters when executing the candidate action. For example, if the candidate action is to attack at a certain angle, the corresponding behavior parameter is the angle value; for another example, if the candidate action is to attack at a certain coordinate, the corresponding behavior parameter is the coordinate value; for another example, if the candidate action is to attack a certain enemy hero, the corresponding behavior parameter is the identity of the enemy hero.

[0091] After determining the action to be performed by the virtual robot at the next moment, the behavior parameters of the action to be performed are obtained, and then the virtual robot is controlled to perform the action to be performed according to the behavior parameters.

[0092] In an optional implementation, obtaining behavior parameters of the action to be performed includes:

[0093] It is determined whether the category of the behavior parameter of the action to be executed is consistent with the category of the action target of the action to be executed; if they are consistent, the behavior parameter of the action to be executed is determined according to the action target of the action to be executed.

[0094] The category of the behavior parameter is used to characterize the parameter type of the behavior parameter, for example, it may be an angle, a position coordinate, or an identity identifier.

[0095] The category of the action target is used to characterize the type of the action target, for example, it may be an angle, a position coordinate, or an identity identifier.

[0096] If the category of the behavior parameter of the action to be executed is consistent with the category of the action target of the action to be executed, the behavior parameter of the action to be executed can be determined based on the action target of the action to be executed. For example, the action target is 0°, the category of the action target is angle, and the behavior parameter category corresponding to direction A is angle. Therefore, the category of the behavior parameter is consistent with the category of the action target, then the behavior parameter of the action to be executed is determined to be the parameter of the action target, that is, 0°.

[0097] In some embodiments, if the category of the behavior parameters of the action to be performed and the category of the action target of the action to be performed are inconsistent, it is determined whether the action target of the action to be performed is the virtual robot to be controlled; if the action target of the action to be performed is the virtual robot to be controlled, the behavior parameters of the action to be performed are determined to be preset empty parameters; if the action target of the action to be performed is not the virtual robot to be controlled, the behavior parameters of the action to be performed are determined based on the relative position relationship between the action target of the action to be performed and the virtual robot to be controlled at the current moment.

[0098] If the category of the behavior parameter of the action to be executed is inconsistent with the category of the action target of the action to be executed, it is determined whether the action target of the action to be executed is the virtual robot to be controlled, wherein the category of the action target may be an identity identifier, and whether it is the robot to be controlled may be determined based on the identity identifier; if the action target of the action to be executed is the virtual robot to be controlled, it is determined that the behavior parameter of the action to be executed is a preset empty parameter, that is, if the action to be executed is executed towards oneself, the behavior parameter is an empty parameter.

[0099] If the action target of the action to be executed is not the virtual robot to be controlled, the behavior parameters of the action to be executed are determined according to the relative position relationship between the action target of the action to be executed and the virtual robot to be controlled at the current moment. For example, if the action target is an enemy hero and the behavior parameter is the angle of the enemy hero relative to the virtual robot to be controlled, the angle of the enemy hero relative to the robot to be controlled can be determined according to the relative position relationship between the action target of the action to be executed (such as the enemy hero) and the virtual robot to be controlled at the current moment.

[0100] Figure 7 This is a schematic diagram of the structure of a virtual robot action determination device provided in an embodiment of the present application, which can be integrated into an electronic device. Figure 7 As shown, the device comprises:

[0101] The acquisition module 501 is used to acquire the executable behaviors of the virtual robot to be controlled in the game and the game state information at the current moment;

[0102] A determination module 502 is used to determine multiple candidate actions according to multiple action targets corresponding to the executable behavior, wherein the candidate actions are used to represent game actions formed by applying the executable behavior to the action target, and different candidate actions correspond to different action targets;

[0103] The determination module 502 is further used to determine the action to be performed by the virtual robot to be controlled at the next moment from multiple candidate actions according to the game state information at the current moment.

[0104] In an optional implementation, the determination module 502 is specifically configured to:

[0105] According to the game state information at the current moment, the behavior of the virtual robot to be controlled is predicted to obtain the execution probability of the virtual robot to be controlled for multiple candidate actions at the next moment;

[0106] According to the execution probabilities of the multiple actions, an action to be executed is determined from the multiple candidate actions.

[0107] In an optional implementation, the determination module 502 is specifically configured to:

[0108] Performing legitimacy detection on multiple candidate actions to obtain detection results of multiple candidate actions;

[0109] According to the detection results of the multiple candidate actions, the multiple candidate actions are filtered to obtain filtered actions;

[0110] According to the game status information at the current moment, the action to be executed is determined from the filtered actions.

[0111] In an optional implementation, the determination module 502 is specifically configured to:

[0112] According to the game status information at the current moment, the behavior of the virtual robot to be controlled is predicted to obtain the execution probability of the virtual robot to be controlled for the filtered action at the next moment;

[0113] According to the execution probabilities of the filtered actions, the actions to be executed are determined from the filtered actions.

[0114] In an optional implementation, the determination module 502 is specifically configured to:

[0115] Determine whether the executable behavior corresponding to each candidate action is available at the current moment;

[0116] If the executable behavior corresponding to the candidate action is available at the current moment, then determine whether the executable behavior corresponding to the candidate action can act on the action target of the candidate action at the current moment;

[0117] If the executable behavior corresponding to the candidate action can act on the action target of the candidate action at the current moment, the detection result of the candidate action is determined to be a legal action.

[0118] In an optional implementation, the determination module 502 is specifically configured to:

[0119] If the executable behavior corresponding to the candidate action cannot act on the action target of the candidate action at the current moment, the detection result of the candidate action is determined to be an illegal action.

[0120] In an optional implementation, the determination module 502 is specifically configured to:

[0121] If the executable behavior corresponding to the candidate action is not available at the current moment, the detection result of the candidate action is determined to be an illegal action.

[0122] In an optional implementation, the determination module 502 is specifically configured to:

[0123] Determine whether the behavior parameters of the candidate action can be generated at the current moment;

[0124] If the behavior parameters of the candidate action can be generated, and the action target of the candidate action at the current moment is an actionable target, then determine that the executable behavior corresponding to the candidate action can act on the action target of the candidate action at the current moment;

[0125] If the behavior parameters of the candidate action cannot be generated, or if the behavior parameters of the candidate action can be generated and the action target of the candidate action at the current moment is not an actionable target, it is determined that the executable behavior corresponding to the candidate action cannot act on the action target of the candidate action at the current moment.

[0126] In an optional implementation, the acquisition module 501 is further configured to:

[0127] Get the behavior parameters of the action to be executed;

[0128] The control module 503 is also used to control the virtual robot to execute the action to be executed according to the behavior parameters.

[0129] In an optional implementation, the acquisition module 501 is specifically configured to:

[0130] Determine whether the category of the behavior parameter of the action to be executed is consistent with the category of the action target of the action to be executed;

[0131] If they are consistent, the behavior parameters of the action to be executed are determined according to the action target of the action to be executed.

[0132] In an optional implementation, the acquisition module 501 is specifically configured to:

[0133] If they are inconsistent, determining whether the action target of the action to be executed is the virtual robot to be controlled;

[0134] If the action target of the action to be executed is the virtual robot to be controlled, determining the behavior parameter of the action to be executed is a preset empty parameter;

[0135] If the action target of the action to be executed is not the virtual robot to be controlled, the behavior parameters of the action to be executed are determined according to the relative position relationship between the action target of the action to be executed and the virtual robot to be controlled at the current moment.

[0136] In an optional embodiment, the action target includes: at least one of: a virtual robot to be controlled, a virtual character in the game other than the virtual robot to be controlled, a virtual model, a skill release direction, and a skill release position.

[0137] In the game character action device of this embodiment, the acquisition module is used to obtain the executable behavior of the virtual robot to be controlled in the game and the game status information at the current moment, and the determination module is used to determine multiple candidate actions according to multiple action targets corresponding to the executable behavior, wherein the candidate action is used to represent the game action formed by applying the executable behavior to the action target, and different candidate actions correspond to different action targets. The determination module is also used to determine the action to be executed by the virtual robot to be controlled at the next moment from multiple candidate actions according to the game status information at the current moment. By determining multiple candidate actions through the executable behavior of the virtual robot to be controlled, and determining the action to be executed from multiple candidate actions, the action space is compressed and the computing resource consumption of the virtual robot is reduced.

[0138] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application, such as Figure 8As shown, the device includes: a processor 601, a memory 602 and a bus 603, the memory 602 stores machine-readable instructions executable by the processor 601, when the electronic device is running, the processor 601 communicates with the memory 602 through the bus 603, and the processor 601 executes the machine-readable instructions to perform the following steps:

[0139] Obtain the executable behaviors of the virtual robot to be controlled in the game and the current game status information;

[0140] Determine multiple candidate actions according to multiple action targets corresponding to the executable behavior, wherein the candidate actions are used to represent game actions formed by applying the executable behavior to the action target, and different candidate actions correspond to different action targets;

[0141] According to the game state information at the current moment, the action to be performed by the virtual robot to be controlled at the next moment is determined from multiple candidate actions.

[0142] In an optional implementation, when the processor 601 determines the action to be performed by the virtual robot to be controlled at the next moment from multiple candidate actions according to the game state information at the current moment, it is specifically used to:

[0143] According to the game state information at the current moment, the behavior of the virtual robot to be controlled is predicted to obtain the execution probability of the virtual robot to be controlled for multiple candidate actions at the next moment;

[0144] According to the execution probabilities of the multiple actions, an action to be executed is determined from the multiple candidate actions.

[0145] In an optional implementation, when the processor 601 determines the action to be performed by the virtual robot to be controlled at the next moment from multiple candidate actions according to the game state information at the current moment, it is specifically used to:

[0146] Performing legitimacy detection on multiple candidate actions to obtain detection results of multiple candidate actions;

[0147] According to the detection results of the multiple candidate actions, the multiple candidate actions are filtered to obtain filtered actions;

[0148] According to the game status information at the current moment, the action to be executed is determined from the filtered actions.

[0149] In an optional implementation, when the processor 601 determines the action to be performed by the virtual robot to be controlled at the next moment from the filtered actions according to the game state information at the current moment, it is specifically used to:

[0150] According to the game status information at the current moment, the behavior of the virtual robot to be controlled is predicted to obtain the execution probability of the virtual robot to be controlled for the filtered action at the next moment;

[0151] According to the execution probabilities of the filtered actions, the actions to be executed are determined from the filtered actions.

[0152] In an optional implementation manner, when the processor 601 performs a legitimacy check on multiple candidate actions and obtains the detection results of the multiple candidate actions, it is specifically configured to:

[0153] Determine whether the executable behavior corresponding to each candidate action is available at the current moment;

[0154] If the executable behavior corresponding to the candidate action is available at the current moment, then determine whether the executable behavior corresponding to the candidate action can act on the action target of the candidate action at the current moment;

[0155] If the executable behavior corresponding to the candidate action can act on the action target of the candidate action at the current moment, the detection result of the candidate action is determined to be a legal action.

[0156] In an optional implementation manner, when the processor 601 performs a legitimacy check on multiple candidate actions and obtains the detection results of the multiple candidate actions, it is specifically configured to:

[0157] If the executable behavior corresponding to the candidate action cannot act on the action target of the candidate action at the current moment, the detection result of the candidate action is determined to be an illegal action.

[0158] In an optional implementation manner, when the processor 601 performs a legitimacy check on multiple candidate actions and obtains the detection results of the multiple candidate actions, it is specifically configured to:

[0159] If the executable behavior corresponding to the candidate action is not available at the current moment, the detection result of the candidate action is determined to be an illegal action.

[0160] In an optional implementation, when the processor 601 determines whether the executable behavior corresponding to the candidate action can act on the action target of the candidate action at the current moment, it is specifically configured to:

[0161] Determine whether the behavior parameters of the candidate action can be generated at the current moment;

[0162] If the behavior parameters of the candidate action can be generated, and the action target of the candidate action at the current moment is an actionable target, then determine that the executable behavior corresponding to the candidate action can act on the action target of the candidate action at the current moment;

[0163] If the behavior parameters of the candidate action cannot be generated, or if the behavior parameters of the candidate action can be generated and the action target of the candidate action at the current moment is not an actionable target, it is determined that the executable behavior corresponding to the candidate action cannot act on the action target of the candidate action at the current moment.

[0164] In an optional implementation, the processor 601 is further configured to:

[0165] Get the behavior parameters of the action to be executed;

[0166] According to the behavior parameters, the virtual robot is controlled to execute the actions to be executed.

[0167] In an optional implementation, when the processor 601 executes the process of obtaining the behavior parameter of the action to be executed, it is specifically configured to:

[0168] Determine whether the category of the behavior parameter of the action to be executed is consistent with the category of the action target of the action to be executed;

[0169] If they are consistent, the behavior parameters of the action to be executed are determined according to the action target of the action to be executed.

[0170] In an optional implementation, when the processor 601 executes the process of obtaining the behavior parameter of the action to be executed, it is specifically configured to:

[0171] If they are inconsistent, determining whether the action target of the action to be executed is the virtual robot to be controlled;

[0172] If the action target of the action to be executed is the virtual robot to be controlled, determining the behavior parameter of the action to be executed is a preset empty parameter;

[0173] If the action target of the action to be executed is not the virtual robot to be controlled, the behavior parameters of the action to be executed are determined according to the relative position relationship between the action target of the action to be executed and the virtual robot to be controlled at the current moment.

[0174] In an optional embodiment, the action target includes: at least one of: a virtual robot to be controlled, a virtual character in the game other than the virtual robot to be controlled, a virtual model, a skill release direction, and a skill release position.

[0175] Through the above method, the executable behavior of the virtual robot to be controlled in the game and the game status information at the current moment are obtained, and multiple candidate actions are determined according to the multiple action targets corresponding to the executable behavior, wherein the candidate action is used to represent the game action formed by applying the executable behavior to the action target, and different candidate actions correspond to different action targets. According to the game status information at the current moment, the action to be executed by the virtual robot to be controlled at the next moment is determined from multiple candidate actions. In the present application, multiple candidate actions are determined by the executable behavior of the virtual robot to be controlled, and the action to be executed is determined from multiple candidate actions, which compresses the action space and reduces the computing resource consumption of the virtual robot.

[0176] The embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. The computer program is executed when a processor is running, and the processor performs the following steps:

[0177] Obtain the executable behaviors of the virtual robot to be controlled in the game and the current game status information;

[0178] Determine multiple candidate actions according to multiple action targets corresponding to the executable behavior, wherein the candidate actions are used to represent game actions formed by applying the executable behavior to the action target, and different candidate actions correspond to different action targets;

[0179] According to the game state information at the current moment, the action to be performed by the virtual robot to be controlled at the next moment is determined from multiple candidate actions.

[0180] In an optional implementation, when the processor determines the action to be performed by the virtual robot to be controlled at the next moment from multiple candidate actions according to the game state information at the current moment, it is specifically used to:

[0181] According to the game state information at the current moment, the behavior of the virtual robot to be controlled is predicted to obtain the execution probability of the virtual robot to be controlled for multiple candidate actions at the next moment;

[0182] According to the execution probabilities of the multiple actions, an action to be executed is determined from the multiple candidate actions.

[0183] In an optional implementation, when the processor determines the action to be performed by the virtual robot to be controlled at the next moment from multiple actions according to the game state information at the current moment, it is specifically used to:

[0184] Performing legitimacy detection on multiple candidate actions to obtain detection results of multiple candidate actions;

[0185] According to the detection results of the multiple candidate actions, the multiple candidate actions are filtered to obtain filtered actions;

[0186] According to the game status information at the current moment, the action to be executed is determined from the filtered actions.

[0187] In an optional implementation, when the processor determines the action to be performed by the virtual robot to be controlled at the next moment from the filtered actions according to the game state information at the current moment, it is specifically used to:

[0188] According to the game status information at the current moment, the behavior of the virtual robot to be controlled is predicted to obtain the execution probability of the virtual robot to be controlled for the filtered action at the next moment;

[0189] According to the execution probabilities of the filtered actions, the actions to be executed are determined from the filtered actions.

[0190] In an optional implementation manner, when the processor performs a legitimacy check on multiple candidate actions and obtains the detection results of the multiple candidate actions, it is specifically used to:

[0191] Determine whether the executable behavior corresponding to each candidate action is available at the current moment;

[0192] If the executable behavior corresponding to the candidate action is available at the current moment, then determine whether the executable behavior corresponding to the candidate action can act on the action target of the candidate action at the current moment;

[0193] If the executable behavior corresponding to the candidate action can act on the action target of the candidate action at the current moment, the detection result of the candidate action is determined to be a legal action.

[0194] In an optional implementation manner, when the processor performs a legitimacy check on multiple candidate actions and obtains the detection results of the multiple candidate actions, it is specifically used to:

[0195] If the executable behavior corresponding to the candidate action cannot act on the action target of the candidate action at the current moment, the detection result of the candidate action is determined to be an illegal action.

[0196] In an optional implementation manner, when the processor performs a legitimacy check on multiple candidate actions and obtains the detection results of the multiple candidate actions, it is specifically used to:

[0197] If the executable behavior corresponding to the candidate action is not available at the current moment, the detection result of the candidate action is determined to be an illegal action.

[0198] In an optional implementation, when the processor determines whether the executable behavior corresponding to the candidate action can act on the action target of the candidate action at the current moment, it is specifically configured to:

[0199] Determine whether the behavior parameters of the candidate action can be generated at the current moment;

[0200] If the behavior parameters of the candidate action can be generated, and the action target of the candidate action at the current moment is an actionable target, then determine that the executable behavior corresponding to the candidate action can act on the action target of the candidate action at the current moment;

[0201] If the behavior parameters of the candidate action cannot be generated, or if the behavior parameters of the candidate action can be generated and the action target of the candidate action at the current moment is not an actionable target, it is determined that the executable behavior corresponding to the candidate action cannot act on the action target of the candidate action at the current moment.

[0202] In an optional implementation, the processor is further configured to:

[0203] Get the behavior parameters of the action to be executed;

[0204] According to the behavior parameters, the virtual robot is controlled to execute the actions to be executed.

[0205] In an optional implementation manner, when the processor executes the step of obtaining the behavior parameter of the action to be executed, the processor is specifically configured to:

[0206] Determine whether the category of the behavior parameter of the action to be executed is consistent with the category of the action target of the action to be executed;

[0207] If they are consistent, the behavior parameters of the action to be executed are determined according to the action target of the action to be executed.

[0208] In an optional implementation manner, when the processor executes the step of obtaining the behavior parameter of the action to be executed, the processor is specifically configured to:

[0209] If they are inconsistent, determining whether the action target of the action to be executed is the virtual robot to be controlled;

[0210] If the action target of the action to be executed is the virtual robot to be controlled, determining the behavior parameter of the action to be executed is a preset empty parameter;

[0211] If the action target of the action to be executed is not the virtual robot to be controlled, the behavior parameters of the action to be executed are determined according to the relative position relationship between the action target of the action to be executed and the virtual robot to be controlled at the current moment.

[0212] In an optional embodiment, the action target includes: at least one of: a virtual robot to be controlled, a virtual character in the game other than the virtual robot to be controlled, a virtual model, a skill release direction, and a skill release position.

[0213] In the embodiment of the present application, the computer program can also execute other machine-readable instructions when run by the processor to execute other methods described in the embodiment. For the specific execution method steps and principles, please refer to the description of the embodiment, which will not be repeated here.

[0214] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0215] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0216] In addition, each functional unit in the embodiments provided in the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0217] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0218] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0219] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The protection scope of the present application is not limited thereto. Although the present application is described in detail with reference to the above-mentioned embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-mentioned embodiments within the technical scope disclosed in the present application, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application. They should all be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A method for determining a virtual robot action, characterized in that: include: Obtain the executable behaviors of the virtual robot to be controlled in the game and the current game status information; Determine multiple candidate actions according to multiple action targets corresponding to the executable behavior, wherein the candidate actions are used to represent game actions formed by applying the executable behavior to the action targets, and different candidate actions correspond to different action targets, and the action targets include: at least one of the virtual robot to be controlled, a virtual character in the game other than the virtual robot to be controlled, a virtual model, a skill release direction, and a skill release position; Determining, according to the game state information at the current moment, the action to be performed by the virtual robot to be controlled at the next moment from the multiple candidate actions; The step of determining the action to be performed by the virtual robot to be controlled at the next moment from the plurality of candidate actions according to the game state information at the current moment comprises: Performing a legitimacy check on the multiple candidate actions to obtain the check results of the multiple candidate actions; Filtering the multiple candidate actions according to the detection results of the multiple candidate actions to obtain filtered actions; According to the game status information at the current moment, the action to be executed is determined from the filtered actions.

2. The method according to claim 1, characterized in that The step of determining the action to be performed by the virtual robot to be controlled at the next moment from the plurality of candidate actions according to the game state information at the current moment comprises: According to the game state information at the current moment, predict the behavior of the virtual robot to be controlled to obtain the execution probability of the virtual robot to be controlled for the multiple candidate actions at the next moment; The action to be executed is determined from the multiple candidate actions according to the execution probabilities of the multiple actions.

3. The method according to claim 1, characterized in that The step of determining the action to be executed from the filtered actions according to the game state information at the current moment includes: According to the game state information at the current moment, the behavior of the virtual robot to be controlled is predicted to obtain the execution probability of the virtual robot to be controlled for the filtered action at the next moment; The to-be-executed action is determined from the filtered actions according to the execution probabilities of the filtered actions.

4. The method according to claim 2, characterized in that: The performing of a legality check on the plurality of candidate actions to obtain the detection results of the plurality of candidate actions includes: Determine whether the executable behavior corresponding to each candidate action is available at the current moment; If the executable behavior corresponding to the candidate action is available at the current moment, determining whether the executable behavior corresponding to the candidate action can act on the action target of the candidate action at the current moment; If the executable behavior corresponding to the candidate action can act on the action target of the candidate action at the current moment, the detection result of the candidate action is determined to be a legal action.

5. The method according to claim 4, characterized in that The performing of a legitimacy detection on the plurality of candidate actions to obtain detection results of the plurality of candidate actions further includes: If the executable behavior corresponding to the candidate action cannot act on the action target of the candidate action at the current moment, the detection result of the candidate action is determined to be an illegal action.

6. The method according to claim 4, characterized in that The performing of a legitimacy detection on the plurality of candidate actions to obtain detection results of the plurality of candidate actions further includes: If the executable behavior corresponding to the candidate action is not available at the current moment, the detection result of the candidate action is determined to be an illegal action.

7. The method according to claim 4, characterized in that The determining whether the executable behavior corresponding to the candidate action can act on the action target of the candidate action at the current moment includes: Determining whether the behavior parameter of the candidate action can be generated at the current moment; If the behavior parameter of the candidate action can be generated, and the action target of the candidate action at the current moment is an actionable target, determining that the executable behavior corresponding to the candidate action can act on the action target of the candidate action at the current moment; If the behavior parameters of the candidate action cannot be generated, or if the behavior parameters of the candidate action can be generated and the action target of the candidate action at the current moment is not the actionable target, then it is determined that the executable behavior corresponding to the candidate action cannot be acted on the action target of the candidate action at the current moment.

8. The method according to claim 1, characterized in that The method further comprises: Obtaining behavior parameters of the action to be executed; According to the behavior parameters, the virtual robot is controlled to execute the action to be executed.

9. The method according to claim 8, characterized in that The step of obtaining the behavior parameter of the action to be executed includes: Determining whether the category of the behavior parameter of the to-be-performed action is consistent with the category of the action target of the to-be-performed action; If they are consistent, the behavior parameters of the action to be executed are determined according to the action target of the action to be executed.

10. The method according to claim 9, characterized in that The step of obtaining the behavior parameters of the action to be executed further includes: If they are inconsistent, determining whether the action target of the to-be-executed action is the to-be-controlled virtual robot; If the action target of the to-be-performed action is the to-be-controlled virtual robot, determining that the behavior parameter of the to-be-performed action is a preset empty parameter; If the action target of the action to be performed is not the virtual robot to be controlled, the behavior parameters of the action to be performed are determined according to the relative position relationship between the action target of the action to be performed and the virtual robot to be controlled at the current moment.

11. A virtual robot action determination device, characterized in that: include: An acquisition module is used to acquire the executable behaviors of the virtual robot to be controlled in the game and the game status information at the current moment; A determination module, configured to determine a plurality of candidate actions according to a plurality of action targets corresponding to the executable behavior, wherein the candidate actions are used to represent game actions formed by applying the executable behavior to the action targets, and different candidate actions correspond to different action targets, and the action targets include: at least one of the virtual robot to be controlled, a virtual character in the game other than the virtual robot to be controlled, a virtual model, a skill release direction, and a skill release position; The determination module is further used to determine the action to be performed by the virtual robot to be controlled at the next moment from the multiple candidate actions according to the game state information at the current moment; The determination module is specifically used for: Performing a legitimacy check on the multiple candidate actions to obtain the check results of the multiple candidate actions; Filtering the multiple candidate actions according to the detection results of the multiple candidate actions to obtain filtered actions; According to the game status information at the current moment, the action to be executed is determined from the filtered actions.

12. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate through the bus, and the processor executes the machine-readable instructions to execute the virtual robot action determination method according to any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for determining a virtual robot action according to any one of claims 1 to 10 is executed.

Images