Model training method and device, virtual object control method and device, equipment and storage medium

Abstract

The invention provides a model training method and device, a virtual object control method and device, equipment and a storage medium, and relates to the technical field of data processing. The method comprises the steps of obtaining a target instruction sent by a first intelligent virtual object in a first camp; determining a target action of a second intelligent virtual object in the first camp by adopting a preset decision model according to the target instruction; controlling the second intelligent virtual object to execute the target action to obtain an action execution result of the second intelligent virtual object; and training the preset decision model according to the action execution result to obtain a target decision model. Compared with the prior art, the problem that man-machine cooperation is not good enough is avoided.

Description

Technical Field

[0001] This application relates to the field of data processing technologies, and more specifically, to a method, apparatus, device, and storage medium for model training and controlling virtual objects. Background Art

[0002] With the continuous development of video games, multiplayer battle games have become a widely popular game type.

[0003] In multiplayer battle games, players usually need to form teams to fight against other players. However, in the game, it is inevitable to encounter some players with insufficient skills or unwilling to communicate and cooperate, which has a great impact on the game experience. To solve this problem, Artificial Intelligence (AI) technology has been introduced into multiplayer battle games to simulate an intelligent game character as a player's teammate.

[0004] However, the current AI teammates perform poorly in the game, especially in terms of human-machine cooperation, and there are problems with insufficient human-machine cooperation. Summary of the Invention

[0005] The purpose of this application is to provide a method, apparatus, device, and storage medium for model training and controlling virtual objects to solve the problem of insufficient human-machine cooperation in the prior art in view of the above deficiencies in the prior art.

[0006] To achieve the above purpose, the technical solutions adopted in the embodiments of this application are as follows:

[0007] In a first aspect, an embodiment of this application provides a decision model training method applied to a test battle scenario, where the test battle scenario includes a first camp, and the method includes:

[0008] Obtain a target instruction sent by a first intelligent virtual object in the first camp;

[0009] According to the target instruction, use a preset decision model to determine a target action of a second intelligent virtual object in the first camp;

[0010] Control the second intelligent virtual object to execute the target action to obtain an action execution result of the second intelligent virtual object;

[0011] According to the action execution result, train the preset decision model to obtain the target decision model.

[0012] Second aspect, an embodiment of the present application provides a method for controlling virtual objects, which is applied to a game battle scenario. The game battle scenario includes a first camp. The method further includes:

[0013] Obtain a target instruction issued by a player virtual object in the first camp;

[0014] According to the target instruction, use the target decision model of the intelligent virtual object in the first camp to determine the target action of the intelligent virtual object; wherein, the target decision model is a decision model trained by the method described in the first aspect above;

[0015] Control the intelligent virtual object to execute the target action.

[0016] Third aspect, another embodiment of the present application provides a decision model training device, which includes: an acquisition module, a determination module, and a control module, where:

[0017] The acquisition module is used to obtain a target instruction issued by a first intelligent virtual object in the first camp;

[0018] The determination module is used to determine the target action of a second intelligent virtual object in the first camp according to the target instruction by using a preset decision model;

[0019] The control module is used to control the second intelligent virtual object to execute the target action to obtain the action execution result of the second intelligent virtual object;

[0020] The determination module is specifically used to train the preset decision model according to the action execution result to obtain the target decision model.

[0021] Fourth aspect, another embodiment of the present application provides a control device for virtual objects, which includes: an acquisition module, a determination module, and a control module, where:

[0022] The acquisition module is used to obtain a target instruction issued by a player virtual object in the first camp;

[0023] The determination module is used to determine the target action of the intelligent virtual object according to the target instruction by using the target decision model of the intelligent virtual object in the first camp; wherein, the target decision model is a decision model trained by the method described in the first aspect above;

[0024] The control module is used to control the intelligent virtual object to execute the target action.

[0025] In a fifth aspect, another embodiment of the present application provides an electronic device, including: a processor, a storage medium, and a bus. The storage medium stores machine-readable instructions executable by the processor. When the electronic device runs, the processor communicates with the storage medium through the bus. The processor executes the machine-readable instructions to perform the steps of the method according to any one of the above-mentioned first aspect or second aspect.

[0026] In a sixth aspect, another embodiment of the present application provides a storage medium, on which a computer program is stored. When the computer program is run by a processor, it performs the steps of the method according to any one of the above-mentioned first aspect or second aspect.

[0027] The beneficial effect of the present application is as follows: By using the decision model training method provided by the present application, after determining the target instruction, the target action of the second intelligent virtual object is determined according to the current preset decision model. After the second intelligent virtual object executes the target action, the action execution result of the second intelligent object is obtained, and the preset decision model is retrained according to the action execution result of the second intelligent virtual object to obtain the target decision model. In this way of obtaining the target decision model, the determination of the target action not only considers the target instruction, but also, since the target decision model is reversely trained according to the action execution result, the determined execution action will better cooperate with the first intelligent virtual object in the first camp, making the cooperation degree between the first intelligent virtual object and the second intelligent virtual object in the first camp better. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] To more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings required for the embodiments. It should be understood that the following drawings only show some embodiments of the present application and should not be regarded as limiting the scope. For those of ordinary skill in the art, other relevant drawings can be obtained based on these drawings without creative efforts.

[0029] Figure 1 It is a schematic flowchart of a decision model training method provided by an embodiment of the present application;

[0030] Figure 2 It is a schematic flowchart of a decision model training method provided by another embodiment of the present application;

[0031] Figure 3 It is a schematic flowchart of a decision model training method provided by another embodiment of the present application;

[0032] Figure 4 It is a schematic flowchart of a control method for a virtual object provided by another embodiment of the present application;

[0033] Figure 5Schematic structural diagram of a decision model device provided by an embodiment of the present application;

[0034] Figure 6 Schematic structural diagram of a decision model device provided by an embodiment of the present application;

[0035] Figure 7 Schematic structural diagram of a control device for a virtual object provided by an embodiment of the present application;

[0036] Figure 8 Schematic structural diagram of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0037] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Apparently, the described embodiments are some but not all of the embodiments of the present application.

[0038] The components of the embodiments of the present application usually described and illustrated in the accompanying drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the claimed present application, but merely represents selected embodiments of the present application. All other embodiments obtained by those skilled in the art based on the embodiments of the present application without creative efforts fall within the scope of protection of the present application.

[0039] In one embodiment of the present application, the decision model training method can run on a local terminal device or a server. When the decision model training method runs on the server, the method can be implemented and executed based on a cloud interaction system, where the cloud interaction system includes a server and a client device.

[0040] In an optional implementation manner, various cloud applications can run under the cloud interaction system, such as cloud games. Taking cloud games as an example, cloud games refer to a game mode based on cloud computing. In the operation mode of cloud games, the running entity of the game program and the presenting entity of the game screen are separated. The storage and running of the decision model training method are completed on the cloud game server, and the role of the client device is for data reception, sending, and game screen presentation. For example, the client device can be a display device with data transmission function near the player side, such as a mobile terminal, a television, a computer, a palm computer, etc.; however, the information processing is performed by the cloud game server in the cloud. When playing a game, the player operates the client device to send an operation instruction to the cloud game server. The cloud game server runs the game according to the operation instruction, encodes and compresses data such as the game screen, returns it to the client device through the network, and finally, the game screen is decoded and output through the client device.

[0041] In an optional embodiment, taking a game as an example, the local terminal device stores a game program and is used to present game screens. The local terminal device is used to interact with players through a graphical user interface, that is, conventionally, the game program is downloaded and installed on an electronic device and run. The ways for the local terminal device to provide the graphical user interface to players can include various methods. For example, it can be rendered and displayed on the display screen of the terminal, or provided to players through holographic projection. For example, the local terminal device can include a display screen and a processor. The display screen is used to present the graphical user interface, and the graphical user interface includes game screens. The processor is used to run the game, generate the graphical user interface, and control the display of the graphical user interface on the display screen.

[0042] In addition, the flowcharts used in this application illustrate the operations implemented according to some embodiments of this application. It should be understood that the operations in the flowchart may not be implemented in sequence, and steps without a logical context relationship can be reversed or implemented simultaneously. In addition, those skilled in the art can add one or more other operations to the flowchart or remove one or more operations from the flowchart under the guidance of the content of this application.

[0043] The following combines multiple specific application examples to explain a decision model training method provided by the embodiments of this application. Figure 1 The flowchart of a decision model training method provided by an embodiment of this application is applied to a test battle scenario. The test battle scenario includes a first camp, as Figure 1 shown, and the method includes:

[0044] S101: Obtain a target instruction issued by a first intelligent virtual object in the first camp.

[0045] In the embodiments of this application, the test battle scenario further includes a second camp that is opposed to the first camp. That is, the test battle scenario includes a first camp and a second camp that are opposed to each other. Each camp has a preset number of intelligent virtual objects. For example, both the first camp and the second camp have three intelligent virtual objects; it can also be five intelligent virtual objects. Of course, the number of intelligent virtual objects in each camp can be flexibly adjusted and is not limited to those given in the above embodiments, as long as the number of intelligent virtual objects included in the first camp and the second camp is the same.

[0046] In the embodiments of the present application, in order to enable the AI to train the ability for different instructions, the present application sets two types of intelligent virtual objects for the first camp, namely the first intelligent virtual object (Key Player, KP) and the second intelligent virtual object (Normal Player, NP); among them, the number of the first intelligent virtual object and the second intelligent virtual object is at least one, and the other intelligent virtual objects in the first camp are the second intelligent virtual objects. The first intelligent virtual object is used to simulate a player, and the second intelligent virtual object is used to simulate an intelligent teammate.

[0047] In some possible embodiments, it is necessary to simulate one intelligent virtual object in the first virtual camp as a player, and the remaining intelligent virtual objects as teammates. That is, in each test battle scenario, one intelligent virtual object is determined from the first virtual camp as the first intelligent virtual object for simulating a player. The way to determine the first intelligent virtual object can be, for example, randomly determining one intelligent virtual object from the first virtual camp as the first intelligent virtual object, and determining that the other intelligent virtual objects in the first camp except the first intelligent virtual object are all second intelligent virtual objects. The second intelligent virtual object is used to execute corresponding actions according to the target instruction issued by the first intelligent virtual object; or according to the current state of the first intelligent virtual object and the current states of the virtual objects in the second camp. The first intelligent virtual object is used to simulate a player and issue a target instruction to the remaining second intelligent virtual objects. In some possible embodiments, the target instruction can be, for example, an instruction in a preset instruction library. The preset instruction set can include, for example: an attack instruction, a first support instruction, and a second support instruction (retreat instruction). Among them, the attack instruction is used to indicate initiating an attack action on the target virtual object in the second camp. The attack method can be, for example, sending a skill attack or a normal attack; the support instruction is used to make the second intelligent virtual objects in the first camp move around the first intelligent virtual object to bear the offensive operations initiated by other virtual objects in the second camp for the first virtual object, that is, to bear the damage for the first virtual object, so that the first virtual object can continuously output in the battle area while ensuring the health value; the second support instruction (retreat instruction) is used to make the second virtual objects in the first camp move around the first intelligent virtual object to bear the offensive operations initiated by other virtual objects in the second camp for the first virtual object, that is, to bear the damage for the first virtual object, to cover the retreat of the first virtual object, that is, to cover the first virtual object to withdraw (exit) from the current battle area, and avoid the situation where the health value of the first virtual object becomes 0 due to the offensive operations initiated by other virtual objects in the second camp; the way to issue the target instruction can be, for example: randomly obtaining an instruction from the preset instruction set as the target instruction; or it can also be to determine the target instruction according to the current state of the first intelligent virtual object and the current states of the virtual objects in the second camp.

[0048] The specific manner of determining the target instruction based on the status information can be illustrated by the following embodiments: For example, if there is an intelligent virtual object in the second camp with a blood volume lower than the first threshold, the target instruction is determined as an attack instruction; if the blood volume of the first intelligent virtual object is higher than the first threshold and lower than the second threshold, the target instruction is determined as the first support instruction; if the blood volume of the first intelligent virtual object is lower than the first threshold, the target instruction is determined as the second support instruction; it should be understood that the above embodiments are only illustrative, and the specific manner of determining the target instruction based on the current status of the first intelligent virtual object and the current status of each virtual object in the second camp can be flexibly adjusted according to user needs, and is not limited to those given in the above embodiments.

[0049] In some other possible embodiments, in order to ensure the accuracy of training, different ways of determining the target instruction may be used to determine the target instruction, or different ways of determining the target instruction are used, and the target instruction is determined according to a preset ratio. For example, in multiple test battle scenarios during the training process, the ratio of the number of test battle scenarios without instructions, the number of test battle scenarios in which the target instruction is determined according to a preset rule (according to the current status), and the number of test battle scenarios in which the target instruction is randomly determined can be 1:1:1. It should be understood that the above embodiments are only illustrative, and the specific way of determining the target instruction can be flexibly adjusted according to user needs, and is not limited to those given in the above embodiments.

[0050] S102: According to the target instruction, use a preset decision model to determine the target action of the second intelligent virtual object in the first camp.

[0051] In the embodiments of the present application, the preset instruction set includes execution strategies (target actions) corresponding to different instructions. For example, if the current instruction is an attack instruction, the second intelligent virtual object preferentially attacks the target that the first intelligent virtual object is attacking; or, if the current instruction is a support instruction, the second intelligent virtual object rushes to the side of the first intelligent virtual object to disperse the fire of the second camp (disperse the firepower of the virtual objects in the second camp, stagger the output burst period of the virtual objects in the second camp, so that the second camp cannot form a situation of outnumbering and cause the first intelligent virtual object to be killed); or, if the current instruction is a retreat instruction, the second intelligent virtual object covers the first intelligent virtual object to retreat.

[0052] In some other possible embodiments, if the target instruction is an attack instruction, the manner of determining the target action of the second intelligent virtual object can be, for example: According to the target instruction, determine the target action of the second intelligent virtual object, and determine the attack target from the second camp; control the second intelligent virtual object to execute the target action on the attack target.

[0053] That is to say, in the attack instruction state, the attack object corresponding to the execution of the target action needs to be determined in advance. For example, it can be directly determined that the target being attacked by the first intelligent virtual object is the attack target; or, it can also be determined that a certain virtual object in the second camp closest to the second intelligent virtual object is the attack target; or, it can also be determined that a certain virtual object in the second camp with the lowest remaining health within the attack range of the second intelligent virtual object is the attack target; or, it can also be determined that a virtual object of a preset type in the second camp is the attack target (for example, preferentially attack virtual objects of the mage type or preferentially attack virtual objects of the shooter type); or, it can also determine the attack target among multiple virtual objects in the second camp according to a preset priority. It should be understood that the above embodiments are only illustrative, and the specific method for determining the attack target can be flexibly adjusted according to user needs and is not limited to those given in the above embodiments.

[0054] In order for the intelligent virtual object to perceive the existence of the instruction, a preset number of instructions need to be modeled into the state of the intelligent virtual object. The modeling method can be, for example, a vector of length 3 + N (N is the number of virtual objects in each camp). The first three dimensions are the one-hot encoding vectors of which instruction the current target instruction is in, and the last N dimensions are the one-hot vectors of the attack target. In some possible embodiments, multiple instructions can be modeled into the state of the intelligent virtual object in the form of a table, for example.

[0055] Table 1 below shows the content of the preset instruction set provided by an embodiment of the present application. As shown in Table 1, the preset instruction set may include the following content, for example:

[0056] Vector subscript Value taken Meaning 0 0 / 1 Whether in an attack command 1 0 / 1 Whether in a support command 2 0 / 1 Whether in a retreat command 3 0 / 1 Whether to attack the first enemy … … … N 0 / 1 Whether to attack the Nth enemy

[0057] Table 1

[0058] It should be noted that the three instructions with vector subscripts from 0 to 2 are mutually exclusive, that is, at a certain moment in a test battle scenario, only one instruction state can be in, and multiple instructions cannot be in at the same time. When the instruction value is 1, it means the current instruction takes effect, and when the instruction value is 0, it means the current instruction is invalid; so only one of the values of the first three dimensions of the instruction can be 1 at the same moment. For example, if the AI is in the support state, then the instruction values corresponding to the 0th and 2nd dimensions of the instruction are both 0, and the value corresponding to the 1st dimension is 1. For the last N dimensions, it is only valid when in the attack command, and is 0 in other states, and only one target can be attacked at the same time.

[0059] Among them, the newly issued instruction will overwrite the previously issued instruction. That is to say, if other players initiate a new target instruction during the execution of the target instruction, the second intelligent virtual object will always execute the most recently issued target instruction.

[0060] S103: Control the second intelligent virtual object to execute the target action, and obtain the action execution result of the second intelligent virtual object.

[0061] Taking the target action as an attack action as an example for illustration, the execution results corresponding to the attack action can be, for example: causing damage to the attack target, or causing damage to a non-attack target, or not causing damage to any virtual object of the second camp; different execution results may correspond to different reward feedbacks. For example, if damage is caused to the attack target, additional reward feedback can be obtained. If no damage is caused to the attack target, or damage is caused to a non-attack target, there is no reward feedback; or, for some execution results that do not meet the preset requirements, there may be no reward feedback and there may also be a certain penalty at the same time, so that the preset decision model can be trained or adjusted according to the reward feedback later.

[0062] In the embodiments of the present application, in order to enable the NP to learn to execute the corresponding instructions correctly at the same time, certain reward guidance is needed. Otherwise, the reinforcement learning AI cannot learn the expected behavior. The reward feedback information can also be preset in the preset instruction set, and the setting of the specific reward feedback information can be as shown in Table 2:

[0063]

[0064] Table 2

[0065] S104: Train the preset decision model according to the action execution result to obtain the target decision model.

[0066] In the embodiments of the present application, the reward feedback corresponding to the execution of the target action can be determined according to the action execution result; the preset decision model is adjusted according to the reward feedback until the preset decision model meets the preset stop condition, and then the target decision model is obtained.

[0067] In some possible embodiments, intelligent virtual objects with different object attributes may all use the same target decision model; it is also possible that due to the different object attributes of each intelligent virtual object, for example, in a camp, there may be virtual objects of the mage type, warrior type, support type, shooter type, jungle type, etc., and each type of virtual object has its own corresponding target decision model, so that the decision-making can be more accurate. It should be understood that the above embodiments are only illustrative, and the configuration of the specific target decision model can be flexibly adjusted according to user needs, and is not limited to those given in the above embodiments.

[0068] In the embodiments of the present application, the reward feedback corresponding to the execution of the target action can be determined according to the action execution result; the preset decision model is adjusted according to the reward feedback until the preset decision model meets the preset stop condition, and then the target decision model is obtained.

[0069] Among them, the way to determine the preset stopping condition can be, for example: update the historical battle results according to the battle results of the current test battle; if the proportion of victories of the first camp in the historical battle results meets the preset requirements, it is determined that the preset decision model meets the preset stopping condition.

[0070] That is to say, after each battle ends, the historical battle results of the first camp will be updated according to the current battle results (the first camp succeeds / fails). If the winning rate of the historical battle results of the first camp (the proportion of battles won in all historical battles) no longer increases, it is determined that the preset decision model converges and meets the preset stopping condition.

[0071] In the embodiments of the present application, since the training of the model requires multiple battles, in order to ensure the accuracy of model training, after each battle / game ends and before entering the next game, it is necessary to re-determine the first intelligent virtual object and the second intelligent virtual object in the first camp. The determination method can be, for example: if the current game ends, exit the test battle scene corresponding to the current game; enter the test battle scene corresponding to the next game, and randomly determine an intelligent virtual object in the first camp as the first intelligent virtual object, and determine the other intelligent virtual objects in the first camp except the first intelligent virtual object as the second intelligent virtual object.

[0072] Among them, using the above-mentioned deep reinforcement learning algorithm to train intelligent virtual objects can improve the in-game level of the second intelligent virtual object, and the target actions executed by the second intelligent virtual object can be adjusted in real time according to the operations of the first intelligent virtual object and changes in the game environment; moreover, since the model training process of the present application is a self-play training, the trained decision model can be used for the first camp or the second camp. The second virtual object can communicate with the first virtual object, and the behavior of the second intelligent virtual object can be controlled by the first intelligent virtual object, so that the cooperation between the first intelligent virtual object and the second intelligent virtual object is better.

[0073] In an embodiment of the present application, after setting the training environment to an N - vs - N team - based battle environment, that is, there are N executing virtual objects in both the first camp and the second camp, and setting the training goal as the first camp defeating all intelligent virtual objects in the second camp, then modeling the state, action, reward, and neural network. Among them, the state is the observation of the intelligent virtual object, equivalent to the human eye; the action is the action that the intelligent virtual object can execute, equivalent to how a player operates the keyboard and mouse; the reward is the feedback obtained after the intelligent virtual object and the game environment perform an interaction operation (after executing the target action); the neural network is equivalent to the brain of the intelligent virtual object, responsible for making decisions, that is, inputting the current state observed by the intelligent virtual object (the target instruction information initiated by the first intelligent virtual object in the first camp and the state information of each virtual object) into the neural network. After reasoning, the neural network will output the target action that the second intelligent virtual object in the first camp needs to execute; then the second intelligent virtual object will receive a reward signal. After the current game ends, the decision - making model of the intelligent virtual object will be updated according to the preset Proximal Policy Optimization (PPO) algorithm to maximize the reward signal, and then the next game will start until the winning rate of the first camp no longer increases, at which point it is determined that the current algorithm converges.

[0074] Using the decision - making model training method provided by the present application, after determining the target instruction, the target action of the second intelligent virtual object will be determined according to the current preset decision - making model. After the second intelligent virtual object executes the target action, the action execution result of the second intelligent object will be obtained, and the preset decision - making model will be retrained according to the action execution result of the second intelligent virtual object to obtain the target decision - making model. In this way of obtaining the target decision - making model, the determination of the target action not only considers the target instruction, but also because the target decision - making model is reversely trained according to the action execution result, the determined execution action will better cooperate with the first intelligent virtual object in the first camp, making the cooperation degree between the first intelligent virtual object and the second intelligent virtual object in the first camp better.

[0075] Optionally, based on the above - mentioned embodiment, the embodiment of the present application can also provide a decision - making model training method. The following uses the accompanying drawings to exemplify the implementation process of determining the action execution result in the above - mentioned method. Figure 2 It is a schematic flowchart of a decision - making model training method provided by another embodiment of the present application, as Figure 2 shown, S103 may include:

[0076] S111: Control the second intelligent virtual object to execute the target action, and determine whether the target action is completed according to the end condition corresponding to the target action.

[0077] In some possible embodiments, in addition to sending target instructions, an end condition needs to be set for each type of instruction to prevent the AI from continuously executing the target action until the battle ends. Among them, different target actions may correspond to different end conditions. For example, if the target action is an attack action, the end condition is that the attack target is eliminated, or the duration of the attack instruction corresponding to the attack action exceeds a preset time threshold; if the target action is a first support action, the end condition is that the distance between the second intelligent virtual object and the first intelligent virtual object is less than or equal to a first preset distance, or after a preset time threshold, the distance between the second intelligent virtual object and the first intelligent virtual object is greater than the first preset distance; if the target action is a second support action, the end condition is that the distance between the first intelligent virtual object and the virtual object in the second camp is greater than or equal to a second preset distance, or after a preset time threshold, the distance between the second intelligent virtual object and the virtual object in the second camp is less than the second preset distance.

[0078] In some other possible embodiments, the configuration of the end condition can also be, for example, pre-configured in a preset instruction set. Different target instructions correspond to different target actions and different end conditions. The specific configuration of the end condition can be shown in Table 3, for example:

[0079]

[0080] Table 3

[0081] S112: If the target action is executed completely, obtain the action execution result of the second intelligent virtual object.

[0082] The action execution result can indicate, for example, whether the second intelligent virtual object has completed the target action or has automatically ended without completing the target action after a preset time. Different execution results may correspond to different reward / punishment feedbacks. Through different action execution results, the preset decision model can be well trained.

[0083] Optionally, based on the above embodiments, the embodiments of the present application can also provide a decision model training method. The implementation process of determining the action execution result in the above method is illustrated below with reference to the accompanying drawings. Figure 3 It is a schematic flowchart of a decision model training method provided by another embodiment of the present application. If the target instruction is an attack instruction, then as Figure 3 shown, S103 may include:

[0084] S121: According to the target instruction, determine the target action of the second intelligent virtual object and determine the attack target from the second camp.

[0085] That is to say, if the current target instruction is an attack instruction, before determining the target action, it is necessary to determine the attack target. The determination method of the attack target can be, for example, determining the attack target according to the marking operation of the first intelligent virtual object; or, it can be determining the virtual object with the lowest health value in the second camp as the attack target; or, it can also be determining the virtual object in the second camp that is closest to the first intelligent virtual object that issues the attack instruction as the attack target; or, it can be determining the target that the first intelligent virtual object is attacking as the attack target; or, it can also be determining the virtual object of the target type in the second camp (such as shooter virtual object, mage virtual object or jungle virtual object, etc.) as the target virtual object. It should be understood that the above embodiments are only exemplary descriptions, and the specific determination method of the target virtual object can be flexibly adjusted according to user needs, and is not limited to those given in the above embodiments.

[0086] S122: Control the second intelligent virtual object to perform the target action on the attack target.

[0087] The way to perform the target action on the attack target can be, for example, performing a preset combo action (a combination of multiple preset skills, such as first performing skill A, then performing skill B, and finally performing skill C, etc.); or, it can also be performing a normal attack operation; or, it can also be performing a preset combo action and continuing to perform a normal attack operation after the execution ends, etc. The specific determination method of the target action can be flexibly adjusted according to user needs, and is not limited to those given in the above embodiments.

[0088] By using the decision model training method provided in this application, since the decision model determines the reward information according to the execution result when the second virtual object in the first camp performs the target action, and is repeatedly trained according to the reward information, the obtained decision model can, in the subsequent use process, well determine the target action corresponding to the second virtual object according to the current game stage, or according to the states of each virtual object, according to the target instruction initiated by the player, thereby greatly improving the human-machine cooperation effect and thus improving the player's gaming experience.

[0089] Optionally, on the basis of the above embodiments, the embodiments of this application can also provide a control method for virtual objects. The following uses the accompanying drawings to exemplarily illustrate the implementation process of determining the action execution result in the above method. Figure 4 It is a schematic flowchart of a control method for a virtual object provided by an embodiment of this application, which is applied to a game battle scenario. The game battle scenario includes a first camp, as Figure 4 shown, the method includes:

[0090] S201: Obtain the target instruction issued by the player virtual object in the first camp.

[0091] Among them, players in the first camp can issue attack commands, retreat commands, or support commands for a target location according to the current game progress and their own needs; the operation of selecting the specific target location can be the target location marked by the player on the mini-map, or it can also be the target location marked by the player on the scene map by dragging the scene map. The target command can be triggered by the preset shortcut of each command, or can be triggered by clicking on the command control. The specific way of triggering the target command and the way of determining the target location can be flexibly adjusted according to the user's needs, and are not limited to those given in the above embodiments.

[0092] Each player in the same game session can issue a target command, and the newly issued target command will overwrite the previously issued target command.

[0093] S202: According to the target command, use the target decision model of the intelligent virtual object in the first camp to determine the target action of the intelligent virtual object.

[0094] Among them, the target decision model is a decision model trained by the method of any one of the above Figures 1 - 3 The decision model obtained by training.

[0095] The game battle scene also includes a second camp that is opposed to the first camp; if the target command is an attack command, the target decision model can be used to determine the target action of the intelligent virtual object according to the attack command, and determine the attack target from the second camp; control the intelligent virtual object to execute the target action on the attack target.

[0096] That is to say, if the target command is an attack command, it is necessary to first determine the attack target and the target action according to the current attack command, and then execute the target action on the determined attack target.

[0097] S203: Control the intelligent virtual object to execute the target action.

[0098] That is to say, using the decision model trained by the method of any one of the present application Figures 1 - 3 During the subsequent use process, the intelligent virtual object in the same camp as the player will not only determine the corresponding target action according to the current state of the virtual object controlled by the player; but also determine the corresponding target action according to the target command issued by the player; that is, the intelligent virtual object in the same camp as the player will monitor in real time whether the player issues a target command, and at the same time monitor the state of the virtual object controlled by the player in real time, so as to determine the corresponding target action according to the target command / state issued by the player, so as to better cooperate with the player to play the game and improve the player's game experience.

[0099] It should be noted that in some possible embodiments, the intelligent virtual object will preferentially determine the corresponding target action according to the target instruction initiated by the player; if the player does not initiate a target instruction, the intelligent virtual object will monitor the state of the virtual object controlled by the player in real time. For example, if there is an intelligent virtual object in the second camp with a blood volume lower than the first threshold, it is determined that the target action corresponding to the current intelligent virtual object is an attack action; if the blood volume of the virtual object controlled by the player is higher than the first threshold and lower than the second threshold, it is determined that the target action corresponding to the current intelligent virtual object is the first support action; if the blood volume of the first intelligent virtual object is lower than the first threshold, it is determined that the target action corresponding to the current intelligent virtual object is the second support action. It should be understood that the above embodiments are only illustrative. The priority of the specific intelligent virtual object to determine the target action can be flexibly adjusted according to user needs, or the state of the virtual object controlled by the player can be the highest priority, and the target instruction initiated by the player is secondary. The specific way to determine the target action of the intelligent virtual object can be flexibly adjusted according to user needs, and is not limited to the above embodiments.

[0100] Using the control method of the virtual object provided by the present application, since the target decision model is trained according to the above Figures 1 - 3 method, therefore, the target action determined by using the target decision model can better cooperate with the player virtual object in the first camp, so that in a game, the intelligent virtual object can, through the pre-trained target decision model, make the intelligent virtual object operate in the game according to the player's intention, and there can be an interaction between the player and the intelligent virtual object, and the player can control the behavior of the intelligent virtual object, so that the intelligent virtual object can better cooperate with the player virtual object in the first camp, making the player's game experience better.

[0101] The following explains the decision model training device provided by the present application in conjunction with the accompanying drawings. The decision model training device can execute any of the above Figures 1 - 4 decision model training methods, and its specific implementation and beneficial effects are referred to above and will not be elaborated below.

[0102] Figure 5 is a schematic structural diagram of a decision model training device provided by an embodiment of the present application. As Figure 5 shown, the device includes: an acquisition module 301, a determination module 302, and a control module 303, where:

[0103] The acquisition module 301 is used to acquire the target instruction sent by the first intelligent virtual object in the first camp;

[0104] The determination module 302 is used to determine the target action of the second intelligent virtual object in the first camp according to the target instruction by using a preset decision model;

[0105] A control module 303, configured to control a second intelligent virtual object to execute a target action and obtain an action execution result of the second intelligent virtual object;

[0106] A determination module 302, specifically configured to train a preset decision model according to the action execution result to obtain a target decision model.

[0107] Optionally, the determination module 302 is specifically configured to randomly determine an intelligent virtual object in a first camp as a first intelligent virtual object, and determine other intelligent virtual objects in the first camp except the first intelligent virtual object as second intelligent virtual objects; the second intelligent virtual objects are used to execute corresponding actions according to a target instruction issued by the first intelligent virtual object; or, according to the current state of the first intelligent virtual object and the current states of the virtual objects in a second camp.

[0108] Optionally, the determination module 302 is specifically configured to, if the current game ends, exit the test battle scenario corresponding to the current game; enter the test battle scenario corresponding to the next game, and randomly determine an intelligent virtual object in the first camp as the first intelligent virtual object again, and determine other intelligent virtual objects in the first camp except the first intelligent virtual object as second intelligent virtual objects.

[0109] Optionally, an acquisition module 301 is specifically configured to randomly acquire a target instruction from a preset instruction set.

[0110] Optionally, the determination module 302 is specifically configured to determine a target instruction according to the current state of the first intelligent virtual object and the current states of the virtual objects in the second camp.

[0111] Optionally, the determination module 302 is specifically configured to, if there is an intelligent virtual object in the second camp with a blood volume lower than a first threshold, determine the target instruction as an attack instruction; if the blood volume of the first intelligent virtual object is higher than the first threshold and less than a second threshold, determine the target instruction as a first support instruction; if the blood volume of the first intelligent virtual object is lower than the first threshold, determine the target instruction as a second support instruction.

[0112] Optionally, the control module 303 is specifically configured to control the second intelligent virtual object to execute a target action, and determine whether the target action is completed according to an end condition corresponding to the target action;

[0113] The acquisition module 301 is specifically configured to, if the target action is completed, acquire the action execution result of the second intelligent virtual object.

[0114] Optionally, if the target action is an attack action, the end condition is that the attack target is eliminated, or the duration of the attack instruction corresponding to the attack action exceeds a preset time threshold;

[0115] If the target action is the first support action, the end condition is that the distance between the second intelligent virtual object and the first intelligent virtual object is less than or equal to the first preset distance, or after the preset time threshold, the distance between the second intelligent virtual object and the first intelligent virtual object is greater than the first preset distance;

[0116] If the target action is the second support action, the end condition is that the distance between the first intelligent virtual object and the virtual object in the second camp is greater than or equal to the second preset distance, or after the preset time threshold, the distance between the second intelligent virtual object and the virtual object in the second camp is less than the second preset distance.

[0117] Optionally, the determining module 302 is specifically configured to determine the target action of the second intelligent virtual object according to the target instruction, and determine the attack target from the second camp;

[0118] The control module 303 is specifically configured to control the second intelligent virtual object to execute the target action on the attack target.

[0119] Optionally, based on the above embodiments, the embodiment of the present application can further provide a decision model training device. The implementation process of the above Figure 5 given device is illustrated below in conjunction with the accompanying drawings. Figure 6 FIG. is a schematic structural diagram of a decision model training device provided by another embodiment of the present application. As Figure 6 shown, the device further includes: an adjustment module 304, where:

[0120] The determining module 302 is specifically configured to determine the reward feedback corresponding to the execution of the target action according to the action execution result;

[0121] The adjustment module 304 is configured to adjust the preset decision model according to the reward feedback until the preset decision model meets the preset stop condition, and then obtain the target decision model.

[0122] As Figure 6 shown, the device further includes: an update module 305, configured to update the historical battle result according to the battle result of the current test battle;

[0123] The determining module 302 is specifically configured to determine that the preset decision model meets the preset stop condition if the proportion of victories of the first camp in the historical battle results meets the preset requirements.

[0124] Using the decision model training device provided by the present application, after determining the target instruction, the target action of the second intelligent virtual object will be determined according to the current preset decision model. After the second intelligent virtual object executes the target action, the action execution result of the second intelligent object will be obtained, and the preset decision model will be retrained according to the action execution result of the second intelligent virtual object to obtain the target decision model. In this way of obtaining the target decision model, the determination of the target action not only considers the target instruction, but also because the target decision model is reversely trained according to the action execution result, the determined execution action will better cooperate with the first intelligent virtual object in the first camp, making the cooperation degree between the first intelligent virtual object and the second intelligent virtual object in the first camp better.

[0125] Figure 7 The structural schematic diagram of the control device for virtual objects provided by an embodiment of the present application is as Figure 7 shown. The device includes: an acquisition module 401, a determination module 402, and a control module 403, where:

[0126] The acquisition module 401 is configured to acquire a target instruction issued by a player virtual object in the first camp;

[0127] The determination module 402 is configured to determine the target action of the intelligent virtual object according to the target instruction by using the target decision model of the intelligent virtual object in the first camp; where the target decision model is a decision model trained by the method of any one of the above Figures 1 - 3 items;

[0128] The control module 403 is configured to control the intelligent virtual object to execute the target action.

[0129] Optionally, if the target instruction is an attack instruction, the determination module 402 is specifically configured to determine the target action of the intelligent virtual object according to the attack instruction by the target decision model, and determine an attack target from the second camp;

[0130] The control module 403 is specifically configured to control the intelligent virtual object to execute the target action on the attack target.

[0131] Using the control device for virtual objects provided by the present application, since the target decision model is based on the above Figures 1 - 3It is trained by the method described above. Therefore, the target actions determined by the target decision-making model can better cooperate with the player virtual objects in the first camp. In a game session, the intelligent virtual object can, through the pre-trained target decision-making model, perform operations in the game according to the player's intention. Interaction can occur between the player and the intelligent virtual object, and the player can control the behavior of the intelligent virtual object, so that the intelligent virtual object can better cooperate with the player virtual objects in the first camp, resulting in a better gaming experience for the player.

[0132] The above-mentioned modules can be one or more integrated circuits configured to implement the above method. For example: one or more Application Specific Integrated Circuits (ASICs), or, one or more microprocessors, or, one or more Field Programmable Gate Arrays (FPGAs), etc. Again, when a certain above-mentioned module is implemented in the form of a processing element scheduling program code, the processing element can be a general-purpose processor, such as a Central Processing Unit (CPU) or other processors that can call program code. Again, these modules can be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0133] Figure 8 It is a schematic structural diagram of an electronic device provided by an embodiment of the present application. This electronic device can be integrated into a terminal device or a chip of a terminal device.

[0134] As Figure 8 shown, this electronic device includes: a processor 501, a bus 502, and a storage medium 503.

[0135] The processor 501 is used to store programs. The processor 501 calls the programs stored in the storage medium 503 to execute the above-mentioned Figures 1 - 4 corresponding method embodiments.

[0136] Specifically, the steps for the processor 501 to execute the above decision model training method include:

[0137] Obtain the target instruction sent by the first intelligent virtual object in the first camp;

[0138] According to the target instruction, use a preset decision model to determine the target action of the second intelligent virtual object in the first camp;

[0139] Control the second intelligent virtual object to execute the target action to obtain the action execution result of the second intelligent virtual object;

[0140] Train the preset decision model according to the action execution result to obtain a target decision model.

[0141] In some embodiments, the test battle scenario further includes a second camp opposing the first camp; before the processor 501 executes obtaining the target instruction issued by the first intelligent virtual object in the first camp, it further includes:

[0142] Randomly determine an intelligent virtual object in the first camp as the first intelligent virtual object, and determine the other intelligent virtual objects in the first camp except the first intelligent virtual object as the second intelligent virtual objects; the second intelligent virtual objects are used to execute corresponding actions according to the target instruction issued by the first intelligent virtual object; or according to the current state of the first intelligent virtual object and the current states of the virtual objects in the second camp.

[0143] Optionally, the processor 501 is further configured to execute:

[0144] If the current game ends, exit the test battle scenario corresponding to the current game;

[0145] Enter the test battle scenario corresponding to the next game, and randomly determine an intelligent virtual object in the first camp as the first intelligent virtual object again, and determine the other intelligent virtual objects in the first camp except the first intelligent virtual object as the second intelligent virtual objects.

[0146] In some embodiments, the test battle scenario further includes a second camp opposing the first camp; when the processor 501 executes obtaining the target instruction issued by the first intelligent virtual object in the first camp, it further includes:

[0147] Randomly obtain the target instruction from a preset instruction set.

[0148] In some embodiments, when the processor 501 executes obtaining the target instruction issued by the first intelligent virtual object in the first camp, it further includes:

[0149] Determine the target instruction according to the current state of the first intelligent virtual object and the current states of the virtual objects in the second camp.

[0150] In some embodiments, when the processor 501 executes determining the target instruction according to the current state of the first intelligent virtual object and the current states of the virtual objects in the second camp, it further includes:

[0151] If there is an intelligent virtual object in the second camp with a blood volume lower than the first threshold, determine that the target instruction is an attack instruction;

[0152] If the blood volume of the first intelligent virtual object is higher than the first threshold and lower than the second threshold, determine that the target instruction is a first support instruction;

[0153] If the blood volume of the first intelligent virtual object is lower than the first threshold, determine that the target instruction is a second support instruction.

[0154] In some embodiments, when the processor 501 executes controlling the second intelligent virtual object to execute the target action and obtaining the action execution result of the second intelligent virtual object, it further includes:

[0155] Control the second intelligent virtual object to execute the target action, and determine whether the target action is completed according to the end condition corresponding to the target action;

[0156] If the target action is completed, obtain the action execution result of the second intelligent virtual object.

[0157] Optionally, if the target action is an attack action, the end condition is that the attack target is eliminated, or the duration of the attack instruction corresponding to the attack action exceeds a preset time threshold;

[0158] If the target action is a first support action, the end condition is that the distance between the second intelligent virtual object and the first intelligent virtual object is less than or equal to a first preset distance, or after a preset time threshold, the distance between the second intelligent virtual object and the first intelligent virtual object is greater than the first preset distance;

[0159] If the target action is a second support action, the end condition is that the distance between the first intelligent virtual object and the virtual object in the second camp is greater than or equal to a second preset distance, or after a preset time threshold, the distance between the second intelligent virtual object and the virtual object in the second camp is less than the second preset distance.

[0160] In some embodiments, if the target instruction is an attack instruction, when the processor 501 executes determining the target action of the second intelligent virtual object in the first camp according to the target instruction by using a preset decision model, it further includes:

[0161] According to the target instruction, determine the target action of the second intelligent virtual object and determine an attack target from the second camp;

[0162] Control the second intelligent virtual object to execute the target action on the attack target.

[0163] In some embodiments, when the processor 501 trains the preset decision model according to the execution result of the action to obtain the target decision model, it further includes:

[0164] Determine a reward feedback corresponding to the execution of the target action according to the execution result of the action;

[0165] Adjust the preset decision model according to the reward feedback until the preset decision model meets the preset stop condition, and then obtain the target decision model.

[0166] In some embodiments, when the processor 501 executes until the preset decision model meets the preset stop condition, it further includes:

[0167] Update the historical battle results according to the battle results of the current test battle;

[0168] If the proportion of victories of the first camp in the historical battle results meets the preset requirements, it is determined that the preset decision model meets the preset stop condition.

[0169] In addition, the steps for the specific processor 501 to execute the control method of the virtual object described above include:

[0170] Obtain a target instruction issued by a player virtual object in the first camp;

[0171] According to the target instruction, use the target decision model of the intelligent virtual object in the first camp to determine the target action of the intelligent virtual object; wherein, the target decision model is a decision model trained by the method described in any one of the above Figures 1 - 3 The decision model obtained by training;

[0172] Control the intelligent virtual object to execute the target action.

[0173] In some embodiments, if the target instruction is an attack instruction, when the processor 501 executes using the target decision model of the intelligent virtual object in the first camp to determine the target action of the intelligent virtual object, it is further used to execute:

[0174] The target decision model determines the target action of the intelligent virtual object according to the attack instruction and determines an attack target from the second camp;

[0175] Control the intelligent virtual object to execute the target action on the attack target.

[0176] When using the electronic device provided in this application, after determining the target instruction, the target action of the second intelligent virtual object will be determined according to the current preset decision model. After the second intelligent virtual object executes the target action, the action execution result of the second intelligent object will be obtained, and the preset decision model will also be retrained according to the action execution result of the second intelligent virtual object to obtain the target decision model. In this way of obtaining the target decision model, the determination of the target action not only considers the target instruction, but also, since the target decision model is reversely trained according to the action execution result, the determined execution action will better cooperate with the first intelligent virtual object in the first camp, making the cooperation between the first intelligent virtual object and the second intelligent virtual object in the first camp better.

[0177] Optionally, this application also provides a program product, such as a storage medium, on which a computer program is stored, including a program that, when run by a processor, executes the corresponding embodiment of the above method.

[0178] The steps for a specific processor to execute the above decision model training method include:

[0179] Obtain the target instruction issued by the first intelligent virtual object in the first camp;

[0180] According to the target instruction, use the preset decision model to determine the target action of the second intelligent virtual object in the first camp;

[0181] Control the second intelligent virtual object to execute the target action to obtain the action execution result of the second intelligent virtual object;

[0182] According to the action execution result, train the preset decision model to obtain the target decision model.

[0183] In some embodiments, when the processor executes the step of obtaining the target instruction issued by the first intelligent virtual object in the first camp, it further includes:

[0184] Randomly obtain the target instruction from the preset instruction set.

[0185] In some embodiments, the test battle scenario further includes a second camp opposing the first camp; before the processor executes the step of obtaining the target instruction issued by the first intelligent virtual object in the first camp, it is further used to execute:

[0186] Randomly determine an intelligent virtual object in the first camp as the first intelligent virtual object, and determine the other intelligent virtual objects in the first camp except the first intelligent virtual object as the second intelligent virtual objects; the second intelligent virtual objects are used to execute corresponding actions according to the target instruction issued by the first intelligent virtual object; or, according to the current state of the first intelligent virtual object and the current states of the virtual objects in the second camp.

[0187] Optionally, the processor is further configured to execute:

[0188] If the current game ends, exit the test battle scenario corresponding to the current game;

[0189] Enter the test battle scenario corresponding to the next game, and randomly determine an intelligent virtual object in the first camp as the first intelligent virtual object again, and determine the other intelligent virtual objects in the first camp except the first intelligent virtual object as the second intelligent virtual objects.

[0190] In some embodiments, when the processor executes the operation of obtaining the target instruction issued by the first intelligent virtual object in the first camp, it further includes:

[0191] Determine the target instruction according to the current state of the first intelligent virtual object and the current states of the virtual objects in the second camp.

[0192] In some embodiments, when the processor executes the operation of determining the target instruction according to the current state of the first intelligent virtual object and the current states of the virtual objects in the second camp, it further includes:

[0193] If there is an intelligent virtual object in the second camp with a blood volume lower than the first threshold, determine the target instruction as an attack instruction;

[0194] If the blood volume of the first intelligent virtual object is higher than the first threshold and less than the second threshold, determine the target instruction as the first support instruction;

[0195] If the blood volume of the first intelligent virtual object is lower than the first threshold, determine the target instruction as the second support instruction.

[0196] In some embodiments, when the processor executes the operation of controlling the second intelligent virtual object to execute the target action and obtaining the action execution result of the second intelligent virtual object, it further includes:

[0197] Control the second intelligent virtual object to execute the target action, and determine whether the target action is completed according to the end condition corresponding to the target action;

[0198] If the target action is completed, obtain the action execution result of the second intelligent virtual object.

[0199] Optionally, if the target action is an attack action, the end condition is that the attack target is eliminated, or the duration of the attack instruction corresponding to the attack action exceeds a preset time threshold;

[0200] If the target action is a first support action, the end condition is that the distance between the second intelligent virtual object and the first intelligent virtual object is less than or equal to a first preset distance, or after a preset time threshold, the distance between the second intelligent virtual object and the first intelligent virtual object is greater than the first preset distance;

[0201] If the target action is a second support action, the end condition is that the distance between the first intelligent virtual object and the virtual object in the second camp is greater than or equal to a second preset distance, or after a preset time threshold, the distance between the second intelligent virtual object and the virtual object in the second camp is less than the second preset distance.

[0202] In some embodiments, if the target instruction is an attack instruction, when the processor executes determining the target action of the second intelligent virtual object in the first camp according to the target instruction by using a preset decision model, it further includes:

[0203] Determine the target action of the second intelligent virtual object according to the target instruction, and determine an attack target from the second camp;

[0204] Control the second intelligent virtual object to execute the target action on the attack target.

[0205] In some embodiments, when the processor executes training the preset decision model according to the action execution result to obtain the target decision model, it further includes:

[0206] Determine a reward feedback corresponding to executing the target action according to the action execution result;

[0207] Adjust the preset decision model according to the reward feedback until the preset decision model meets a preset stop condition, then obtain the target decision model.

[0208] In some embodiments, when the processor executes until the preset decision model meets a preset stop condition, it further includes:

[0209] Update the historical battle result according to the battle result of the current test battle;

[0210] If, in the historical battle results, the proportion of victories of the first camp meets the preset requirements, it is determined that the preset decision model meets the preset stop condition.

[0211] In addition, the steps for a specific processor to execute the control method of the virtual object include:

[0212] Obtain a target instruction issued by a player virtual object in the first camp;

[0213] According to the target instruction, use the target decision model of the intelligent virtual object in the first camp to determine the target action of the intelligent virtual object; wherein, the target decision model is a decision model trained by the method described in any of the above Figures 1 - 3 Any of the described methods.

[0214] Control the intelligent virtual object to execute the target action.

[0215] In some embodiments, if the target instruction is an attack instruction, when the processor executes using the target decision model of the intelligent virtual object in the first camp to determine the target action of the intelligent virtual object, it is further used to execute:

[0216] The target decision model determines the target action of the intelligent virtual object according to the attack instruction, and determines an attack target from the second camp;

[0217] Control the intelligent virtual object to execute the target action on the attack target.

[0218] Using the storage medium provided by the present application, after determining the target instruction, it will determine the target action of the second intelligent virtual object according to the current preset decision model, and after the second intelligent virtual object executes the target action, obtain the action execution result of the second intelligent object, and will also re-train the preset decision model according to the action execution result of the second intelligent virtual object to obtain the target decision model. In this way of obtaining the target decision model, the determination of the target action not only considers the target instruction, and because the target decision model is reversely trained according to the action execution result, the determined execution action will better cooperate with the first intelligent virtual object in the first camp, making the cooperation degree between the first intelligent virtual object and the second intelligent virtual object in the first camp better.

[0219] In several embodiments provided by this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections between each other can be through some interfaces. The indirect couplings or communication connections of the devices or units can be in electrical, mechanical or other forms.

[0220] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0221] In addition, in each embodiment of this application, the functional units can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of a combination of hardware and software functional units.

[0222] The above-mentioned integrated units implemented in the form of software functional units can be stored in a computer-readable storage medium. The above-mentioned software functional units are stored in a storage medium and include several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (English: processor) to execute some steps of the methods described in each embodiment of this application. The foregoing storage medium includes: USB flash drives, mobile hard disks, read-only memories (English: Read-Only Memory, abbreviated as: ROM), random access memories (English: Random Access Memory, abbreviated as: RAM), magnetic disks or optical disks and other various media that can store program codes.

Claims

1. A method for training a decision-making model, characterized in that, applied to a test battle scenario, the test battle scenario includes a first camp, and the method includes: Obtaining a target instruction issued by a first intelligent virtual object in the first camp; According to the target instruction, using a preset decision-making model, determining a target action of a second intelligent virtual object in the first camp; Controlling the second intelligent virtual object to execute the target action to obtain an action execution result of the second intelligent virtual object; Training the preset decision-making model according to the action execution result to obtain a target decision-making model.

2. The method according to claim 1, characterized in that, the test battle scenario further includes a second camp opposing the first camp; before obtaining the target instruction issued by the first intelligent virtual object in the first camp, the method further includes: Randomly determining an intelligent virtual object in the first camp as the first intelligent virtual object, and determining other intelligent virtual objects in the first camp except the first intelligent virtual object as the second intelligent virtual object; the second intelligent virtual object is used to execute corresponding actions according to the target instruction issued by the first intelligent virtual object, or according to the current state of the first intelligent virtual object and the current states of the virtual objects in the second camp.

3. The method according to claim 2, characterized in that, the method further includes: If the current game ends, exiting the test battle scenario corresponding to the current game; Entering the test battle scenario corresponding to the next game, and randomly determining an intelligent virtual object in the first camp as the first intelligent virtual object again, and determining other intelligent virtual objects in the first camp except the first intelligent virtual object as the second intelligent virtual object.

4. The method according to claim 1, characterized in that, the obtaining of the target instruction issued by the first intelligent virtual object in the first camp includes: Randomly obtaining the target instruction from a preset instruction set.

5. The method according to claim 1, characterized in that, the test battle scenario further includes a second camp opposing the first camp; the obtaining of the target instruction issued by the first intelligent virtual object in the first camp includes: Determining the target instruction according to the current state of the first intelligent virtual object and the current states of the virtual objects in the second camp.

6. The method according to claim 5, characterized in that, the determining of the target instruction according to the current state of the first intelligent virtual object and the current states of the virtual objects in the second camp includes: If there is an intelligent virtual object in the second camp with a blood volume lower than a first threshold, determining the target instruction as an attack instruction; If the blood volume of the first intelligent virtual object is higher than the first threshold and less than a second threshold, determining the target instruction as a first support instruction; If the blood volume of the first intelligent virtual object is lower than the first threshold, determining the target instruction as a second support instruction.

7. The method according to claim 1, characterized in that, Controlling the second intelligent virtual object to execute the target action and obtaining the action execution result of the second intelligent virtual object includes: Controlling the second intelligent virtual object to execute the target action, and determining whether the target action is completed according to the end condition corresponding to the target action; If the target action is completed, obtain the action execution result of the second intelligent virtual object.

8. The method according to claim 7, wherein, if the target action is an attack action, the end condition is that the attack target is eliminated, or the duration of the attack instruction corresponding to the attack action exceeds a preset time threshold; if the target action is a first support action, the end condition is that the distance between the second intelligent virtual object and the first intelligent virtual object is less than or equal to a first preset distance, or after a preset time threshold, the distance between the second intelligent virtual object and the first intelligent virtual object is greater than the first preset distance; if the target action is a second support action, the end condition is that the distance between the first intelligent virtual object and the virtual object in the second camp is greater than or equal to a second preset distance, or after a preset time threshold, the distance between the second intelligent virtual object and the virtual object in the second camp is less than the second preset distance.

9. The method according to claim 1, wherein, if the target instruction is an attack instruction, then according to the target instruction, using a preset decision model to determine the target action of the second intelligent virtual object in the first camp includes: According to the target instruction, determine the target action of the second intelligent virtual object and determine an attack target from the second camp; Control the second intelligent virtual object to execute the target action on the attack target.

10. The method according to claim 1, wherein, training the preset decision model according to the action execution result to obtain the target decision model includes: Determine the reward feedback corresponding to executing the target action according to the action execution result; Adjust the preset decision model according to the reward feedback until the preset decision model meets the preset stop condition, then obtain the target decision model.

11. The method according to claim 10, wherein, the until the preset decision model meets the preset stop condition includes: Update the historical battle results according to the battle result of the current test battle; If the proportion of victories of the first camp in the historical battle results meets the preset requirements, it is determined that the preset decision model meets the preset stop condition.

12. A method for controlling a virtual object, wherein, applied to a game battle scenario, the game battle scenario includes a first camp, and the method further includes: Obtain a target instruction issued by a player virtual object in the first camp; According to the target instruction, use the target decision-making model of the intelligent virtual object in the first camp to determine the target action of the intelligent virtual object; wherein, the target decision-making model is a decision-making model trained by the method described in any one of the above claims 1-11; Control the intelligent virtual object to execute the target action.

13. The method according to claim 12, characterized in that, the game battle scenario further includes a second camp opposing the first camp; if the target instruction is an attack instruction, then using the target decision-making model of the intelligent virtual object in the first camp to determine the target action of the intelligent virtual object includes: the target decision-making model determines the target action of the intelligent virtual object according to the attack instruction, and determines an attack target from the second camp; Control the intelligent virtual object to execute the target action on the attack target.

14. A decision-making model training device, characterized in that, the device includes: an acquisition module, a determination module and a control module, wherein: the acquisition module is used to acquire the target instruction issued by the first intelligent virtual object in the first camp; the determination module is used to determine the target action of the second intelligent virtual object in the first camp according to the target instruction by using a preset decision-making model; the control module is used to control the second intelligent virtual object to execute the target action to obtain the action execution result of the second intelligent virtual object; the determination module is specifically used to train the preset decision-making model according to the action execution result to obtain a target decision-making model.

15. A control device for a virtual object, characterized in that, the device includes: an acquisition module, a determination module and a control module, wherein: the acquisition module is used to acquire the target instruction issued by the player virtual object in the first camp; the determination module is used to determine the target action of the intelligent virtual object according to the target instruction by using the target decision-making model of the intelligent virtual object in the first camp; wherein, the target decision-making model is a decision-making model trained by the method described in any one of the above claims 1-8; the control module is used to control the intelligent virtual object to execute the target action.

16. An electronic device, characterized in that, the device includes: a processor, a storage medium and a bus, the storage medium stores machine-readable instructions executable by the processor, when the electronic device runs, the processor communicates with the storage medium through the bus, and the processor executes the machine-readable instructions to execute the method described in any one of the above claims 1-13.

17. A storage medium, characterized in that, a computer program is stored on the storage medium, and when the computer program is run by a processor, it executes the method described in any one of the above claims 1-13.

Images