Virtual object control method and device, electronic equipment and storage medium

Abstract

The invention provides a virtual object control method and device, electronic equipment and a storage medium, a graphical user interface is provided through terminal equipment, at least part of a game scene of a current virtual game is displayed on the graphical user interface, and the game scene comprises a first virtual object which is not controlled by a player. The method comprises the following steps: extracting a first attribute state feature of a first preset output in a current virtual game; extracting game state characteristics of the virtual game; determining a first output action from the first preset output according to the first attribute state feature and the game state feature; and forming an action strategy for the first virtual object according to a second attribute state feature of a second preset output in the virtual game and the attribute state feature of the first output action so as to control the first virtual object to execute the action strategy in the game scene. According to the invention, the correlation between the skill output and the target output can be improved.

Description

Technical Field

[0001] The present application relates to the field of computer application technology, and in particular to a control method, device, electronic device and storage medium for a virtual object. Background Art

[0002] Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0003] With the research and progress of artificial intelligence technology, artificial intelligence technology has been studied and applied in many fields. For example, artificial intelligence is currently used in many scenarios in game development and application, which is called game artificial intelligence, or game AI. For example, in the process of game development, game AI can replace the role of testers, and obtain test data by introducing game AI to participate in the game to achieve game performance testing. For another example, in the process of game application, if there are not enough players in the game, game players can actively introduce game AI to play games with real people.

[0004] Game AI is an important part of competitive games. Current game AI solutions are mainly divided into two categories: non-learning and learning. The non-learning solution uses finite state machines, behavior trees, etc. to design and implement the behavior logic of game AI, and selects appropriate behavior nodes to execute according to the game environment and character status. Although this solution is easy to implement, it also has many problems. For example, the behavior tree method requires strong prior knowledge to preset a large number of behavior nodes and conditions to meet the needs of the scene, which makes its strategy lack diversity and flexibility, low combat intensity, poor anthropomorphism, and easy to be predicted and cracked by other players.

[0005] Learning solutions generally use reinforcement learning methods to train and optimize game AI strategies, obtain reward signals through interaction with the environment, and thus improve the combat capabilities of game AI. Although reinforcement learning methods can improve the diversity of strategies, combat intensity, and anthropomorphism, the existing design methods lack universal expression capabilities and need to be developed separately for different gameplay and game characters, which makes their development and maintenance costs high and difficult to adapt to the increase and update of game content. Summary of the invention

[0006] In view of this, the embodiments of the present application provide at least a method, device, electronic device and storage medium for controlling a virtual object to overcome at least one of the above-mentioned defects.

[0007] In a first aspect, an exemplary embodiment of the present application provides a method for controlling a virtual object, wherein a graphical user interface is provided through a terminal device, and at least a portion of a game scene of a current virtual game is displayed on the graphical user interface, wherein the game scene includes a first virtual object that is not controlled by a player, and the following processing is performed at each decision point in the course of the current virtual game: extracting a first attribute state feature of a first preset output in the current virtual game, wherein the first preset output is one of a skill output and a target output; extracting a game state feature of the current virtual game; determining a first output action from the first preset output based on the extracted first attribute state feature and the game state feature; forming an action strategy for the first virtual object based on a second attribute state feature of a second preset output in the current virtual game and the attribute state feature of the determined first output action, so as to control the first virtual object to execute the action strategy in the game scene, wherein the second preset output is the other of the skill output and the target output.

[0008] In a second aspect, an embodiment of the present application further provides a control device for a virtual object, which provides a graphical user interface through a terminal device, and displays at least part of the game scene of this virtual game on the graphical user interface, wherein the game scene includes a first virtual object that is not controlled by a player, and the device includes: an attribute feature extraction module, which extracts a first attribute state feature of a first preset output in this virtual game at each decision point in the process of this virtual game, wherein the first preset output is one of a skill output and a target output; a game feature extraction module, which extracts a game state feature of this virtual game; an action output module, which determines a first output action from the first preset output based on the extracted first attribute state feature and the game state feature; and a strategy execution module, which forms an action strategy for the first virtual object based on the second attribute state feature of the second preset output in this virtual game and the attribute state feature of the determined first output action, so as to control the first virtual object to execute the action strategy in the game scene, wherein the second preset output is the other of the skill output and the target output.

[0009] In a third aspect, an embodiment of the present application further provides an electronic device, a processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the storage medium through the bus, and the processor executes the machine-readable instructions to perform the steps of the above-mentioned virtual object control method.

[0010] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above-mentioned virtual object control method are executed.

[0011] The control method, device, electronic device and storage medium of the virtual object provided in the embodiments of the present application realize decoupling from the specific logic of the game, improve the versatility of the game AI, and facilitate porting and use between different games and different skills.

[0012] In addition, in the embodiment of the present application, the output actions for the virtual object are divided into two links: skill output and target output. By using one of the output actions as the input of another output action, the output action can be adjusted and optimized according to the previous output action, thereby increasing the correlation between the skill output and the target output, and helping to reduce the occurrence of mismatching of output skills and targets.

[0013] In addition, the control method, device, electronic device and storage medium of the virtual object according to the embodiments of the present application can also improve the decision-making ability of the game AI, so that the game AI has a higher anthropomorphic effect and intelligence level.

[0014] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0016] Figure 1 A flowchart showing a method for controlling a virtual object provided by an exemplary embodiment of the present application;

[0017] Figure 2 A schematic diagram showing the skill release shape and skill size provided by an exemplary embodiment of the present application;

[0018] Figure 3 A flowchart showing the steps of obtaining the game status characteristics of the current virtual game provided by an exemplary embodiment of the present application;

[0019] Figure 4 A flowchart showing steps of forming an action strategy for a first virtual object provided by an exemplary embodiment of the present application;

[0020] Figure 5 A schematic diagram showing a first processing scheme for forming an action strategy for a first virtual object provided by an exemplary embodiment of the present application;

[0021] Figure 6 A schematic diagram of a network structure corresponding to the first processing solution provided by an exemplary embodiment of the present application is shown;

[0022] Figure 7 A schematic diagram showing a second processing scheme for forming an action strategy for a first virtual object provided by an exemplary embodiment of the present application;

[0023] Figure 8 A schematic diagram showing the structure of a control device for a virtual object provided by an exemplary embodiment of the present application;

[0024] Fig. 9 A schematic diagram of the structure of an electronic device provided by an exemplary embodiment of the present application is shown. DETAILED DESCRIPTION

[0025] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the drawings in the present application only serve the purpose of explanation and description and are not used to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn in real proportion. The flowchart used in this application shows the operations implemented according to some embodiments of the present application. It should be understood that the operations of the flowchart can be implemented out of sequence, and the steps without logical context can be reversed in order or implemented simultaneously. In addition, those skilled in the art, under the guidance of the content of the present application, can add one or more other operations to the flowchart, or remove one or more operations from the flowchart.

[0026] The terms "a", "an", "the" and "said" are used in this specification to indicate the presence of one or more elements / components / etc.; the terms "including" and "having" are used to express an open-ended inclusion and mean that additional elements / components / etc. may exist in addition to the listed elements / components / etc.; the terms "first" and "second" etc. are used only as labels and are not intended to limit the quantity of their objects.

[0027] It should be understood that in the embodiments of the present application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" is merely a way to describe the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the previous and subsequent associated objects are in an "or" relationship. "Including A, B and / or C" means including any one, any two, or any three of A, B, and C.

[0028] It should be understood that in the embodiments of the present application, "B corresponding to A", "B corresponding to A", "A corresponds to B", or "B corresponds to A" means that B is associated with A, and B can be determined according to A. Determining B according to A does not mean determining B only according to A, and B can also be determined according to A and / or other information.

[0029] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0030] In the process of game development, game AI can replace the role of testers, and by introducing game AI to participate in the game, test data can be obtained to achieve game performance testing. For example, in the process of game application, if there are not enough players in the game, game players can actively introduce game AI to play games with real people.

[0031] Game AI is an important part of competitive games. Traditional game AI mainly adopts rule-based solutions, such as finite state machines and behavior trees, but the generated behavioral strategies are specific, the combat intensity is low, the degree of anthropomorphism is not enough, and it is easy to be recognized by other players. In recent years, reinforcement learning methods have given AI powerful decision-making capabilities, making it impressive in the field of games. However, the general expression ability of existing reinforcement learning solutions is insufficient. For example, for a variety of game play and occupation types, different AI combat models need to be built for various game play and occupations. That is, different game play requires independent design of modeling solutions, and new occupations require retraining of game AI, etc. This consumes a lot of computing resources, is time-consuming and labor-intensive, and will cause an increase in development and maintenance costs.

[0032] In response to at least one of the above problems, the present application proposes a control method, device, electronic device and storage medium for a virtual object to provide a general combat model solution for game AI.

[0033] First, the names involved in the embodiments of the present application are introduced.

[0034] Terminal equipment:

[0035] The terminal device involved in the embodiments of the present application mainly refers to an intelligent device used to provide a game screen (such as a relevant setting / configuration interface in the game, an interface for presenting a game scene), and capable of controlling and operating a virtual character. The terminal device may include, but is not limited to, any one of the following devices: a smart phone, a tablet computer, a portable computer, a desktop computer, a game console, a personal digital assistant (PDA), an e-book reader, an MP4 (Moving Picture Experts Group Audio Layer IV, dynamic image expert compression standard audio layer 4) player, etc. The terminal device is installed and running with an application that supports game scenes, such as an application that supports three-dimensional game scenes. The application may include, but is not limited to, any one of a virtual reality application, a three-dimensional map program, a military simulation program, a MOBA game, a multiplayer gunfight survival game, and a third-person shooting game (TPS, Third-Personal Shooting Game). Optionally, the application may be a stand-alone application, such as a stand-alone 3D game program, or a network-connected application.

[0036] Graphical User Interface:

[0037] It is an interface display format for communication between people and computers, allowing users to manipulate icons, logos or menu options on the screen using input devices such as a mouse, keyboard and / or game controller, and also allowing users to manipulate icons or menu options on the screen by performing touch operations on the touch screen of a touch terminal to select commands, start programs or perform other tasks. Exemplarily, the graphical user interface may include virtual objects such as game characters, NPC characters, and AI characters that execute game logic in the game scene. The interface includes any visible controls or elements that can be seen, for example, game controls (such as skill controls, mobile controls, function controls, etc.), indicator signs (such as direction indicator signs, role indicator signs, etc.), information display areas (such as number of people defeated, game time, etc.), or game setting controls (such as system settings, stores, gold coins, etc.), and may also include controls such as pictures, input boxes, and text boxes, some of which respond to user operations.

[0038] Game scene:

[0039] It is a virtual environment displayed (or provided) when the application is running on a terminal device or server. Optionally, the game scene is a simulation of the real world, or a semi-simulation and semi-fictional virtual environment, or a purely fictional virtual environment. The game scene can be any one of a two-dimensional virtual environment, a 2.5-dimensional virtual environment, and a three-dimensional virtual environment. The virtual environment can be the sky, land, ocean, etc. Among them, the game scene is a scene in which the user controls the complete game logic of the virtual character. Optionally, the game scene is also used for a virtual environment battle between at least two virtual characters, and there are virtual resources in the game scene that can be used by at least two virtual characters.

[0040] Virtual Objects:

[0041] Refers to a virtual object in a game scene, which may be a virtual character controlled by a player, including but not limited to at least one of a virtual person, a virtual animal, and an anime character, and may also be a virtual object (NPC) controlled by a non-player. Optionally, when the virtual environment is a three-dimensional virtual environment, the virtual object may be a three-dimensional virtual model, and each virtual object has its own shape and volume in the three-dimensional virtual environment, occupying a portion of the space in the three-dimensional virtual environment. Optionally, the virtual object is a three-dimensional character constructed based on three-dimensional human skeleton technology, and the virtual object achieves different external images by wearing different skins. In some implementations, the virtual object may also be implemented using a 2.5-dimensional or 2-dimensional model, which is not limited in the embodiments of the present application.

[0042] There may be multiple virtual objects in the virtual scene, which are virtual characters controlled by the player (i.e., characters controlled by the player through input devices) and / or artificial intelligence (AI) set in the game scene battle through training. Optionally, the virtual object is a virtual character competing in the game scene. Optionally, the number of virtual characters in the game scene battle is preset, or dynamically determined according to the number of terminal devices joining the virtual game, which is not limited in the embodiments of the present application.

[0043] In an optional embodiment, the terminal device may be a local terminal device. Taking a game as an example, the local terminal device stores a game program and is used to present a game screen. The local terminal device is used to interact with the player through a graphical user interface, that is, the game program is downloaded and installed by an electronic device and run conventionally. The local terminal device may provide the graphical user interface to the player in a variety of ways, for example, it may be rendered and displayed on a display screen of the terminal device, or provided to the player through a holographic projection. For example, the local terminal device may include a display screen and a processor, the display screen is used to present a graphical user interface, the graphical user interface includes a game screen, and the processor is used to run the game, generate a graphical user interface, and control the display of the graphical user interface on the display screen.

[0044] The application scenarios to which the present application is applicable are introduced. The present application can be applied to the field of game technology, in which multiple players participating in the game join the same virtual game.

[0045] Before entering the virtual game, the player can select different role attributes, such as identity attributes, for the virtual objects in the virtual game. By assigning different role attributes to determine different camps, the player can win the game by performing the tasks assigned by the game in different game stages of the virtual game. For example, multiple virtual objects with role attribute A can "eliminate" virtual objects with role attribute B in the battle stage to win the game. Here, role attributes can also be randomly assigned to each virtual object participating in the virtual game when entering the virtual game.

[0046] The implementation environment provided in one embodiment of the present application may include: a first terminal device, a server, and a second terminal device. The first terminal device and the second terminal device communicate with the server respectively to achieve data communication. In this embodiment, the first terminal device and the second terminal device are respectively installed with an application program that executes the control method of the virtual character provided by the present application, and the server is a server end that executes the control method of the virtual character provided by the present application. Through the application program, the first terminal device and the second terminal device can communicate with the server respectively.

[0047] Taking the first terminal device as an example, the first terminal device establishes communication with the server by running an application. In an optional embodiment, the server establishes a virtual game according to the game request of the application. Among them, the parameters of the virtual game can be determined according to the parameters in the received game request. For example, the parameters of the virtual game may include the number of people participating in the virtual game, the level of the characters participating in the virtual game, etc. When the first terminal device receives the response of the game server, the game scene corresponding to the virtual game is displayed through the graphical user interface of the first terminal device. The first terminal device is a device controlled by the first user. The virtual object displayed in the graphical user interface of the first terminal device is the player character controlled by the first user. The first user inputs an operation instruction through the graphical user interface to control the virtual object to perform a corresponding operation in the game scene.

[0048] Taking the second terminal device as an example, the second terminal device establishes communication with the server by running an application. In an optional embodiment, the server establishes a virtual game according to the game request of the application. Among them, the parameters of the virtual game can be determined according to the parameters in the received game request. For example, the parameters of the virtual game may include the number of people participating in the virtual game, the level of the characters participating in the virtual game, etc. When the second terminal device receives the response from the server, the game scene corresponding to the virtual game is displayed through the graphical user interface of the second terminal device. The second terminal device is a device controlled by the second user, and the virtual object displayed in the graphical user interface of the second terminal device is the player character controlled by the second user. The second user inputs operation instructions through the graphical user interface to control the virtual object to perform corresponding operations in the virtual scene.

[0049] The server performs data calculation based on the game data reported by the first terminal device and the second terminal device, and synchronizes the calculated game data to the first terminal device and the second terminal device, so that the first terminal device and the second terminal device control the graphical user interface to render the corresponding game scene and / or virtual object according to the synchronization data sent by the game server.

[0050] In this embodiment, the virtual object controlled by the first terminal device and the virtual object controlled by the second terminal device are virtual objects in the same virtual game. Among them, the virtual object controlled by the first terminal device and the virtual object controlled by the second terminal device can have the same role attributes, or can have different role attributes. The virtual object controlled by the first terminal device and the virtual object controlled by the second terminal device belong to the same camp, or belong to different camps with hostile relations. The game AI virtual object participating in the virtual game can have a camp, for example, it can belong to the camp to which the virtual object controlled by the first terminal device belongs, or the camp to which the virtual object controlled by the second terminal device belongs. The game AI virtual object may not have a camp to which it belongs, and interact with other virtual objects in the virtual game.

[0051] It should be noted that a virtual game may include two or more virtual objects, and different virtual objects may correspond to different terminal devices. That is to say, in a virtual game, there are more than two terminal devices that send and synchronize game data with the game server respectively.

[0052] The control method of the virtual character provided in the embodiment of the present application can be applied to any of virtual reality applications, three-dimensional map programs, military simulation programs, multiplayer online tactical competitive games (MOBA), multiplayer gun battle survival games, third-person combat games, and first-person combat games.

[0053] The control method of the virtual character in one embodiment of the present application can be run on a local terminal device or a server. When the control method is run on a server, the method can be implemented and executed based on a cloud interaction system, wherein the cloud interaction system includes a server and a client device.

[0054] In an optional embodiment, various cloud applications can be run under the cloud interaction system, such as cloud games. Taking cloud games as an example, cloud games refer to a game mode based on cloud computing. In the operation mode of cloud games, the operating body of the game program and the main body of the game screen presentation are separated. The storage and operation of the control method are completed on the cloud game server. The role of the client device is used for receiving and sending data and presenting the game screen. For example, the client device can be a display device with data transmission function close to the user side, such as a mobile terminal, a TV, a computer, a handheld computer, etc.; but the cloud game server in the cloud is used for information processing. When playing the game, the player operates the client device to send an operation instruction to the cloud game server. The cloud game server runs the game according to the operation instruction, encodes and compresses the game screen and other data, and returns it to the client device through the network. Finally, the client device decodes and outputs the game screen.

[0055] In an optional embodiment, taking a game as an example, a local terminal device stores a game program and is used to present a game screen. The local terminal device is used to interact with the player through a graphical user interface, that is, the game program is downloaded and installed by an electronic device and run conventionally. The local terminal device may provide the graphical user interface to the player in a variety of ways, for example, it may be rendered and displayed on a display screen of the terminal, or provided to the player through a holographic projection. For example, the local terminal device may include a display screen and a processor, the display screen is used to present a graphical user interface, the graphical user interface includes a game screen, and the processor is used to run the game, generate a graphical user interface, and control the display of the graphical user interface on the display screen.

[0056] In a possible implementation, an embodiment of the present invention provides a method for controlling a virtual character, providing a graphical user interface through a terminal device, wherein the terminal device may be the local terminal device mentioned above, or may be a client device in the cloud interaction system mentioned above.

[0057] To facilitate understanding of the present application, the control method, device, electronic device and storage medium of the virtual character provided in the embodiments of the present application are introduced in detail below.

[0058] See also Figure 1 , which is a flowchart of a method for controlling a virtual object provided in an exemplary embodiment of the present application, which is generally applied to a game server, such as the cloud game server described above, but the present application is not limited thereto.

[0059] A graphical user interface is provided through a terminal device, and at least a part of the game scene of the current virtual game is displayed on the graphical user interface, and the game scene includes a first virtual object that is not controlled by the player. Exemplarily, the first virtual object can be executed at a preset decision point in the process of the current virtual game. Figure 1 The processing steps are shown.

[0060] In an embodiment of the present application, the first virtual object is an artificial intelligence character controlled by a non-player participating in the virtual game. The first virtual object can be configured to move in the game scene according to an action strategy, for example, controlling the first virtual object to run, jump, crawl, etc., and can also control the first virtual object to use the virtual skills, virtual props, etc. provided by the virtual game to interact with other virtual objects in the game.

[0061] For example, multiple decision points will appear in sequence during the course of this virtual game, and each decision point in the course of this virtual game can be executed. Figure 1For example, every time a decision point is triggered, the virtual game can automatically execute the control method of the virtual object of the present application to reduce the intervention of game players. The decision point can be pre-embedded in the virtual game, and when the game process meets the triggering condition of the decision point, a decision is triggered. Alternatively, the decision point can also be automatically triggered at a preset time interval during the game process. In addition, it can also be executed only at some decision points. Figure 1 Processing mechanism, other mechanisms are executed for other decision points, and this application does not impose any restrictions on this.

[0062] like Figure 1 As shown, the control method of the virtual object of the exemplary embodiment of the present application specifically includes:

[0063] Step S101: extracting a first attribute state feature of a first preset output in this virtual game.

[0064] In an embodiment of the present application, the action space for the virtual object is divided into two links: skill output and target output, and the first preset output is one of the skill output and the target output.

[0065] Here, skill output may include but is not limited to professional skills. In the embodiment of the present application, for professional skills with weak universality, a universal skill description representation is used to express them. Exemplarily, professional skills may be expressed using at least one of the following skill description representations: skill positioning, skill release shape, skill size for describing the skill release shape, skill release range, skill release target, skill cooling time, and skill segment number.

[0066] See Table 1 below, which shows the classification of skill positioning. By way of example, the skill positioning in the embodiment of the present application includes 22 types, which can cover all skills of all occupations, and each skill can be composed of a combination of two positionings in the table.

[0067] Table 1

[0068]

[0069] For example, the skill release shapes may include but are not limited to the following six categories: self, rectangle, circle, fan, single body and trapezoid, and each skill release shape corresponds to the size of the skill, such as Figure 2 shown.

[0070] The skill release target is used to characterize the target of the skill, that is, it indicates what type of target the skill is released on, including but not limited to at least one of the following: self, teammates, self and teammates, enemies, all. The skill release range indicates the maximum distance at which the skill can be released, which can be represented by a single continuous value scalar. In addition to the above general skill representations, professional skills can also include skill cooldown time, whether it can be used, and skill segment number.

[0071] Here, the skill output may include not only professional skills but also displacement skills. Exemplarily, the displacement skills may include but are not limited to displacement skill types and flag words, where the flag words are used to indicate the available status of the displacement skills at the current decision point, that is, whether the displacement skills are available is indicated by assigning a value to the flag word.

[0072] The target output is used to indicate the target to which the skill output corresponds, and the target output includes at least one of the following objects: the first virtual object itself, the second virtual object belonging to the same team as the first virtual object, and the third virtual object belonging to the enemy team as the first virtual object.

[0073] In the embodiment of the present application, skill description representation is used to express occupation-related skills, so that the game AI can fully understand and abstract the skill characteristics it carries.

[0074] Step S102: extracting game state features of this virtual game.

[0075] Here, the game state characteristics include various characteristics that can be used to characterize the game state of the virtual game, as shown below. Figure 3 Let's introduce the process of obtaining game state characteristics.

[0076] Figure 3 A flowchart showing the steps of obtaining the game status characteristics of the current virtual game provided by an exemplary embodiment of the present application.

[0077] Reference Figure 3 In step S1021, the game status information at the current decision point in this virtual game is obtained.

[0078] Here, when the decision point is triggered, the game status information of the current virtual game at the time of triggering can be obtained. Exemplarily, the game status information may include but is not limited to multiple attribute fields for characterizing the game status of the current virtual game, and the multiple attribute fields include but are not limited to at least one of the following items: skill status information, radar map information, terrain information, and team status information.

[0079] As an example, the skill state information includes information for characterizing various professional skills and displacement skills included in this virtual game, and the terrain information may include terrain information within a preset range of the position of the first virtual object in the game scene at the current decision point. For example, the preset range may be a range corresponding to the game screen presented on the graphical user interface, or may be a maximum skill release range corresponding to the first virtual object. The terrain information is used to characterize terrain information used to build the game scene, such as raised peaks, sunken canyons, hillsides, and walls of different materials.

[0080] The radar chart information includes a radar game map, which may include but is not limited to at least one of the following items: the position of the first virtual object and its corresponding perspective, the first virtual object, the teammates of the first virtual object, and the enemy NPC of the first virtual object. Exemplarily, the positional relationship between the first virtual object and the teammates and / or enemies can also be displayed in the radar game map. The team status information includes the various states of all virtual objects participating in this virtual game, such as health value, position, speed value, damage value, etc.

[0081] In step S1022, feature extraction is performed on the multiple attribute fields respectively to obtain multiple game attribute features.

[0082] Here, feature extraction can be performed on multiple attribute fields of the skill status information to obtain multiple game attribute features corresponding to the skill information. Similarly, radar map information, terrain information, and team status information can be extracted separately to obtain game attribute features corresponding to each information.

[0083] It should be understood that various existing feature extraction methods can be used to perform feature extraction, and this application does not limit the content of this part.

[0084] In step S1023, all game attribute features are concatenated to obtain the game state features of this virtual game.

[0085] In a preferred embodiment, the game state feature may include a game state vector, each element of which corresponds to a pair of game attribute features. In other words, the feature concatenation may refer to arranging the game attribute features in a predetermined order to form a game state vector.

[0086] Preferably, the above-mentioned multiple attribute fields may also include situation information and / or self-category information. For example, the situation information is used to characterize whether the first virtual object can win at the current moment. For example, when the situation information is configured as a value of 1, it means: the current number of survivors of the own side> the number of survivors of the enemy, or the current number of survivors of the own side=the number of survivors of the enemy and the total damage of the own side>the total damage of the enemy, and the value is 0. The self-category information can be used to characterize the game level of the virtual object, or the type of the virtual object (such as attack type, auxiliary type).

[0087] In this case, the game state features of this virtual game can be obtained in the following way: input the game attribute features corresponding to the skill state information, radar map information, terrain information, and team state information into at least one fully connected layer to obtain the attribute combination features, and splice the game attribute features corresponding to the situation information, the game attribute features corresponding to the own category information, and the attribute combination features to obtain the game state features of this virtual game. This helps to improve the flexibility of obtaining game state features, that is, when transplanting between different games and different skills, it can be flexibly determined whether to introduce the own category information and / or situation information. In addition, the above feature processing method also helps to improve the winning rate of virtual objects.

[0088] In the embodiment of the present application, situation information is introduced to guide the game AI to adopt different strategies, such as aggressive or conservative, through the situation information indicating whether the team can win at the current moment, which helps to improve the combat strength of the game AI. Through testing and verification, the winning rate of adding situation information and not adding situation information is 59.3%: 40.7%. Compared with the existing behavior tree solution, the winning rate can be maintained at more than 99%.

[0089] By extracting the above-mentioned attribute state characteristics and game state characteristics, the versatility of game AI can be improved and the development time and maintenance cost of game AI can be reduced.

[0090] return Figure 1 , step S103: determining a first output action from a first preset output according to the extracted first attribute state feature and the game state feature.

[0091] In an embodiment of the present application, the first preset output may include multiple first actions. In this case, the first attribute state feature may include at least one attribute feature corresponding to each first action, and the first output action is one determined from the multiple first actions.

[0092] Exemplarily, the first output action can be determined in the following manner: for each first action, the game state feature is fused with at least one attribute feature corresponding to the first action to obtain a fused feature corresponding to the first action; based on the fused feature corresponding to each first action, the first output action is determined from multiple first actions.

[0093] In a preferred embodiment, the first attribute state characteristic can be represented in a matrix form. For example, the first attribute state characteristic can include a first preset characteristic matrix, each row of the first preset characteristic matrix corresponds to a first action, and each element in each row is used to represent an attribute characteristic corresponding to the first action.

[0094] In this case, the fusion feature corresponding to each first action can be obtained by performing a dot product between the first preset feature matrix and the game state vector to obtain a first fusion feature vector, wherein each element in the first fusion feature vector is used to characterize the fusion feature of a corresponding first action. Here, the fusion feature corresponding to each first action is used to characterize the probability of the action being selected.

[0095] Here, the game state vector can be an M-dimensional column vector, and each row in the first preset feature matrix is ​​dot-producted with the game state vector to obtain the fusion feature of the first action corresponding to the row, and the fusion features of each first action are used to form an M-dimensional first fusion feature column vector. It should be understood that in order to make the above matrix and vector dot-product operations possible, it is necessary to ensure that the number of columns of the matrix is ​​consistent with the dimension of the vector. Exemplarily, various methods can be used to make the two computable, such as through the processing of a fully connected layer.

[0096] In a preferred embodiment, the first action can be screened by introducing a first available action mask, which can refer to an available mask related to the first preset output, that is, the available mask is used to characterize the first action that can be used by the first virtual object at the current decision point. Based on this, the first available action mask is used to screen out the available first actions from multiple first actions. If the available first action is not unique, the available first actions can be further screened based on the fusion features. For example, the first action with the largest value of the fusion feature is determined as the first output action.

[0097] Step S104: Based on the second attribute state characteristics of the second preset output in the current virtual game and the attribute state characteristics of the determined first output action, an action strategy for the first virtual object is formed to control the first virtual object to execute the action strategy in the game scene. Here, the second preset output is the other of the skill output and the target output.

[0098] In an embodiment of the present application, the output action of the first virtual object is divided into two links: skill output and target output. In order to increase the correlation between skill output and target output, the present application adopts an autoregressive operation, that is, one of the output actions is used as the input of another output action, so that the output action can be adjusted and optimized according to the previous output action.

[0099] In addition, based on the correlation between the two actions, the mismatching of output skills and targets can also be reduced. Tests have shown that the mismatching rate is reduced by 62.5%. By introducing the skill-related target output legal matrix (such as the logo word in the displacement skill), the mismatching situation can be completely eliminated, thereby improving the training efficiency of the game.

[0100] Refer to the following Figure 4 Let's introduce the process of forming an action strategy for the first virtual object.

[0101] Figure 4 A flowchart showing the steps of forming an action strategy for a first virtual object provided by an exemplary embodiment of the present application.

[0102] Reference Figure 4 , in step S1041, the game state feature is superimposed on the attribute state feature of the first output action to obtain the superimposed feature of the first output action;

[0103] In the case where the first attribute state feature includes the first preset feature matrix, the attribute state feature of the first output action includes each element in the row vector corresponding to the first output action extracted from the first preset feature matrix.

[0104] In the case where the game state feature is a game state vector, the superposition feature of the first output action can be obtained by vector × multiplication. For example, each element in the vector corresponding to the first output action and each element in the game state vector can be multiplied in pairs according to bit correspondence to obtain the superposition feature of the first output action.

[0105] In step S1042, the second attribute state feature of the second preset output is fused with the superimposed feature of the first output action to form an action strategy for the first virtual object.

[0106] In the embodiment of the present application, the second preset output may include multiple second actions, and the second attribute state characteristics include at least one attribute characteristic corresponding to each second action.

[0107] Exemplarily, an action strategy for the first virtual object can be formed in the following manner: for each second action, the attribute feature corresponding to the second action is fused with the superimposed feature of the first output action to obtain the fused feature corresponding to the second action; based on the fused feature corresponding to each second action, a second output action is determined from multiple second actions; based on the superimposed feature of the first output action and the attribute state feature of the second output action, an action strategy for the first virtual object is formed.

[0108] In a preferred embodiment, the superimposed feature of the first output action is an action superimposed vector, which can represent the second attribute state feature in matrix form. For example, the second attribute state feature can include a second preset feature matrix, each row of the second preset feature matrix corresponds to a second action, and each element in each row is used to represent an attribute feature corresponding to the second action.

[0109] In this case, the fusion feature corresponding to each second action is obtained in the following manner: dot product is performed on the second preset feature matrix and the action superposition vector to obtain a second fusion feature vector, and each element in the second fusion feature vector is used to characterize the fusion feature of a corresponding second action.

[0110] Here, the action superposition vector may be an M-dimensional column vector, and each row in the second preset feature matrix is ​​dot-producted with the action superposition vector to obtain the fusion feature of the second action corresponding to the row, and the fusion features of each second action are used to form an M-dimensional second fusion feature column vector. It should be understood that in order to enable the dot product operation of the above matrix and the vector, it is necessary to ensure that the number of columns of the matrix is ​​consistent with the dimension of the vector.

[0111] In a preferred embodiment, the second action can be screened by introducing a second available action mask, which can refer to an available mask related to the first preset output, that is, the available mask is used to characterize the available second action associated with the first virtual object and the first preset output at the current decision point. Based on this, the available second action is screened out from multiple second actions through the second available action mask. If the available second action is not unique, the available second action can be further screened based on the fusion feature. For example, the second action with the largest value of the fusion feature is determined as the second output action.

[0112] The action strategy for the first virtual object includes a combination of a first output action and a second output action, that is, a combination of skill output and target output, wherein the superimposed characteristics of the first output action cover the state characteristics of the action itself and the game state characteristics to capture the correlation and dependency between the action and the game state, and use the first output action as the input of the second output action, thereby increasing the correlation between the two actions. Compared with the prior art solutions, the combat AI of the present application has been greatly improved in combat intensity and versatility.

[0113] In one example, the present application uses an autoregressive neural network to process game state information, and designs two autoregressive structures for two categories of action space (skill output and target output).

[0114] In the first processing scheme, the skill output and skill feature matrix can be obtained according to the neural network reasoning, and then the skill feature corresponding to the skill is selected, and after being fused with the feature of the game state information, the target is output.

[0115] The following is combined with the situation where the first preset output is skill output Figure 5 and Figure 6 The processing procedure of the first solution mentioned above is introduced.

[0116] Figure 5 A schematic diagram showing a first processing scheme for forming an action strategy for a first virtual object provided by an exemplary embodiment of the present application.

[0117] In this example, the first preset output is a skill output, which includes occupational skills and displacement skills, and the first preset feature matrix includes a skill feature matrix.

[0118] like Figure 5 As shown, the skill output information in this virtual game is obtained. The skill output information may include action information of various professional skills and displacement skills in this virtual game. Exemplarily, the displacement skills may include but are not limited to at least one of the following items: sprinting, jumping, dodging, and walking.

[0119] Feature extraction is performed on the acquired skill output information to obtain a plurality of skill attribute features, and a skill feature matrix is ​​formed through the acquired plurality of skill attribute features.

[0120] See also Figure 6The specific method of forming the skill feature matrix in the dashed box shown includes: extracting features of each professional skill in this virtual game to form a professional skill feature matrix, Skill_desc represents multiple professional skills, Scope, cd, etc include attribute features corresponding to the skill, such as skill release range, cooling time, etc. Extracting features of each displacement skill in this virtual game to form a displacement skill feature matrix, Action Id represents multiple types of displacement skills, and Legal represents a flag word.

[0121] For example, for displacement skills, the Id of the displacement skill is first embedded and then fused with the remaining features. For professional skills, the general skill representation is used, the skill location numbers in Table 1 are embedded, and the skill release shape and skill release target are encoded using one-hot encoding, and then the remaining features are fused.

[0122] The formed professional skill feature matrix and displacement skill feature matrix are subjected to feature shape alignment processing. For example, the professional skill feature matrix and displacement skill feature matrix can be subjected to FC (Fully Connected Layers) processing respectively to perform feature shape alignment so that the number of columns of the two are consistent.

[0123] The occupational skill feature matrix and the displacement skill feature matrix after feature shape alignment are concatenated to obtain a skill feature matrix. Exemplarily, the above matrix concatenation may refer to placing each row of the occupational skill feature matrix below the last row of the displacement skill feature matrix, or placing each row of the displacement skill feature matrix below the last row of the occupational skill feature matrix, or arranging the rows of the two matrices in a certain order or randomly.

[0124] In an embodiment of the present application, a deep reinforcement learning algorithm is used to train a combat AI, and firstly, the state s, action a and reward r are defined.

[0125] Exemplarily, the state S used in this application includes the following eight parts: occupational skill information, displacement skill information, radar map information, terrain information, team status information, gain status (buff) information, situation information and self-category information. For occupational skill information with weak universality, this application uses universal skill representation to express it, and universal skill representation includes but is not limited to: skill positioning, skill release shape, skill size, skill release target, and skill release range.

[0126] Action a of this application uses a combination of two actions: skills and targets. Skill output includes displacement skills and professional skills, and targets include oneself, teammates, and opponents. When executing a decision, a skill is selected according to the neural network, and then a target is selected according to the skill, and finally a combined action is executed.

[0127] The reward function r of the present application adopts a combination of personal rewards and team rewards. Personal rewards include killing opponents, damage, injury and treatment, and team rewards include the number of team deaths and the final victory or defeat. Optionally, in order to promote the active combat of the game AI, the reward function also introduces a time-related penalty, that is, a penalty is executed every time a decision is made. Exemplarily, the above-mentioned reward function can be used as an evaluation indicator for game AI training. After each training is completed, the parameters in the autoregressive network are adjusted based on the reward function to change the training strategy so that the strategy ends the battle faster.

[0128] In a preferred embodiment of the present application, the multiple attribute fields also include gain status information, and the gain status information includes status attributes corresponding to all virtual objects participating in the current virtual game.

[0129] In this case, the game attribute features corresponding to the team status information can be obtained in the following ways: feature extraction is performed on the attribute field used to characterize the gain status information to obtain multiple gain status features, the multiple gain status features are compressed to obtain a fused gain feature, feature extraction is performed on the attribute field used to characterize the team status information to obtain the team status features corresponding to each virtual object participating in this virtual game, and the gain features are respectively spliced ​​with the team status features corresponding to each virtual object to form the game attribute features corresponding to the team status information.

[0130] exist Figure 6 In the network structure shown, a deep neural network based on autoregression is designed to process game state information and output actions: including the use of a Multi-Head Attention structure to process skill information and team state information, thereby capturing the correlation and dependency between skills and team members.

[0131] For the buff information, its corresponding ID is embedded and encoded, and then fused with the remaining features. Then, the features are compressed through two layers of FC and one layer of Maxpooling to obtain a fused gain feature.

[0132] For example, for each virtual object, a fused gain feature can be added after each team state feature corresponding to the virtual object, or placed before each team state feature, or placed at other positions for splicing. The team state feature of each virtual object after splicing is directly processed by Embedding and Multi-Head Attention.

[0133] For displacement skills, the displacement skill ID is first embedded, then fused with the remaining features, and finally processed by Embedding and Multi-Head Attention.

[0134] For professional skills, the general skill representation is used to perform the embedding operation on the skill location numbers in Table 1, and the skill release shape and skill release target are encoded using one-hot encoding. The remaining features are then fused and finally processed by Embedding and Multi-Head Attention.

[0135] Two layers of FC are used to process other scalar states (such as radar map information, terrain information, own category information, and situation information).

[0136] In this example, each game attribute feature corresponding to displacement skills, professional skills, radar map, terrain information, and team status information is input into two layers of FC to obtain attribute combination features, and then the game attribute features corresponding to the situation information, the game attribute features corresponding to the own category information, and the attribute combination features are concatenated to obtain the game status features of this virtual game.

[0137] The above-mentioned game state features are dot-producted with the skill feature matrix after embedding processing, and the dot product results are processed through the available skill mask to determine the output skill output. The skill feature matrix is ​​gather+sigmoid processed to obtain a row of attribute features corresponding to the skill output, and then the attribute features corresponding to the skill output are superimposed (× multiplied) with the game state features after embedding processing to obtain the superimposed features of the skill output, and the superimposed features of the skill output are dot-producted (dot) with the target feature matrix to obtain the fusion features corresponding to each target, and the dot product results are processed through the available target mask to determine the output target output.

[0138] Each row of the target feature matrix corresponds to a target, and each element in each row of the target feature matrix corresponds to the attribute characteristics of the target. Here, each target forming the target feature matrix may include but is not limited to all virtual objects participating in this virtual game, for example, the first virtual object itself, at least one second virtual object that is a teammate of the first virtual object (which may be a virtual object controlled by other terminal devices or an NPC), and at least one third virtual object that is an enemy of the first virtual object (which may be a virtual object controlled by other terminal devices or an NPC).

[0139] In one example, a row of attribute features corresponding to a target may be the game attribute features obtained by extracting the team status information as described above, which may include, but are not limited to: the health value, position, speed value, and damage value corresponding to the target.

[0140] In another example, a row of attribute features corresponding to a target may also be the game attribute features obtained by extracting the team state information and the gain state information. Exemplarily, a fused gain feature is obtained by compressing multiple gain state features, and the fused gain feature is spliced ​​with the team state feature corresponding to the target to form the game attribute feature of the target.

[0141] For output actions use Figure 6 The first processing scheme is an autoregressive network, which first outputs the skill output, then selects the corresponding skill feature based on the selected skill output, and then performs a dot product operation on the skill feature and the game state feature, so that the target selection is associated with the skill output. Before the target output, introducing the available target mask related to the skill can avoid the situation where the skill output and the target output do not match, thereby accelerating the policy learning process.

[0142] In the second processing scheme, the target output and target feature matrix can be obtained according to the neural network inference, and then the target feature corresponding to the target is selected and fused with the feature of the game state information to output the skill (such as Figure 7 shown).

[0143] The difference between the second processing scheme and the first processing scheme is that in the first processing scheme, the skills are output first and then the targets are selected, while in the second processing scheme, the targets are output first and then the skills are selected. The two schemes are similar in terms of feature extraction, processing, superposition and fusion methods in the processing process, and this application will not go into details about this part.

[0144] Figure 5 and Figure 7Two different autoregressive structures are shown. The essence of these two structures is to strengthen the connection between outputs, thereby reducing the mismatch between skills and target selection. After testing and verification, the winning rate of the first processing scheme is approximately 60%:40% compared with the second processing scheme.

[0145] In a preferred embodiment of the present application, the control method of the virtual object may further include: after executing the action strategy for the first virtual object, executing a reward strategy. Exemplarily, the reward strategy includes individual rewards and / or team rewards, and the individual rewards include positive impact rewards for characterizing the feedback results of the action strategy and negative impact rewards for settlement of decision points.

[0146] The control method of the virtual object of the present application can at least achieve the following effective effects:

[0147] To reduce development time, this application trains combat AI in a martial arts scenario, and subsequently integrates the Imperial City 1v1 and intelligent NPC scenarios. Compared with the existing reinforcement learning solution, it effectively shortens the development cycle of integration.

[0148] Reduce maintenance costs. This solution has a certain degree of scalability when adding new professions. The existing reinforcement learning solution needs to be retrained when adding a new profession, which takes about 650,000 rounds. This application only needs to fine-tune the original model for 50,000 rounds, and the winning rate can exceed the existing solution, which is 13 times more efficient.

[0149] High strength and anthropomorphism: Compared with the existing behavior tree solution, the winning rate of this application remains above 99%, and the AI ​​trained through reinforcement learning is closer to humans in combat, and can dodge skills, cooperate with teammates to focus on the target, escape with low health, etc.

[0150] In order to solve the problem of poor universality of current game AI combat models, this solution uses a deep reinforcement learning algorithm, adopts a universal skill description representation to express the occupational skill status with weak universality, combines the Embedding technology to design an autoregressive network structure, and introduces the situation status to improve AI expressiveness, so that the combat model can be well compatible with different types of intelligent agents and gameplay, reducing the time and cost of game AI development.

[0151] Based on the same application concept, an embodiment of the present application also provides a control device for a virtual object corresponding to the method provided in the above embodiment. Since the principle of solving the problem by the device in the embodiment of the present application is similar to the control method for a virtual object in the above embodiment of the present application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0152] Figure 8 This is a schematic diagram of the structure of a virtual object control device provided by an exemplary embodiment of the present application. Figure 8As shown, a graphical user interface is provided by a terminal device, and the graphical user interface displays at least part of the game scene of the current virtual game, and the game scene includes a first virtual object that is not controlled by the player. The control device 200 of the virtual object includes:

[0153] An attribute feature extraction module 210 extracts a first attribute state feature of a first preset output in the current virtual game, where the first preset output is one of a skill output and a target output;

[0154] A game feature extraction module 220 extracts game state features of the current virtual game;

[0155] The action output module 230 determines a first output action from the first preset output according to the extracted first attribute state feature and game state feature;

[0156] The strategy execution module 240 forms an action strategy for the first virtual object based on the second attribute state characteristics of the second preset output in this virtual game and the attribute state characteristics of the determined first output action, so as to control the first virtual object to execute the action strategy in the game scene, wherein the second preset output is the other of the skill output and the target output.

[0157] In one possible implementation of the present application, the first preset output includes multiple first actions, and the first attribute state characteristics include at least one attribute characteristic corresponding to each first action, wherein the action output module 230 determines the first output action in the following manner: for each first action, the game state characteristics are fused with at least one attribute characteristic corresponding to the first action to obtain the fused characteristic corresponding to the first action, and based on the fused characteristic corresponding to each first action, the first output action is determined from the multiple first actions.

[0158] In one possible implementation of the present application, the first attribute state feature includes a first preset feature matrix, each row of the first preset feature matrix corresponds to a first action, and each element in each row is used to characterize an attribute feature corresponding to the first action, and the game state feature includes a game state vector, wherein the action output module 230 obtains the fusion feature corresponding to each first action in the following manner: dot product the first preset feature matrix with the game state vector to obtain a first fusion feature vector, and each element in the first fusion feature vector is used to characterize the fusion feature of a corresponding first action.

[0159] In one possible implementation of the present application, the first preset output includes skill output, and the skill output includes professional skills and displacement skills. The first preset feature matrix includes a skill feature matrix, wherein the attribute feature extraction module 210 forms the skill feature matrix in the following manner: extracting features from each professional skill in this virtual game to form a professional skill feature matrix, extracting features from each displacement skill in this virtual game to form a displacement skill feature matrix, performing feature shape alignment processing on the formed professional skill feature matrix and displacement skill feature matrix, and splicing the professional skill feature matrix and displacement skill feature matrix after the feature shape alignment processing to obtain a skill feature matrix.

[0160] In a possible implementation manner of the present application, the attribute state feature of the first output action includes each element in a row vector corresponding to the first output action extracted from a first preset feature matrix.

[0161] In a possible implementation of the present application, the game feature extraction module 220 is also used to obtain game status information at the current decision point in this virtual game, the game status information includes multiple attribute fields for characterizing the game status of this virtual game, and the multiple attribute fields include at least one of the following items: skill status information, radar map information, terrain information, team status information; feature extraction is performed on the multiple attribute fields respectively to obtain multiple game attribute features; all game attribute features are spliced ​​to obtain the game status features of this virtual game.

[0162] In a possible implementation manner of the present application, the multiple attribute fields further include situation information and / or own category information.

[0163] Among them, the game feature extraction module 220 is also used to input each game attribute feature corresponding to the skill status information, radar map information, terrain information, and team status information into at least one fully connected layer to obtain attribute combination features, and to splice the game attribute features corresponding to the situation information, the game attribute features corresponding to the own category information, and the attribute combination features to obtain the game status features of this virtual game.

[0164] In a possible implementation of the present application, the multiple attribute fields also include gain status information, wherein the game feature extraction module 220 obtains the game attribute features corresponding to the team status information in the following manner: performing feature extraction on the attribute field used to characterize the gain status information to obtain multiple gain status features, compressing the multiple gain status features to obtain a one-dimensional gain feature, performing feature extraction on the attribute field used to characterize the team status information to obtain the team status feature corresponding to each virtual object participating in this virtual game, and splicing the gain feature with the team status feature corresponding to each virtual object to form the game attribute feature corresponding to the team status information.

[0165] In one possible implementation of the present application, the strategy execution module 240 forms an action strategy for the first virtual object in the following manner: superimposing the game state characteristics onto the attribute state characteristics of the first output action to obtain the superimposed characteristics of the first output action; and fusing the second attribute state characteristics of the second preset output with the superimposed characteristics of the first output action to form an action strategy for the first virtual object.

[0166] In a possible implementation of the present application, the second preset output includes multiple second actions, and the second attribute state characteristics include at least one attribute feature corresponding to each second action, wherein the strategy execution module 240 forms an action strategy for the first virtual object in the following manner: for each second action, the attribute feature corresponding to the second action is fused with the superimposed feature of the first output action to obtain the fused feature corresponding to the second action, based on the fused feature corresponding to each second action, the second output action is determined from the multiple second actions, and the action strategy is formed based on the superimposed feature of the first output action and the attribute state characteristics of the second output action.

[0167] In one possible implementation of the present application, the second attribute state feature includes a second preset feature matrix, each row of the second preset feature matrix corresponds to a second action, and each element in each row is used to characterize an attribute feature corresponding to the second action, and the superposition feature of the first output action includes an action superposition vector, wherein the strategy execution module 240 obtains the fusion feature corresponding to each second action in the following manner: dot product the second preset feature matrix with the action superposition vector to obtain a second fusion feature vector, and each element in the second fusion feature vector is used to characterize the fusion feature of a corresponding second action.

[0168] In one possible implementation of the present application, the strategy execution module 240 is also used to execute a reward strategy after executing the action strategy, wherein the reward strategy includes individual rewards and / or team rewards, and the individual rewards include positive impact rewards for characterizing the feedback results of the action strategy and negative impact rewards for settlement at decision points.

[0169] In one possible implementation of the present application, the skill output includes professional skills, and the professional skills are expressed using at least one skill description representation of the following items: skill positioning, skill release shape, skill size used to describe the skill release shape, skill release range, skill release target, skill cooling time, skill number of segments, and / or, the target output is used to indicate the application target corresponding to the skill output, and the target output includes one of the following objects: the first virtual object itself, a second virtual object belonging to the same team as the first virtual object, and a third virtual object belonging to the enemy team as the first virtual object.

[0170] In a possible implementation manner of the present application, the skill output further includes a displacement skill, and the displacement skill includes a displacement skill type and a flag word, and the flag word is used to indicate the available status of the displacement skill at the current decision point.

[0171] Through the control device for virtual objects in the embodiment of the present application, the versatility of game AI is improved and the development and maintenance costs are reduced.

[0172] See also Fig. 9 , Fig. 9 The schematic diagram of the structure of the electronic device provided by the exemplary embodiment of the present application. Fig. 9 As shown, the electronic device 300 includes a processor 310 , a memory 320 and a bus 330 .

[0173] The memory 320 stores machine-readable instructions executable by the processor 310. When the electronic device 300 is running, the processor 310 communicates with the memory 320 via the bus 330. When the machine-readable instructions are executed by the processor 310, the steps of the control method of the virtual object in any of the above embodiments can be executed, which are specifically as follows:

[0174] A graphical user interface is provided through a terminal device, and at least a part of the game scene of this virtual game is displayed on the graphical user interface, wherein the game scene includes a first virtual object that is not controlled by a player, and the following processing is performed at each decision point in the process of this virtual game: extracting a first attribute state feature of a first preset output in this virtual game, wherein the first preset output is one of a skill output and a target output; extracting a game state feature of this virtual game; determining a first output action from the first preset output according to the extracted first attribute state feature and the game state feature; forming an action strategy for the first virtual object according to a second attribute state feature of a second preset output in this virtual game and the attribute state feature of the determined first output action, so as to control the first virtual object to execute the action strategy in the game scene, wherein the second preset output is the other of the skill output and the target output.

[0175] Through the electronic device of the embodiment of the present application, the versatility of game AI is improved and the development and maintenance costs are reduced.

[0176] The present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method for controlling a virtual object in any of the above embodiments can be executed, which are as follows:

[0177] A graphical user interface is provided through a terminal device, and at least a part of the game scene of this virtual game is displayed on the graphical user interface, wherein the game scene includes a first virtual object that is not controlled by a player, and the following processing is performed at each decision point in the process of this virtual game: extracting a first attribute state feature of a first preset output in this virtual game, wherein the first preset output is one of a skill output and a target output; extracting a game state feature of this virtual game; determining a first output action from the first preset output according to the extracted first attribute state feature and the game state feature; forming an action strategy for the first virtual object according to a second attribute state feature of a second preset output in this virtual game and the attribute state feature of the determined first output action, so as to control the first virtual object to execute the action strategy in the game scene, wherein the second preset output is the other of the skill output and the target output.

[0178] The computer-readable storage medium of the embodiment of the present application improves the versatility of game AI and reduces development and maintenance costs.

[0179] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the system and device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. In the several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0180] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0181] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0182] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program codes.

[0183] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for controlling a virtual object, characterized in that: A graphical user interface is provided through a terminal device, wherein the graphical user interface displays at least a portion of a game scene of the current virtual game, wherein the game scene includes a first virtual object that is not controlled by a player, and the method includes: Extracting a first attribute state feature of a first preset output in the current virtual game, where the first preset output is one of a skill output and a target output; Extract the game status features of this virtual game; Determining a first output action from the first preset output according to the extracted first attribute state feature and the game state feature; Based on the second attribute state characteristics of the second preset output in this virtual game and the attribute state characteristics of the determined first output action, an action strategy for the first virtual object is formed to control the first virtual object to execute the action strategy in the game scene, and the second preset output is the other of the skill output and the target output.

2. The method according to claim 1, characterized in that The first preset output includes a plurality of first actions, the first attribute state feature includes at least one attribute feature corresponding to each first action, The first output action is determined in the following manner: For each first action, the game state feature is fused with at least one attribute feature corresponding to the first action to obtain a fused feature corresponding to the first action. Based on the fusion feature corresponding to each first action, a first output action is determined from the multiple first actions.

3. The method according to claim 2, characterized in that The first attribute state feature includes a first preset feature matrix, each row of the first preset feature matrix corresponds to a first action, and each element in each row is used to characterize an attribute feature corresponding to the first action. The game state feature includes a game state vector. The fusion feature corresponding to each first action is obtained in the following way: A dot product is performed on the first preset feature matrix and the game state vector to obtain a first fused feature vector, wherein each element in the first fused feature vector is used to characterize a fused feature of a corresponding first action.

4. The method according to claim 3, characterized in that The first preset output includes skill output, the skill output includes occupational skills and displacement skills, the first preset feature matrix includes a skill feature matrix, Among them, the skill feature matrix is ​​formed in the following way: Extract features of each professional skill in this virtual game to form a professional skill feature matrix. Extract features of each displacement skill in this virtual game to form a displacement skill feature matrix. Perform feature shape alignment processing on the formed occupational skill feature matrix and displacement skill feature matrix. The occupational skill feature matrix and the displacement skill feature matrix after feature shape alignment are spliced ​​to obtain a skill feature matrix.

5. The method according to claim 3, characterized in that: The attribute state feature of the first output action includes each element in the row vector corresponding to the first output action extracted from the first preset feature matrix.

6. The method according to claim 1, characterized in that The steps of extracting the game state features of this virtual game include: Obtaining game state information at a current decision point in the current virtual game, the game state information comprising a plurality of attribute fields for characterizing the game state of the current virtual game, the plurality of attribute fields comprising at least one of the following items: skill state information, radar map information, terrain information, and team state information; Performing feature extraction on the multiple attribute fields respectively to obtain multiple game attribute features; All game attribute features are spliced ​​together to obtain the game state features of this virtual game.

7. The method according to claim 6, characterized in that The multiple attribute fields also include situation information and / or self-category information, The step of combining all game attribute features to obtain the game state features of this virtual game includes: Input the game attribute features corresponding to the skill status information, radar map information, terrain information, and team status information into at least one fully connected layer to obtain attribute combination features. The game attribute features corresponding to the situation information, the game attribute features corresponding to the category information itself, and the attribute combination features are spliced ​​together to obtain the game state features of this virtual game.

8. The method according to claim 6, characterized in that The plurality of attribute fields also include gain status information, Among them, the game attribute characteristics corresponding to the team status information are obtained in the following way: Extracting features from attribute fields used to characterize gain state information to obtain multiple gain state features. The plurality of gain state characteristics are compressed to obtain a gain characteristic, Feature extraction is performed on the attribute fields used to characterize the team status information to obtain the team status features corresponding to each virtual object participating in this virtual game. The gain features are respectively concatenated with the team status features corresponding to each virtual object to form game attribute features corresponding to the team status information.

9. The method according to claim 1, characterized in that: The action strategy for the first virtual object is formed in the following way: Superimposing the game state feature onto the attribute state feature of the first output action to obtain the superimposed feature of the first output action; The second attribute state feature of the second preset output is fused with the superimposed feature of the first output action to form an action strategy for the first virtual object.

10. The method according to claim 9, characterized in that The second preset output includes a plurality of second actions, and the second attribute state characteristics include at least one attribute characteristic corresponding to each second action. The action strategy for the first virtual object is formed in the following manner: For each second action, the attribute feature corresponding to the second action is fused with the superposition feature of the first output action to obtain the fused feature corresponding to the second action. Based on the fusion features corresponding to each second action, a second output action is determined from the plurality of second actions, The action strategy is formed based on the superposition feature of the first output action and the attribute state feature of the second output action.

11. The method according to claim 10, characterized in that The second attribute state feature includes a second preset feature matrix, each row of the second preset feature matrix corresponds to a second action, each element in each row is used to characterize an attribute feature corresponding to the second action, and the superposition feature of the first output action includes an action superposition vector. The fusion feature corresponding to each second action is obtained in the following way: A dot product is performed on the second preset feature matrix and the action superposition vector to obtain a second fused feature vector, wherein each element in the second fused feature vector is used to represent a fused feature of a corresponding second action.

12. The method according to claim 1, characterized in that Also includes: After executing the action strategy, the reward strategy is executed. The reward strategy includes individual rewards and / or team rewards, and the individual rewards include positive impact rewards for characterizing the feedback results of the action strategy and negative impact rewards for settlement at decision points.

13. The method according to claim 1, characterized in that The skill output includes a professional skill, and the professional skill is expressed using at least one of the following skill description representations: skill positioning, skill release shape, skill size for describing the skill release shape, skill release range, skill release target, skill cooldown time, and skill segment number. And / or, the target output is used to indicate the target to which the skill output corresponds, and the target output includes at least one of the following objects: the first virtual object itself, a second virtual object belonging to the same team as the first virtual object, and a third virtual object belonging to the enemy team as the first virtual object.

14. The method according to claim 13, characterized in that The skill output also includes a displacement skill, and the displacement skill includes a displacement skill type and a flag word, and the flag word is used to indicate the available status of the displacement skill at the current decision point.

15. A control device for a virtual object, characterized in that: A graphical user interface is provided through a terminal device, wherein the graphical user interface displays at least a portion of a game scene of the current virtual game, wherein the game scene includes a first virtual object that is not controlled by a player, and the apparatus includes: An attribute feature extraction module is used to extract a first attribute state feature of a first preset output in the current virtual game, where the first preset output is one of a skill output and a target output; A game feature extraction module extracts game status features of this virtual game; an action output module, determining a first output action from the first preset output according to the extracted first attribute state feature and game state feature; The strategy execution module forms an action strategy for the first virtual object based on the second attribute state characteristics of the second preset output in this virtual game and the attribute state characteristics of the determined first output action, so as to control the first virtual object to execute the action strategy in the game scene, wherein the second preset output is the other of the skill output and the target output.

16. An electronic device, characterized in that: include: A processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the storage medium communicate via the bus, and the processor executes the machine-readable instructions to perform the steps of any method as claimed in claims 1 to 14.

17. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 14 are executed.

Images