Virtual object control method and device, electronic device, and storage medium

Abstract

The application provides a virtual object control method and device, electronic equipment and a storage medium. A graphical user interface is provided by a terminal device, and a game scene of at least part of a current virtual game is displayed on the graphical user interface. The game scene includes a first virtual object controlled by a non-player. The method includes: extracting a first attribute state feature of a first preset output in the current virtual game; extracting a game state feature of the current virtual game; determining a first output action from the first preset output according to the first attribute state feature and the game state feature; and forming an action strategy for the first virtual object according to a second attribute state feature of a second preset output in the current virtual game and an attribute state feature of the first output action, so as to control the first virtual object to execute the action strategy in the game scene. The application helps to improve the correlation between skill output and target output.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer application, and in particular, to a virtual object control method and device, electronic equipment and storage medium. BACKGROUND

[0002] Artificial intelligence (AI) is the use of digital computers or digital computer-controlled machines to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology of computer science, which aims to understand the essence of intelligence and produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that machines have the functions of perception, reasoning and decision-making.

[0003] With the research and progress of artificial intelligence technology, artificial intelligence technology is researched and applied in many fields. For example, at present, artificial intelligence is used in many scenarios in game development and application, which is called game AI. For example, in the game development process, game AI can replace the role of test personnel, and by introducing game AI to participate in the game to obtain test data, the game performance test can be realized. For another example, in the game application process, game players can actively introduce game AI to play with real people when the number of game players is not enough.

[0004] Game AI is an important part of competitive games. Current game AI solutions mainly fall into two categories: non-learning and learning. The non-learning solution uses finite state machines (FSM), behavior trees (BT), etc. to design and implement the behavior logic of game AI, and selects appropriate behavior nodes for execution according to the game environment and character state. Although this solution is easy to implement, it also has many problems, such as the behavior tree method requiring strong prior knowledge to preset a large number of behavior nodes and conditions to meet the scene requirements, which makes the strategy lack diversity and flexibility, has low battle strength, and poor anthropomorphism, and is easy to be predicted and cracked by other players.

[0005] The learning solution generally uses reinforcement learning methods to train and optimize the strategy of game AI, and through interaction with the environment to obtain reward signals, thereby improving the battle capability of game AI. Although the reinforcement learning method can improve the diversity, battle strength and anthropomorphism of the strategy, the existing design method lacks general expressiveness, and needs to be developed separately for different game play and game characters, which makes the development and maintenance cost high, and difficult to adapt to the increase and update of game content. SUMMARY

[0006] In view of this, embodiments of the present application provide at least one method, device, electronic device, and storage medium for controlling a virtual object to overcome at least one of the above-mentioned defects.

[0007] In a first aspect, an exemplary embodiment of the present application provides a method for controlling a virtual object, providing a graphical user interface through a terminal device, displaying at least a portion of a game scene of a current virtual game, wherein the game scene includes a first virtual object that is not controlled by a player, and performing the following processing at each decision point in the process of the current virtual game: extracting a first attribute state feature of a first preset output in the current virtual game, wherein the first preset output is one of a skill output and a target output; extracting a game state feature of the current virtual game; determining a first output action from the first preset output based on the extracted first attribute state feature and the game state feature; forming an action strategy for the first virtual object based on a second attribute state feature of a second preset output in the current virtual game and the attribute state feature of the determined first output action, to control the first virtual object to execute the action strategy in the game scene, wherein the second preset output is the other of the skill output and the target output.

[0008] In a second aspect, an embodiment of the present application further provides a device for controlling a virtual object, which provides a graphical user interface through a terminal device, and displays at least a portion of a game scene of this virtual game, wherein the game scene includes a first virtual object that is not controlled by a player, and the device includes: an attribute feature extraction module, which extracts a first attribute state feature of a first preset output in this virtual game at each decision point in the process of this virtual game, wherein the first preset output is one of a skill output and a target output; a game feature extraction module, which extracts a game state feature of this virtual game; an action output module, which determines a first output action from the first preset output based on the extracted first attribute state feature and the game state feature; and a strategy execution module, which forms an action strategy for the first virtual object based on the second attribute state feature of the second preset output in this virtual game and the attribute state feature of the determined first output action, so as to control the first virtual object to execute the action strategy in the game scene, wherein the second preset output is the other of the skill output and the target output.

[0009] In a third aspect, an embodiment of the present application further provides an electronic device, a processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor. When the electronic device is running, the processor and the storage medium communicate through the bus, and the processor executes the machine-readable instructions to perform the steps of the above-mentioned virtual object control method.

[0010] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned virtual object control method are executed.

[0011] The control method, device, electronic device, and storage medium for virtual objects provided in the embodiments of the present application achieve decoupling from the specific logic of the game, improve the versatility of game AI, and facilitate porting and use between different games and different skills.

[0012] In addition, in the embodiment of the present application, the output actions for virtual objects are divided into two links: skill output and target output. By using one of the output actions as the input of another output action, the output action can be adjusted and optimized according to the previous output action, thereby increasing the correlation between the skill output and the target output, and helping to reduce the occurrence of mismatching of output skills and targets.

[0013] In addition, the control method, device, electronic device and storage medium of the virtual object according to the embodiments of the present application can also improve the decision-making ability of the game AI, so that the game AI has a higher anthropomorphic effect and intelligence level.

[0014] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.

[0016] Figure 1 A flowchart showing a method for controlling a virtual object provided by an exemplary embodiment of the present application is shown;

[0017] Figure 2 A schematic diagram showing the skill release shape and skill size provided by an exemplary embodiment of the present application;

[0018] Figure 3 A flowchart showing the steps of obtaining game status characteristics of the current virtual game provided by an exemplary embodiment of the present application;

[0019] Figure 4 A flowchart showing steps of forming an action strategy for a first virtual object provided by an exemplary embodiment of the present application;

[0020] Figure 5 Fig. 1 shows a schematic diagram of a first processing scheme for forming an action policy for a first virtual object according to an example embodiment of the present application;

[0021] Figure 6 Fig. 2 shows a schematic diagram of a network structure corresponding to the first processing scheme according to an example embodiment of the present application;

[0022] Figure 7 Fig. 3 shows a schematic diagram of a second processing scheme for forming an action policy for a first virtual object according to an example embodiment of the present application;

[0023] Figure 8 Fig. 4 shows a schematic diagram of a control device for a virtual object according to an example embodiment of the present application;

[0024] Figure 9 Fig. 5 shows a schematic diagram of an electronic device according to an example embodiment of the present application. DETAILED DESCRIPTION

[0025] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. It should be understood that the drawings in the present application only aim to illustrate and describe the present application, and are not used to limit the protection scope of the present application. In addition, it should be understood that the schematic drawings are not drawn according to the actual proportions. The flowcharts in the present application show the operations implemented according to some embodiments of the present application. It should be understood that the operations of the flowcharts can not be implemented in sequence, and the steps without logical context relationship can be reversed in sequence or implemented simultaneously. In addition, one or more other operations can be added to the flowcharts or one or more operations can be removed from the flowcharts under the guidance of the content of the present application.

[0026] The terms “one”, “an”, “the” and “said” are used in the present specification to denote the presence of one or more elements / components / etc.; the terms “include” and “have” are used in the present specification to denote an open-ended inclusion in such a way that additional elements / components / etc. can be present in addition to the listed elements / components / etc.; the terms “first” and “second” are used only as labels, and are not intended to limit the number of objects.

[0027] It should be understood that in the embodiments of the present application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the previous and subsequent associated objects are in an "or" relationship. "Including A, B and / or C" means including any one, any two, or any three of A, B, and C.

[0028] It should be understood that in the embodiments of the present application, "B corresponding to A," "B corresponding to A," "A corresponds to B," or "B corresponds to A" means that B is associated with A and B can be determined based on A. Determining B based on A does not mean determining B based solely on A; B can also be determined based on A and / or other information.

[0029] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present application.

[0030] During game development, game AI can replace human testers, allowing them to participate in the game and collect test data to test game performance. For example, during game development, if there aren't enough players, players can proactively introduce game AI to play alongside real players.

[0031] Game AI is a crucial component of competitive gaming. Traditional game AI primarily utilizes rule-based approaches, such as finite state machines and behavior trees. However, the resulting behavioral strategies are highly specialized, exhibit low combat intensity, lack anthropomorphism, and are easily identified by other players. In recent years, reinforcement learning methods have empowered AI with powerful decision-making capabilities, enabling it to demonstrate impressive performance in gaming. However, existing reinforcement learning approaches lack universal expressive power. For example, diverse gameplay and class types require the construction of unique AI combat models for each. This means that different gameplay styles require independent modeling schemes, and newly added classes require retraining the game AI. This consumes significant computing resources, is time-consuming and labor-intensive, and increases development and maintenance costs.

[0032] In response to at least one of the above-mentioned problems, the present application proposes a method, device, electronic device and storage medium for controlling a virtual object to provide a general combat model solution for game AI.

[0033] First, the names involved in the embodiments of this application are introduced.

[0034] Terminal equipment:

[0035] The terminal device involved in the embodiments of the present application mainly refers to an intelligent device used to provide a game screen (such as a relevant setting / configuration interface in the game, an interface for presenting a game scene) and capable of controlling a virtual character. The terminal device may include, but is not limited to, any one of the following devices: a smart phone, a tablet computer, a portable computer, a desktop computer, a game console, a personal digital assistant (PDA), an e-book reader, an MP4 (Moving Picture Experts Group Audio Layer IV, Moving Picture Experts Group Audio Layer 4) player, etc. An application that supports game scenes is installed and running in the terminal device, such as an application that supports three-dimensional game scenes. The application may include, but is not limited to, any one of a virtual reality application, a three-dimensional map program, a military simulation program, a MOBA game, a multiplayer gun battle survival game, and a third-person shooter game (TPS, Third-Personal Shooting Game). Optionally, the application may be a stand-alone application, such as a stand-alone 3D game program, or a network-connected application.

[0036] Graphical User Interface:

[0037] It is an interface display format for communication between people and computers, allowing users to manipulate icons, logos or menu options on the screen using input devices such as a mouse, keyboard and / or game controller, and also allowing users to manipulate icons or menu options on the screen by performing touch operations on the touch screen of a touch terminal to select commands, start programs or perform other tasks. Exemplarily, the graphical user interface may include virtual objects such as game characters, NPC characters, and AI characters that execute game logic in the game scene. The interface includes any visible visual controls or elements, for example, game controls (such as skill controls, movement controls, function controls, etc.), indicator signs (such as direction indicator signs, character indicator signs, etc.), information display areas (such as number of people defeated, game time, etc.), or game setting controls (such as system settings, stores, gold coins, etc.), and may also include controls such as pictures, input boxes, and text boxes, some of which respond to user operations.

[0038] Game scene:

[0039] It is a virtual environment displayed (or provided) when the application is running on a terminal device or server. Optionally, the game scene is a simulation of the real world, or a semi-simulation and semi-fictitious virtual environment, or a purely fictitious virtual environment. The game scene can be any of a two-dimensional virtual environment, a 2.5-dimensional virtual environment, and a three-dimensional virtual environment. The virtual environment can be the sky, land, ocean, etc. Among them, the game scene is a scene in which the user controls the complete game logic of the virtual character. Optionally, the game scene is also used for virtual environment battles between at least two virtual characters, and the game scene has virtual resources that can be used by at least two virtual characters.

[0040] Virtual Objects:

[0041] Refers to a virtual object in a game scene, which may be a virtual character controlled by a player, including but not limited to at least one of a virtual person, a virtual animal, and an anime character, and may also be a virtual object (NPC) controlled by a non-player. Optionally, when the virtual environment is a three-dimensional virtual environment, the virtual object may be a three-dimensional virtual model, each virtual object having its own shape and volume in the three-dimensional virtual environment, and occupying a portion of the space in the three-dimensional virtual environment. Optionally, the virtual object is a three-dimensional character constructed based on three-dimensional human skeleton technology, and the virtual object achieves different external images by wearing different skins. In some implementations, the virtual object may also be implemented using a 2.5-dimensional or 2-dimensional model, which is not limited in the embodiments of the present application.

[0042] There may be multiple virtual objects in a virtual scene, which are virtual characters controlled by players (i.e., characters controlled by players through input devices) and / or artificial intelligence (AI) trained to compete in a game scene. Optionally, the virtual objects are virtual characters competing in the game scene. Optionally, the number of virtual characters in the game scene is preset or dynamically determined based on the number of terminal devices participating in the virtual game, which is not limited in the embodiments of the present application.

[0043] In an optional embodiment, the terminal device may be a local terminal device. Taking a game as an example, the local terminal device stores the game program and is used to present the game screen. The local terminal device is used to interact with the player through a graphical user interface (GUI), i.e., conventionally downloading and installing the game program through an electronic device and running it. The local terminal device may provide the GUI to the player in a variety of ways, such as rendering it on the terminal device's display screen or providing it to the player through holographic projection. For example, the local terminal device may include a display screen and a processor, the display screen being used to present the GUI, which includes the game screen, and the processor being used to run the game, generate the GUI, and control the display of the GUI on the display screen.

[0044] The application scenarios to which this application is applicable are introduced. This application can be applied to the field of game technology, in which multiple players participating in the game join the same virtual game.

[0045] Before entering a virtual game, players can select different character attributes, such as identity attributes, for their virtual objects in the virtual game. By assigning different character attributes to determine different camps, players can win the game by completing tasks assigned by the game during different stages of the virtual game. For example, multiple virtual objects with character attribute A can "eliminate" virtual objects with character attribute B during the battle stage to win the game. Alternatively, character attributes can be randomly assigned to each virtual object participating in the virtual game upon entering the virtual game.

[0046] An implementation environment provided in one embodiment of the present application may include: a first terminal device, a server, and a second terminal device. The first terminal device and the second terminal device each communicate with the server to implement data communication. In this embodiment, the first terminal device and the second terminal device are each installed with an application program that executes the virtual character control method provided in the present application, and the server is a server-side that executes the virtual character control method provided in the present application. Through the application program, the first terminal device and the second terminal device can each communicate with the server.

[0047] Taking the first terminal device as an example, the first terminal device establishes communication with the server by running an application. In an optional embodiment, the server establishes a virtual game based on the game request of the application. The parameters of the virtual game can be determined based on the parameters in the received game request. For example, the parameters of the virtual game may include the number of players participating in the virtual game, the levels of the characters participating in the virtual game, etc. When the first terminal device receives a response from the game server, the game scene corresponding to the virtual game is displayed through the graphical user interface of the first terminal device. The first terminal device is a device controlled by the first user, and the virtual object displayed in the graphical user interface of the first terminal device is the player character controlled by the first user. The first user inputs operation instructions through the graphical user interface to control the virtual object to perform corresponding operations in the game scene.

[0048] Taking the second terminal device as an example, the second terminal device establishes communication with the server by running an application. In an optional embodiment, the server establishes a virtual game based on the game request of the application. The parameters of the virtual game can be determined based on the parameters in the received game request. For example, the parameters of the virtual game may include the number of participants in the virtual game, the level of the characters participating in the virtual game, etc. When the second terminal device receives a response from the server, the game scene corresponding to the virtual game is displayed through the graphical user interface of the second terminal device. The second terminal device is a device controlled by a second user. The virtual object displayed in the graphical user interface of the second terminal device is the player character controlled by the second user. The second user inputs operation instructions through the graphical user interface to control the virtual object to perform corresponding operations in the virtual scene.

[0049] The server performs data calculation based on the game data reported by the first terminal device and the second terminal device, and synchronizes the calculated game data to the first terminal device and the second terminal device, so that the first terminal device and the second terminal device control the graphical user interface to render the corresponding game scene and / or virtual object according to the synchronization data sent by the game server.

[0050] In this embodiment, the virtual object controlled by the first terminal device and the virtual object controlled by the second terminal device are virtual objects in the same virtual game. The virtual object controlled by the first terminal device and the virtual object controlled by the second terminal device may have the same character attributes or different character attributes. The virtual object controlled by the first terminal device and the virtual object controlled by the second terminal device may belong to the same camp or different camps with hostile relations. The game AI virtual object participating in the virtual game may have a camp to which it belongs, for example, it may belong to the camp to which the virtual object controlled by the first terminal device belongs, or the camp to which the virtual object controlled by the second terminal device belongs. The game AI virtual object may also not belong to a camp and may interact with other virtual objects in the virtual game.

[0051] It should be noted that a virtual game may include two or more virtual objects, and different virtual objects may correspond to different terminal devices. That is to say, in a virtual game, there are more than two terminal devices that send and synchronize game data with the game server respectively.

[0052] The control method of the virtual character provided in the embodiments of the present application can be applied to any of virtual reality applications, three-dimensional map programs, military simulation programs, multiplayer online tactical arena games (MOBAs), multiplayer gun battle survival games, third-person combat games, and first-person combat games.

[0053] In one embodiment of the present application, the control method for a virtual character can be run on a local terminal device or a server. When the control method is run on a server, the method can be implemented and executed based on a cloud interaction system, wherein the cloud interaction system includes a server and a client device.

[0054] In an optional embodiment, various cloud applications, such as cloud games, can be run under the cloud interaction system. Taking cloud games as an example, cloud games refer to a gaming method based on cloud computing. In the cloud gaming operation mode, the operating body of the game program and the main body of the game screen presentation are separated. The storage and operation of the control method are completed on the cloud gaming server. The role of the client device is to receive and send data and present the game screen. For example, the client device can be a display device with data transmission function close to the user side, such as a mobile terminal, TV, computer, PDA, etc.; but the cloud gaming server in the cloud is responsible for information processing. When playing the game, the player operates the client device to send operation instructions to the cloud gaming server. The cloud gaming server runs the game according to the operation instructions, encodes and compresses the game screen and other data, and returns it to the client device through the network. Finally, the client device decodes and outputs the game screen.

[0055] In an optional embodiment, taking a game as an example, a local terminal device stores a game program and is used to present the game screen. The local terminal device is used to interact with the player through a graphical user interface, that is, conventionally downloading and installing the game program through an electronic device and running it. The local terminal device can provide the graphical user interface to the player in a variety of ways, for example, it can be rendered and displayed on the terminal's display screen, or provided to the player through holographic projection. For example, the local terminal device may include a display screen and a processor, the display screen is used to present the graphical user interface, the graphical user interface includes the game screen, and the processor is used to run the game, generate the graphical user interface, and control the display of the graphical user interface on the display screen.

[0056] In a possible implementation, an embodiment of the present invention provides a method for controlling a virtual character, providing a graphical user interface through a terminal device, wherein the terminal device can be the local terminal device mentioned above, or a client device in the cloud interaction system mentioned above.

[0057] To facilitate understanding of the present application, the virtual character control method, device, electronic device and storage medium provided in the embodiments of the present application are described in detail below.

[0058] See also Figure 1 , which is a flowchart of a method for controlling a virtual object provided in an exemplary embodiment of the present application, is generally applied to a game server, such as the cloud game server described above, but the present application is not limited thereto.

[0059] A graphical user interface is provided through the terminal device, and the graphical user interface displays at least a portion of the game scene of the current virtual game, and the game scene includes a first virtual object that is not controlled by the player. For example, the game can be executed at a preset decision point in the process of the current virtual game. Figure 1 The processing steps shown.

[0060] In an embodiment of the present application, the first virtual object is an artificial intelligence character controlled by a non-player participating in this virtual game. The first virtual object can be configured to move in the game scene according to an action strategy, for example, controlling the first virtual object to run, jump, crawl, etc., and can also control the first virtual object to use the virtual skills, virtual props, etc. provided by this virtual game to interact with other virtual objects in the game.

[0061] For example, multiple decision points will appear in sequence during the course of this virtual game, and each decision point in the course of this virtual game can be executed. Figure 1For example, every time a decision point is triggered, the virtual game can automatically execute the control method of the virtual object of this application to reduce the intervention of the game player. The decision point can be pre-embedded in the virtual game, and when the game process meets the triggering conditions of the decision point, a decision is triggered. Alternatively, the decision point can also be automatically triggered at a pre-set time interval during the game process. In addition, it can also be executed only at some decision points. Figure 1 Processing mechanism, other decision points execute other mechanisms, this application does not limit this.

[0062] like Figure 1 As shown, the control method of the virtual object of the exemplary embodiment of the present application specifically includes:

[0063] Step S101: extracting a first attribute state feature of a first preset output in this virtual game.

[0064] In an embodiment of the present application, the action space for the virtual object is divided into two links: skill output and target output, and the first preset output is one of the skill output and the target output.

[0065] Here, skill outputs may include, but are not limited to, professional skills. In this embodiment, for less universal professional skills, a general skill description representation is used to express them. For example, professional skills can be expressed using at least one of the following skill description representations: skill positioning, skill release shape, skill size used to describe the skill release shape, skill release range, skill release target, skill cooldown time, and skill segment number.

[0066] Refer to Table 1 below which shows the classification of skill positioning. For example, the skill positioning in the embodiment of the present application includes 22 types, which can cover all skills of all occupations. Each skill can be composed of a combination of two positionings in the table.

[0067] Table 1

[0068]

[0069] For example, the skill release shape may include but is not limited to the following six categories: self, rectangle, circle, fan, single body and trapezoid, each skill release shape corresponds to the size of the skill, such as Figure 2 shown.

[0070] The skill's target is used to identify the target of a skill, specifically the type of target the skill is targeting. This includes, but is not limited to, at least one of the following: self, teammates, self and teammates, enemies, or all. The skill's range indicates the maximum distance a skill can be cast, and can be represented by a single continuous scalar value. In addition to the general skill attributes described above, class skills may also include a cooldown, usability, and skill stage count.

[0071] Here, in addition to professional skills, skill output may also include displacement skills. For example, displacement skills may include but are not limited to displacement skill types and flag words, and the flag words are used to indicate the available status of the displacement skills at the current decision point, that is, whether the displacement skills are available is indicated by assigning the value of the flag word.

[0072] The target output is used to indicate the target to which the skill output corresponds. The target output includes at least one of the following objects: the first virtual object itself, the second virtual object belonging to the same team as the first virtual object, and the third virtual object belonging to the enemy team as the first virtual object.

[0073] In the embodiment of the present application, skill description representation is used to express occupation-related skills, so that the game AI can fully understand and abstract the skill characteristics it carries.

[0074] Step S102: extracting game state features of this virtual game.

[0075] Here, the game state features include various features that can be used to characterize the game state of the virtual game, as shown below. Figure 3 Let's introduce the process of obtaining game state characteristics.

[0076] Figure 3 A flowchart showing the steps of obtaining the game status characteristics of the current virtual game provided by an exemplary embodiment of the present application.

[0077] Reference Figure 3 In step S1021, the game status information at the current decision point in this virtual game is obtained.

[0078] Here, when the decision point is triggered, the game status information of this virtual game at the moment of triggering can be obtained. For example, the game status information may include but is not limited to multiple attribute fields used to characterize the game status of this virtual game, and the multiple attribute fields include but are not limited to at least one of the following items: skill status information, radar map information, terrain information, and team status information.

[0079] For example, skill state information includes information representing various professional skills and displacement skills included in the current virtual game. Terrain information may include terrain information within a preset range of the first virtual object's position in the game scene at the current decision point. Exemplarily, this preset range may be the range corresponding to the game screen presented on the graphical user interface, or the maximum skill release range corresponding to the first virtual object. This terrain information represents the terrain used to construct the game scene, such as raised peaks, sunken canyons, hillsides, and walls of different materials.

[0080] The radar chart information includes a radar game map, which may include, but is not limited to, at least one of the following items: the position of the first virtual object and its corresponding perspective, the first virtual object, the first virtual object's teammates, and the first virtual object's enemy NPCs. For example, the radar game map may also display the positional relationship between the first virtual object and its teammates and / or enemies. Team status information includes the status of all virtual objects participating in this virtual game, such as health value, position, speed value, damage value, etc.

[0081] In step S1022, feature extraction is performed on the multiple attribute fields respectively to obtain multiple game attribute features.

[0082] Here, feature extraction can be performed on multiple attribute fields of the skill status information to obtain multiple game attribute features corresponding to the skill information. Similarly, radar map information, terrain information, and team status information can be extracted separately to obtain game attribute features corresponding to each information.

[0083] It should be understood that various existing feature extraction methods can be used to perform feature extraction, and this application does not limit the content of this part.

[0084] In step S1023, all game attribute features are spliced ​​together to obtain the game state features of this virtual game.

[0085] In a preferred embodiment, the game state features may comprise a game state vector, where each element of the game state vector corresponds to a pair of game attribute features. In other words, feature concatenation may involve arranging the game attribute features in a predetermined order to form the game state vector.

[0086] Preferably, the above-mentioned multiple attribute fields may also include situation information and / or self-category information. For example, the situation information is used to indicate whether the first virtual object can win at the current moment. For example, when the situation information is configured as a value of 1, it means: the current number of survivors on the own side is greater than the number of survivors on the enemy side, or the current number of survivors on the own side is equal to the number of survivors on the enemy side and the total damage of the own side is greater than the total damage of the enemy side. The value of 0 indicates the opposite. The self-category information can be used to indicate the game level of the virtual object or the type of the virtual object (such as attack type or auxiliary type).

[0087] In this case, the game state features of this virtual game can be obtained in the following manner: the game attribute features corresponding to the skill state information, radar map information, terrain information, and team state information are input into at least one fully connected layer to obtain attribute combination features, and the game attribute features corresponding to the situation information, the game attribute features corresponding to the own category information, and the attribute combination features are spliced ​​to obtain the game state features of this virtual game. This helps to improve the flexibility of obtaining game state features, that is, when transplanting and using between different games and different skills, it can be flexibly determined whether to introduce own category information and / or situation information. In addition, the above feature processing method also helps to improve the winning rate of virtual objects.

[0088] In this embodiment, situational information is introduced to guide the game AI in adopting different strategies, such as aggressive or conservative, by indicating whether the team can win at the current moment. This helps improve the AI's combat effectiveness. Testing has shown that the win rate with and without situational information is 59.3% vs. 40.7%, maintaining a win rate above 99% compared to existing behavior tree solutions.

[0089] By extracting the above-mentioned attribute state features and game state features, the versatility of game AI can be improved and the development time and maintenance cost of game AI can be reduced.

[0090] return Figure 1 , step S103: determining a first output action from a first preset output according to the extracted first attribute state feature and the game state feature.

[0091] In an embodiment of the present application, the first preset output may include multiple first actions. In this case, the first attribute state feature may include at least one attribute feature corresponding to each first action, and the first output action is one determined from the multiple first actions.

[0092] Exemplarily, the first output action can be determined in the following manner: for each first action, the game state feature is fused with at least one attribute feature corresponding to the first action to obtain the fusion feature corresponding to the first action; based on the fusion feature corresponding to each first action, the first output action is determined from multiple first actions.

[0093] In a preferred embodiment, the first attribute state feature can be represented in matrix form. For example, the first attribute state feature can include a first preset feature matrix, each row of the first preset feature matrix corresponds to a first action, and each element in each row is used to represent an attribute feature corresponding to the first action.

[0094] In this case, the fused feature corresponding to each first action can be obtained by performing a dot product between the first preset feature matrix and the game state vector to obtain a first fused feature vector, where each element in the first fused feature vector represents the fused feature of the corresponding first action. Here, the fused feature corresponding to each first action represents the probability of that action being selected.

[0095] Here, the game state vector can be an M-dimensional column vector. Each row in the first preset feature matrix is ​​dot-producted with the game state vector to obtain the fused features of the first action corresponding to the row. The fused features of each first action form an M-dimensional first fused feature column vector. It should be understood that in order to perform a dot product operation on the matrix and vector, it is necessary to ensure that the number of columns in the matrix is ​​consistent with the dimension of the vector. Exemplarily, various methods can be used to make the two computable, such as through processing of a fully connected layer.

[0096] In a preferred embodiment, the first action can be screened by introducing a first available action mask, which can refer to an available mask related to the first preset output, that is, the available mask is used to characterize the first action that can be used by the first virtual object at the current decision point. Based on this, the first available action mask is used to screen out the available first action from multiple first actions. If the available first action is not unique, the screening can be continued from the available first actions based on the fusion feature. For example, the first action with the largest value of the fusion feature is determined as the first output action.

[0097] Step S104: Based on the second attribute state characteristics of the second preset output in the current virtual game and the attribute state characteristics of the determined first output action, an action strategy for the first virtual object is formed to control the first virtual object to execute the action strategy in the game scene. Here, the second preset output is the other of the skill output and the target output.

[0098] In an embodiment of the present application, the output action of the first virtual object is divided into two links: skill output and target output. In order to increase the correlation between skill output and target output, the present application adopts an autoregressive operation, that is, one of the output actions is used as the input of another output action, so that the output action can be adjusted and optimized according to the previous output action.

[0099] In addition, based on the correlation between the two actions, it is also possible to reduce the mismatching of output skills and targets. Tests have shown that the mismatching rate is reduced by 62.5%. By introducing a skill-related target output legal matrix (such as the flag words in the displacement skill), the mismatching situation can be completely eliminated, thereby improving the training efficiency of the game.

[0100] Refer to the following Figure 4 Let's introduce the process of forming an action strategy for the first virtual object.

[0101] Figure 4 A flowchart illustrating steps of forming an action strategy for a first virtual object provided by an exemplary embodiment of the present application is shown.

[0102] Reference Figure 4 , in step S1041, the game state feature is superimposed on the attribute state feature of the first output action to obtain the superimposed feature of the first output action;

[0103] In the case where the first attribute state feature includes the first preset feature matrix, the attribute state feature of the first output action includes each element in the row vector corresponding to the first output action extracted from the first preset feature matrix.

[0104] When the game state feature is a game state vector, the superposition feature of the first output action can be obtained by vector multiplication. For example, each element in the vector corresponding to the first output action can be multiplied with each element in the game state vector bit by bit to obtain the superposition feature of the first output action.

[0105] In step S1042 , the second attribute state feature of the second preset output is fused with the superimposed feature of the first output action to form an action strategy for the first virtual object.

[0106] In the embodiment of the present application, the second preset output may include multiple second actions, and the second attribute state characteristics include at least one attribute characteristic corresponding to each second action.

[0107] Exemplarily, an action strategy for the first virtual object can be formed in the following manner: for each second action, the attribute characteristics corresponding to the second action are fused with the superimposed characteristics of the first output action to obtain the fusion characteristics corresponding to the second action; based on the fusion characteristics corresponding to each second action, the second output action is determined from multiple second actions; based on the superimposed characteristics of the first output action and the attribute state characteristics of the second output action, an action strategy for the first virtual object is formed.

[0108] In a preferred embodiment, the superimposed feature of the first output action is an action superimposed vector, which can represent the second attribute state feature in matrix form. For example, the second attribute state feature can include a second preset feature matrix, each row of the second preset feature matrix corresponds to a second action, and each element in each row is used to represent an attribute feature corresponding to the second action.

[0109] In this case, the fusion feature corresponding to each second action is obtained in the following manner: dot product is performed on the second preset feature matrix and the action superposition vector to obtain a second fusion feature vector, and each element in the second fusion feature vector is used to characterize the fusion feature of a corresponding second action.

[0110] Here, the action superposition vector can be an M-dimensional column vector. A dot product is performed on each row of the second preset feature matrix and the action superposition vector to obtain a fused feature of the second action corresponding to that row. The fused features of each second action form an M-dimensional second fused feature column vector. It should be understood that in order for the matrix and vector to perform a dot product operation, the number of columns in the matrix must be consistent with the dimension of the vector.

[0111] In a preferred embodiment, the second action can be screened by introducing a second available action mask, which can refer to an available mask related to the first preset output, that is, the available mask is used to characterize the available second action associated with the first virtual object and the first preset output at the current decision point. Based on this, the available second action is screened out from multiple second actions through the second available action mask. If the available second action is not unique, the screening can be continued from the available second actions based on the fusion feature. For example, the second action with the largest value of the fusion feature is determined as the second output action.

[0112] The action strategy for the first virtual object includes a combination of a first output action and a second output action, that is, a combination of skill output and target output, wherein the superimposed characteristics of the first output action cover the state characteristics of the action itself and the game state characteristics to capture the correlation and dependency between the action and the game state, and use the first output action as the input of the second output action to increase the correlation between the two actions. Compared with the existing technical solutions, the combat AI of the present application has been greatly improved in combat intensity and versatility.

[0113] In one example, the present application uses an autoregressive neural network to process game state information, and designs two autoregressive structures for two categories of action space (skill output and target output).

[0114] In the first processing scheme, the skill output and skill feature matrix can be obtained based on neural network reasoning, and then the skill feature corresponding to the skill is selected, and after being fused with the feature of the game state information, the target is output.

[0115] The following is combined with the situation where the first preset output is skill output Figure 5 and Figure 6 The processing process of the first solution mentioned above is introduced.

[0116] Figure 5 A schematic diagram illustrating a first processing solution for forming an action strategy for a first virtual object provided by an exemplary embodiment of the present application is shown.

[0117] In this example, the first preset output is a skill output, which includes professional skills and displacement skills, and the first preset feature matrix includes a skill feature matrix.

[0118] like Figure 5 As shown, the skill output information in this virtual game is obtained. The skill output information may include action information of various professional skills and displacement skills in this virtual game. For example, the displacement skills may include but are not limited to at least one of the following items: sprinting, jumping, dodging, and walking.

[0119] Feature extraction is performed on the acquired skill output information to obtain a plurality of skill attribute features, and a skill feature matrix is ​​formed using the acquired plurality of skill attribute features.

[0120] See also Figure 6The specific method for forming the skill feature matrix in the dashed box shown includes: extracting features from each class skill in this virtual game to form a class skill feature matrix. Skill_desc represents multiple class skills, and Scope, CD, etc. include the corresponding skill attributes, such as skill range and cooldown. Feature extraction is performed on each displacement skill in this virtual game to form a displacement skill feature matrix. Action Id represents multiple types of displacement skills, and Legal represents a flag word.

[0121] For example, for displacement skills, the ID of the displacement skill is first embedded and then fused with the remaining features. For professional skills, the general skill representation is used, the skill positioning numbers in Table 1 are embedded, and the skill release shape and skill release target are encoded using a one-hot encoding operation, and then the remaining features are fused.

[0122] The formed professional skill feature matrix and displacement skill feature matrix are subjected to feature shape alignment. For example, the professional skill feature matrix and displacement skill feature matrix can be subjected to FC (Fully Connected Layers) processing respectively to perform feature shape alignment so that the number of columns of the two is consistent.

[0123] The occupational skill feature matrix and the displacement skill feature matrix, after feature shape alignment, are concatenated to obtain a skill feature matrix. For example, the above matrix concatenation may refer to placing each row of the occupational skill feature matrix below the last row of the displacement skill feature matrix, or placing each row of the displacement skill feature matrix below the last row of the occupational skill feature matrix, or arranging the rows of the two matrices in a certain order or randomly.

[0124] In an embodiment of the present application, a deep reinforcement learning algorithm is used to train combat AI, and the state s, action a and reward r are first defined.

[0125] For example, the state S used in this application includes the following eight parts: occupational skill information, displacement skill information, radar map information, terrain information, team status information, buff information, situation information, and self-category information. For occupational skill information with weak universality, this application uses a universal skill representation method to express it. Universal skill representation includes but is not limited to: skill positioning, skill release shape, skill size, skill release target, and skill release range.

[0126] The action a of the present application uses a combination of skill and target, the skill output includes displacement skill and professional skill, and the target includes self, teammates and opponents. When executing the decision, a skill is selected according to the neural network, then a target is selected according to the skill, and finally the combined action is executed.

[0127] The reward function r of the present application adopts a combination of individual rewards and team rewards, the individual rewards include killing opponents, damage, damage receiving and treatment, and the team rewards include the number of team deaths and final victory or defeat. Optionally, in order to promote the game AI to fight actively, the reward function also introduces a time-related penalty, that is, a penalty is executed every time a decision is made. Exemplarily, the above reward function can be used as a judgment index for training the game AI, after each training is completed, the parameters in the autoregressive network are adjusted based on the reward function to change the training strategy, so that the strategy can end the battle faster.

[0128] In a preferred embodiment of the present application, the plurality of attribute fields further include gain state information, the gain state information including state attributes corresponding to all virtual objects participating in the current virtual match.

[0129] In this case, the match attribute feature corresponding to the team state information can be obtained in the following way: feature extraction is performed on the attribute field used to represent the gain state information to obtain a plurality of gain state features, compression processing is performed on the plurality of gain state features to obtain a fused gain feature, feature extraction is performed on the attribute field used to represent the team state information to obtain a team state feature corresponding to each virtual object participating in the current virtual match, and the gain feature is spliced with the team state feature corresponding to each virtual object to form the match attribute feature corresponding to the team state information.

[0130] In Figure 6 In the network structure shown, a deep neural network based on autoregression is designed to process game state information and output actions: including using a Multi-Head Attention structure to process skill information and team state information, thereby capturing the correlation and dependency between skills and team members.

[0131] For buff information, the id corresponding thereto is embedded and coded, and then fused with the remaining features, and then feature compression is performed through two layers of FC and one layer of Maxpooling to obtain a fused gain feature.

[0132] For example, for each virtual object, a fused gain feature can be added after, before, or at other locations to the corresponding team status features. The team status features of each virtual object after the splicing are directly processed using Embedding and Multi-Head Attention.

[0133] For displacement skills, the displacement skill ID is first embedded, then fused with the remaining features, and finally processed by Embedding and Multi-Head Attention.

[0134] For professional skills, a general skill representation is used to perform an embedding operation on the skill location numbers in Table 1. A one-hot encoding operation is performed on the skill release shape and skill release target. The remaining features are then integrated and finally processed through embedding and multi-head attention.

[0135] Two layers of FC are used to process other scalar states (such as radar map information, terrain information, self-category information, and situation information).

[0136] In this example, the game attribute features corresponding to displacement skills, professional skills, radar maps, terrain information, and team status information are input into two layers of FC to obtain attribute combination features. Then, the game attribute features corresponding to the situation information, the game attribute features corresponding to the own category information, and the attribute combination features are concatenated to obtain the game status features of this virtual game.

[0137] The above-mentioned game state features are processed by embedding and then dot-producted with the skill feature matrix. The dot product results are processed by the available skill mask to determine the output skill output. The skill feature matrix is ​​subjected to gather+sigmoid processing to obtain a row of attribute features corresponding to the skill output. The attribute features corresponding to the skill output are then superimposed (× multiplied) with the game state features after embedding processing to obtain the superimposed features of the skill output. The superimposed features of the skill output are dot-producted (dot) with the target feature matrix to obtain the fusion features corresponding to each target. The dot product results are processed by the available target mask to determine the output target output.

[0138] Each row of the target feature matrix corresponds to a target, and each element in each row of the target feature matrix corresponds to an attribute feature of the target. Here, the targets forming the target feature matrix may include, but are not limited to, all virtual objects participating in the virtual game, for example, the first virtual object itself, at least one second virtual object that is a teammate of the first virtual object (which may be a virtual object controlled by another terminal device or an NPC), and at least one third virtual object that is an enemy of the first virtual object (which may be a virtual object controlled by another terminal device or an NPC).

[0139] In one example, a row of attribute features corresponding to a target may be the game attribute features obtained by extracting the team status information as described above, which may include, but are not limited to: the health value, position, speed value, and damage value corresponding to the target.

[0140] In another example, a row of attribute features corresponding to a target can also be the match attribute features obtained by extracting the team state information and gain state information. Exemplarily, multiple gain state features are compressed to obtain a fused gain feature, which is then concatenated with the team state feature corresponding to the target to form the match attribute feature for the target.

[0141] For output actions use Figure 6 The first approach involves an autoregressive network. The network first outputs a skill output, then selects a corresponding skill feature based on the selected skill output. This skill feature is then dot-producted with the game state feature to correlate target selection with the skill output. Before the target output is generated, a skill-related available target mask is introduced to avoid mismatches between skill and target outputs, accelerating policy learning.

[0142] In the second processing scheme, the target output and target feature matrix can be obtained based on the neural network reasoning, and then the target feature corresponding to the target is selected and fused with the feature of the game state information to output the skill (such as Figure 7 shown).

[0143] The difference between the second processing scheme and the first processing scheme is that in the first processing scheme, the skills are output first and then the target is selected, while in the second processing scheme, the target is output first and then the skills are selected. The two schemes are similar in terms of feature extraction, processing, superposition, and fusion methods in the processing process, and this application will not go into details about this part.

[0144] Figure 5 and Figure 7Two different autoregressive structures are demonstrated. The essence of these two structures is to strengthen the connection between outputs, thereby reducing the mismatch between skills and target selection. After testing and verification, the winning rate of the first processing scheme is approximately 60%:40% compared with the second processing scheme.

[0145] In a preferred embodiment of the present application, the aforementioned virtual object control method may further include: executing a reward strategy after executing the action strategy for the first virtual object. Exemplarily, the reward strategy includes individual rewards and / or team rewards, with individual rewards including positive impact rewards representing feedback results of the action strategy and negative impact rewards settled at decision points.

[0146] The control method of the virtual object described above can achieve at least the following effective effects:

[0147] To reduce development time, this application trains combat AI in a martial arts scenario, and subsequently integrates it into the Imperial City 1v1 and intelligent NPC scenarios. Compared with existing reinforcement learning solutions, this effectively shortens the development cycle of integration.

[0148] This solution reduces maintenance costs and is scalable for new occupations. Existing reinforcement learning solutions require retraining for each new occupation, requiring approximately 650,000 rounds. This application only requires 50,000 rounds of fine-tuning based on the existing model, surpassing the existing solution in win rate and increasing efficiency by 13 times.

[0149] High strength and anthropomorphism: Compared with existing behavior tree solutions, this application maintains a win rate of over 99%, and the AI ​​trained through reinforcement learning is closer to humans in combat, capable of dodging skills, coordinating with teammates to focus fire on targets, and escaping when health is low.

[0150] To address the poor universality of current game AI combat models, this solution uses a deep reinforcement learning algorithm. It employs universal skill descriptions to represent less universal professional skill states. It also incorporates embedding technology to design an autoregressive network structure, introducing situational states to enhance AI expressiveness. This allows the combat model to be well compatible with different types of intelligent agents and gameplay, reducing the time and cost of game AI development.

[0151] Based on the same application concept, the embodiments of the present application also provide a control device for a virtual object corresponding to the method provided in the above embodiments. Since the principle of solving the problem by the device in the embodiments of the present application is similar to the control method for a virtual object in the above embodiments of the present application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0152] Figure 8 This is a schematic diagram of the structure of a virtual object control device provided by an exemplary embodiment of the present application. Figure 8As shown, a graphical user interface is provided by a terminal device, and the graphical user interface displays at least a portion of the game scene of the current virtual game. The game scene includes a first virtual object that is not controlled by the player. The control device 200 of the virtual object includes:

[0153] An attribute feature extraction module 210 extracts a first attribute state feature of a first preset output in the current virtual game, where the first preset output is one of a skill output and a target output;

[0154] The game feature extraction module 220 extracts the game state features of the current virtual game;

[0155] The action output module 230 determines a first output action from the first preset output according to the extracted first attribute state feature and game state feature;

[0156] The strategy execution module 240 forms an action strategy for the first virtual object based on the second attribute state characteristics of the second preset output in this virtual game and the attribute state characteristics of the determined first output action, so as to control the first virtual object to execute the action strategy in the game scene, where the second preset output is the other of the skill output and the target output.

[0157] In one possible implementation of the present application, the first preset output includes multiple first actions, and the first attribute state feature includes at least one attribute feature corresponding to each first action, wherein the action output module 230 determines the first output action in the following manner: for each first action, the game state feature is fused with at least one attribute feature corresponding to the first action to obtain the fusion feature corresponding to the first action, and based on the fusion feature corresponding to each first action, the first output action is determined from the multiple first actions.

[0158] In one possible implementation of the present application, the first attribute state feature includes a first preset feature matrix, each row of the first preset feature matrix corresponds to a first action, and each element in each row is used to characterize an attribute feature corresponding to the first action, and the game state feature includes a game state vector, wherein the action output module 230 obtains the fusion feature corresponding to each first action in the following manner: performing a dot product between the first preset feature matrix and the game state vector to obtain a first fusion feature vector, and each element in the first fusion feature vector is used to characterize the fusion feature of a corresponding first action.

[0159] In a possible implementation of the present application, the first preset output includes skill output, and the skill output includes professional skills and displacement skills. The first preset feature matrix includes a skill feature matrix, wherein the attribute feature extraction module 210 forms the skill feature matrix in the following manner: feature extraction is performed on each professional skill in this virtual game to form a professional skill feature matrix, feature extraction is performed on each displacement skill in this virtual game to form a displacement skill feature matrix, feature shape alignment is performed on the formed professional skill feature matrix and displacement skill feature matrix, and the professional skill feature matrix and displacement skill feature matrix after the feature shape alignment are spliced ​​to obtain a skill feature matrix.

[0160] In a possible implementation manner of the present application, the attribute state feature of the first output action includes each element in a row vector corresponding to the first output action extracted from a first preset feature matrix.

[0161] In one possible implementation of the present application, the game feature extraction module 220 is also used to obtain game status information at the current decision point in this virtual game, where the game status information includes multiple attribute fields for characterizing the game status of this virtual game, and the multiple attribute fields include at least one of the following items: skill status information, radar map information, terrain information, and team status information; feature extraction is performed on the multiple attribute fields respectively to obtain multiple game attribute features; all game attribute features are spliced ​​to obtain the game status features of this virtual game.

[0162] In a possible implementation of the present application, the multiple attribute fields further include situation information and / or self-category information.

[0163] Among them, the game feature extraction module 220 is also used to input the game attribute features corresponding to the skill status information, radar map information, terrain information, and team status information into at least one fully connected layer to obtain attribute combination features, and splice the game attribute features corresponding to the situation information, the game attribute features corresponding to the own category information, and the attribute combination features to obtain the game status features of this virtual game.

[0164] In one possible implementation of the present application, the multiple attribute fields also include gain status information, wherein the game feature extraction module 220 obtains the game attribute features corresponding to the team status information in the following manner: performing feature extraction on the attribute field used to characterize the gain status information to obtain multiple gain status features, compressing the multiple gain status features to obtain a one-dimensional gain feature, performing feature extraction on the attribute field used to characterize the team status information to obtain the team status feature corresponding to each virtual object participating in this virtual game, and splicing the gain feature with the team status feature corresponding to each virtual object to form the game attribute feature corresponding to the team status information.

[0165] In one possible implementation of the present application, the strategy execution module 240 forms an action strategy for the first virtual object in the following manner: superimposing the game state characteristics onto the attribute state characteristics of the first output action to obtain the superimposed characteristics of the first output action; fusing the second attribute state characteristics of the second preset output with the superimposed characteristics of the first output action to form an action strategy for the first virtual object.

[0166] In one possible implementation of the present application, the second preset output includes multiple second actions, and the second attribute state characteristics include at least one attribute feature corresponding to each second action, wherein the strategy execution module 240 forms an action strategy for the first virtual object in the following manner: for each second action, the attribute feature corresponding to the second action is fused with the superimposed feature of the first output action to obtain the fusion feature corresponding to the second action; based on the fusion feature corresponding to each second action, the second output action is determined from the multiple second actions; and the action strategy is formed based on the superimposed feature of the first output action and the attribute state characteristics of the second output action.

[0167] In one possible implementation of the present application, the second attribute state feature includes a second preset feature matrix, each row of the second preset feature matrix corresponds to a second action, each element in each row is used to characterize an attribute feature corresponding to the second action, and the superposition feature of the first output action includes an action superposition vector, wherein the strategy execution module 240 obtains the fusion feature corresponding to each second action in the following manner: performing a dot product between the second preset feature matrix and the action superposition vector to obtain a second fusion feature vector, and each element in the second fusion feature vector is used to characterize the fusion feature of a corresponding second action.

[0168] In one possible implementation of the present application, the strategy execution module 240 is also used to execute a reward strategy after executing the action strategy, wherein the reward strategy includes individual rewards and / or team rewards, and the individual rewards include positive impact rewards for characterizing the feedback results of the action strategy and negative impact rewards for settlement at decision points.

[0169] In one possible implementation of the present application, the skill output includes professional skills, and the professional skills are expressed using at least one skill description representation of the following items: skill positioning, skill release shape, skill size used to describe the skill release shape, skill release range, skill release target, skill cooling time, skill segment number, and / or, the target output is used to indicate the application target corresponding to the skill output, and the target output includes one of the following objects: the first virtual object itself, the second virtual object belonging to the same team as the first virtual object, and the third virtual object belonging to the enemy team as the first virtual object.

[0170] In a possible implementation manner of the present application, the skill output further includes a displacement skill, and the displacement skill includes a displacement skill type and a flag word, and the flag word is used to indicate the available status of the displacement skill at the current decision point.

[0171] The virtual object control device of the embodiment of the present application improves the versatility of game AI and reduces development and maintenance costs.

[0172] See also Figure 9 , Figure 9 This is a schematic diagram of the structure of an electronic device provided by an exemplary embodiment of the present application. Figure 9 As shown, the electronic device 300 includes a processor 310 , a memory 320 and a bus 330 .

[0173] The memory 320 stores machine-readable instructions executable by the processor 310. When the electronic device 300 is running, the processor 310 communicates with the memory 320 via the bus 330. When the machine-readable instructions are executed by the processor 310, the steps of the virtual object control method in any of the above embodiments can be performed, which are specifically as follows:

[0174] A graphical user interface is provided through a terminal device, and at least a portion of the game scene of this virtual game is displayed on the graphical user interface, wherein the game scene includes a first virtual object that is not controlled by the player, and the following processing is performed at each decision point in the process of this virtual game: extracting a first attribute state feature of a first preset output in this virtual game, wherein the first preset output is one of a skill output and a target output; extracting a game state feature of this virtual game; determining a first output action from the first preset output based on the extracted first attribute state feature and the game state feature; forming an action strategy for the first virtual object based on a second attribute state feature of a second preset output in this virtual game and the attribute state feature of the determined first output action, so as to control the first virtual object to execute the action strategy in the game scene, wherein the second preset output is the other of the skill output and the target output.

[0175] Through the electronic device of the embodiment of the present application, the versatility of game AI is improved and the development and maintenance costs are reduced.

[0176] The present application also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the method for controlling a virtual object in any of the above embodiments are performed, specifically as follows:

[0177] A graphical user interface is provided through a terminal device, and at least a portion of the game scene of this virtual game is displayed on the graphical user interface, wherein the game scene includes a first virtual object that is not controlled by the player, and the following processing is performed at each decision point in the process of this virtual game: extracting a first attribute state feature of a first preset output in this virtual game, wherein the first preset output is one of a skill output and a target output; extracting a game state feature of this virtual game; determining a first output action from the first preset output based on the extracted first attribute state feature and the game state feature; forming an action strategy for the first virtual object based on a second attribute state feature of a second preset output in this virtual game and the attribute state feature of the determined first output action, so as to control the first virtual object to execute the action strategy in the game scene, wherein the second preset output is the other of the skill output and the target output.

[0178] The computer-readable storage medium of the embodiment of the present application improves the versatility of game AI and reduces development and maintenance costs.

[0179] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the system and device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. In the several embodiments provided in this application, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0180] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0181] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0182] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0183] The above are only specific embodiments of the present application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A method for controlling a virtual object, characterized in that: A graphical user interface is provided through a terminal device, wherein the graphical user interface displays at least a portion of a game scene of the current virtual game, wherein the game scene includes a first virtual object that is not controlled by the player, and the method includes: Extracting a first attribute state feature of a first preset output in the current virtual game, where the first preset output is one of a skill output and a target output; Extract the game status features of this virtual game; Determining a first output action from the first preset output according to the extracted first attribute state feature and the game state feature; Based on the second attribute state characteristics of the second preset output in this virtual game and the attribute state characteristics of the determined first output action, an action strategy for the first virtual object is formed to control the first virtual object to execute the action strategy in the game scene, and the second preset output is the other of the skill output and the target output.

2. The method according to claim 1, characterized in that The first preset output includes a plurality of first actions, and the first attribute state feature includes at least one attribute feature corresponding to each first action. The first output action is determined in the following manner: For each first action, the game state feature is fused with at least one attribute feature corresponding to the first action to obtain a fused feature corresponding to the first action. A first output action is determined from the multiple first actions based on the fusion feature corresponding to each first action.

3. The method according to claim 2, characterized in that The first attribute state feature includes a first preset feature matrix, each row of the first preset feature matrix corresponds to a first action, and each element in each row is used to represent an attribute feature corresponding to the first action. The game state feature includes a game state vector. The fusion feature corresponding to each first action is obtained in the following way: Perform a dot product between the first preset feature matrix and the game state vector to obtain a first fused feature vector, where each element in the first fused feature vector is used to represent a fused feature of a corresponding first action.

4. The method according to claim 3, characterized in that The first preset output includes skill output, the skill output includes occupational skills and displacement skills, the first preset feature matrix includes a skill feature matrix, The skill feature matrix is ​​formed in the following way: Extract features of each professional skill in this virtual game to form a professional skill feature matrix. Extract features of each displacement skill in this virtual game to form a displacement skill feature matrix. Perform feature shape alignment processing on the formed professional skill feature matrix and displacement skill feature matrix. The occupational skill feature matrix and the displacement skill feature matrix after feature shape alignment are spliced ​​to obtain the skill feature matrix.

5. The method according to claim 3, characterized in that The attribute state feature of the first output action includes each element in the row vector corresponding to the first output action extracted from the first preset feature matrix.

6. The method according to claim 1, characterized in that The steps for extracting the game state features of this virtual game include: Obtaining game state information at a current decision point in the current virtual game, the game state information including multiple attribute fields used to represent the game state of the current virtual game, the multiple attribute fields including at least one of the following: skill state information, radar map information, terrain information, and team state information; Performing feature extraction on each of the plurality of attribute fields to obtain a plurality of game attribute features; All game attribute features are spliced ​​together to obtain the game state features of this virtual game.

7. The method according to claim 6, characterized in that The multiple attribute fields also include situation information and / or self-category information, The step of combining all game attribute features to obtain the game state features of this virtual game includes: Input the game attribute features corresponding to skill status information, radar map information, terrain information, and team status information into at least one fully connected layer to obtain attribute combination features. The game attribute features corresponding to the situation information, the game attribute features corresponding to the category information itself, and the attribute combination features are spliced ​​together to obtain the game state features of this virtual game.

8. The method according to claim 6, characterized in that The plurality of attribute fields also include gain status information, The game attribute characteristics corresponding to the team status information are obtained in the following ways: Extract features from the attribute fields used to characterize the gain state information to obtain multiple gain state features. Performing compression processing on the plurality of gain state features to obtain a gain feature, Perform feature extraction on the attribute fields used to represent team status information to obtain the team status features corresponding to each virtual object participating in this virtual game. The gain features are spliced ​​with the team status features corresponding to each virtual object to form the game attribute features corresponding to the team status information.

9. The method according to claim 1, characterized in that The action strategy for the first virtual object is formed in the following way: Superimposing the game state feature onto the attribute state feature of the first output action to obtain a superimposed feature of the first output action; The second attribute state feature of the second preset output is fused with the superimposed feature of the first output action to form an action strategy for the first virtual object.

10. The method according to claim 9, characterized in that The second preset output includes a plurality of second actions, and the second attribute state characteristics include at least one attribute characteristic corresponding to each second action. The action strategy for the first virtual object is formed in the following manner: For each second action, the attribute feature corresponding to the second action is fused with the superimposed feature of the first output action to obtain the fused feature corresponding to the second action. Based on the fusion features corresponding to each second action, a second output action is determined from the plurality of second actions. The action strategy is formed based on the superposition feature of the first output action and the attribute state feature of the second output action.

11. The method according to claim 10, characterized in that The second attribute state feature includes a second preset feature matrix, each row of the second preset feature matrix corresponds to a second action, each element in each row is used to characterize an attribute feature corresponding to the second action, and the superposition feature of the first output action includes an action superposition vector. The fusion feature corresponding to each second action is obtained in the following way: Perform a dot product of the second preset feature matrix and the action superposition vector to obtain a second fused feature vector, where each element in the second fused feature vector is used to represent a fused feature of a corresponding second action.

12. The method according to claim 1, characterized in that Also includes: After executing the action strategy, the reward strategy is executed. The reward strategy includes individual rewards and / or team rewards, and the individual rewards include positive impact rewards for representing the feedback results of the action strategy and negative impact rewards for settlement at decision points.

13. The method according to claim 1, wherein The skill output includes professional skills, and the professional skills are expressed using at least one skill description representation of the following items: skill positioning, skill release shape, skill size used to describe the skill release shape, skill release range, skill release target, skill cooldown time, and skill segment number. And / or, the target output is used to indicate the target to which the skill output corresponds, and the target output includes at least one of the following objects: the first virtual object itself, a second virtual object belonging to the same team as the first virtual object, and a third virtual object belonging to the enemy team as the first virtual object.

14. The method according to claim 13, wherein: The skill output also includes a displacement skill, and the displacement skill includes a displacement skill type and a flag word, and the flag word is used to indicate the available status of the displacement skill at the current decision point.

15. A control device for a virtual object, characterized in that: A graphical user interface is provided through a terminal device, wherein the graphical user interface displays at least a portion of a game scene of the current virtual game, wherein the game scene includes a first virtual object that is not controlled by the player, and the apparatus includes: an attribute feature extraction module for extracting a first attribute state feature of a first preset output in the current virtual game, where the first preset output is one of a skill output and a target output; Game feature extraction module, extracting game status features of this virtual game; an action output module, determining a first output action from the first preset output according to the extracted first attribute state feature and the game state feature; The strategy execution module forms an action strategy for the first virtual object based on the second attribute state characteristics of the second preset output in this virtual game and the attribute state characteristics of the determined first output action, so as to control the first virtual object to execute the action strategy in the game scene, where the second preset output is the other of the skill output and the target output.

16. An electronic device, characterized in that: include: A processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor. When the electronic device is running, the processor and the storage medium communicate via the bus, and the processor executes the machine-readable instructions to perform the steps of any method as claimed in claims 1 to 14.

17. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 14 are executed.

Images