Control method and apparatus for virtual object, and electronic device and storage medium

Abstract

Provided are a control method and apparatus for a virtual object, and an electronic device and a storage medium. A graphical user interface is provided by means of a terminal device, at least a part of a game scene of a current virtual battle is displayed on the graphical user interface, and the game scene comprises a first virtual object which is not controlled by a player. The method comprises: extracting a first attribute state feature of a first preset output in a current virtual battle (S101); extracting a game state feature of the current virtual battle (S102); on the basis of the first attribute state feature and the game state feature, determining a first output action from the first preset output (S103); and on the basis of a second attribute state feature of a second preset output in the current virtual battle and an attribute state feature of the first output action, forming an action strategy for a first virtual object (S104), so as to control the first virtual object to execute the action strategy in a game scene, thereby facilitating the improvement of the correlation between a skill output and a target output.

Description

Virtual object control method, device, electronic device and storage medium

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This disclosure claims priority to Chinese patent application number 202311101001.0, filed on August 29, 2023, entitled “Method, device, electronic device and storage medium for controlling virtual objects,” the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present disclosure relates to the field of computer application technology, and in particular to a method, device, electronic device, and storage medium for controlling a virtual object. Background Art

[0004] Artificial Intelligence (AI) refers to the theories, methods, techniques, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, to perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that seeks to understand the essence of intelligence and produce new intelligent machines that can respond in a manner similar to human intelligence. AI also studies the design principles and implementation methods of various intelligent machines, enabling them to possess the capabilities of perception, reasoning, and decision-making.

[0005] With the advancement of AI research and technology, it is being studied and applied in a variety of fields. For example, AI, known as game AI, is currently used in many scenarios in game development and applications. For example, during the game development process, game AI can replace the role of testers. By introducing game AI to participate in the game, test data is obtained to implement game performance testing. Another example is that during the game application process, if the number of players is insufficient, gamers can actively introduce game AI to play with real people.

[0006] Game AI is a crucial component of competitive games. Current game AI solutions fall into two main categories: non-learning and learning. Non-learning solutions employ finite state machines (FSMs) and behavior trees to design and implement the AI's behavioral logic, selecting appropriate behavior nodes for execution based on the game environment and character status. While this approach is easy to implement, it also presents numerous challenges. For example, the behavior tree approach requires extensive prior knowledge to pre-define a large number of behavior nodes and conditions to meet scenario requirements. This results in a lack of strategic diversity and flexibility, low combat intensity, poor anthropomorphism, and is easily predictable and decipherable by other players.

[0007] Learning-based solutions generally employ reinforcement learning methods to train and optimize game AI strategies. By acquiring reward signals through interaction with the environment, the AI's combat capabilities are enhanced. While reinforcement learning methods can improve strategy diversity, combat intensity, and anthropomorphism, the related design methods lack universal expressiveness and require separate development for different gameplay styles and characters. This results in high development and maintenance costs and makes it difficult to adapt to the growth and updates of game content.

[0008] Summary of the Invention

[0009] In view of this, the embodiments of the present disclosure provide at least one method, device, electronic device, and storage medium for controlling a virtual object to overcome at least one of the above-mentioned drawbacks.

[0010] In a first aspect, an exemplary embodiment of the present disclosure provides a method for controlling a virtual object, wherein a graphical user interface is provided through a terminal device, and the graphical user interface displays at least a portion of a game scene of a current virtual game, wherein the game scene includes a first virtual object that is not controlled by a player, and the following processing is performed at each decision point in the process of the current virtual game: extracting a first attribute state feature of a first preset output in the current virtual game, wherein the first preset output is one of a skill output and a target output; extracting a game state feature of the current virtual game; determining a first output action from the first preset output based on the extracted first attribute state feature and the game state feature; forming an action strategy for the first virtual object based on a second attribute state feature of a second preset output in the current virtual game and the attribute state feature of the determined first output action, to control the first virtual object to execute the action strategy in the game scene, wherein the second preset output is the other of the skill output and the target output.

[0011] In a second aspect, an embodiment of the present disclosure further provides a control device for a virtual object, which provides a graphical user interface through a terminal device, and displays at least a portion of a game scene of this virtual game, wherein the game scene includes a first virtual object that is not controlled by a player, and the device includes: an attribute feature extraction module, which extracts a first attribute state feature of a first preset output in this virtual game at each decision point in the process of this virtual game, wherein the first preset output is one of a skill output and a target output; a game feature extraction module, which extracts a game state feature of this virtual game; an action output module, which determines a first output action from the first preset output based on the extracted first attribute state feature and the game state feature; and a strategy execution module, which forms an action strategy for the first virtual object based on a second attribute state feature of a second preset output in this virtual game and the attribute state feature of the determined first output action, so as to control the first virtual object to execute the action strategy in the game scene, wherein the second preset output is the other of the skill output and the target output.

[0012] In a third aspect, an embodiment of the present disclosure further provides an electronic device, a processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor. When the electronic device is running, the processor and the storage medium communicate through the bus, and the processor executes the machine-readable instructions to perform the steps of the above-mentioned virtual object control method.

[0013] In a fourth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned virtual object control method are executed.

[0014] The control method, device, electronic device, and storage medium for virtual objects provided by the embodiments of the present disclosure achieve decoupling from the specific logic of the game, improve the versatility of game AI, and facilitate porting and use between different games and different skills.

[0015] In addition, in the embodiment of the present disclosure, the output actions for virtual objects are divided into two links: skill output and target output. By using one of the output actions as the input of another output action, the output action can be adjusted and optimized according to the previous output action, thereby increasing the correlation between the skill output and the target output, and helping to reduce the occurrence of mismatching of output skills and targets.

[0016] In addition, the control method, device, electronic device and storage medium of the virtual object according to the embodiments of the present disclosure can also improve the decision-making ability of the game AI, so that the game AI has a higher anthropomorphic effect and intelligence level.

[0017] In order to make the above-mentioned objectives, features and advantages of the present disclosure more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present disclosure and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.

[0019] FIG1 is a flowchart illustrating a method for controlling a virtual object provided by one of the exemplary embodiments of the present disclosure;

[0020] FIG2 is a schematic diagram showing a skill release shape and skill size provided by one of the exemplary embodiments of the present disclosure;

[0021] FIG3 is a flowchart showing the steps of obtaining the game status characteristics of the current virtual game provided by one of the exemplary embodiments of the present disclosure;

[0022] FIG4 shows a flowchart of steps for forming an action strategy for a first virtual object provided by one of the exemplary embodiments of the present disclosure;

[0023] FIG5 is a schematic diagram illustrating a first processing solution for forming an action strategy for a first virtual object provided by one of the exemplary embodiments of the present disclosure;

[0024] FIG6 shows a schematic diagram of a network structure corresponding to a first processing solution provided by one of the exemplary embodiments of the present disclosure;

[0025] FIG7 is a schematic diagram illustrating a second processing solution for forming an action strategy for a first virtual object provided by one of the exemplary embodiments of the present disclosure;

[0026] FIG8 is a schematic structural diagram of a virtual object control device provided by one of the exemplary embodiments of the present disclosure;

[0027] FIG9 shows a schematic structural diagram of an electronic device provided by one of the exemplary embodiments of the present disclosure. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. It should be understood that the drawings in the present disclosure are only for the purpose of illustration and description and are not used to limit the scope of protection of the present disclosure. In addition, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in the present disclosure illustrate operations implemented according to some embodiments of the present disclosure. It should be understood that the operations of the flowchart can be implemented out of sequence, and steps that have no logical contextual relationship can be reversed or implemented simultaneously. In addition, those skilled in the art, guided by the contents of the present disclosure, can add one or more other operations to the flowchart, or remove one or more operations from the flowchart.

[0029] The terms "a", "an", "the" and "said" are used in this specification to indicate the presence of one or more elements / components / etc.; the terms "including" and "having" are used to express open-ended inclusion and mean that additional elements / components / etc. may exist in addition to the listed elements / components / etc.; the terms "first" and "second" etc. are used only as labels and are not intended to limit the quantity of their objects.

[0030] It should be understood that in the embodiments of the present disclosure, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent three situations: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the previous and subsequent associated objects are in an "or" relationship. "Including A, B and / or C" means including any one, any two, or any three of A, B, and C.

[0031] It should be understood that in the embodiments of the present disclosure, "B corresponding to A," "B corresponding to A," "A corresponds to B," or "B corresponds to A" means that B is associated with A and B can be determined based on A. Determining B based on A does not mean determining B based solely on A; B can also be determined based on A and / or other information.

[0032] In addition, the described embodiments are only a part of the embodiments of the present disclosure, rather than all of the embodiments. The components of the embodiments of the present disclosure generally described and shown in the drawings herein can be arranged and designed in a variety of different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the drawings is not intended to limit the scope of the disclosure claimed for protection, but merely represents selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present disclosure.

[0033] During game development, game AI can replace human testers, allowing them to participate in the game and collect test data to test game performance. For example, during game development, if there aren't enough players, players can proactively introduce game AI to play alongside real players.

[0034] Game AI is a crucial component of competitive gaming. Traditional game AI primarily utilizes rule-based approaches, such as finite state machines and behavior trees. However, the resulting behavioral strategies are highly specialized, exhibit low combat intensity, lack humanlikeness, and are easily identified by other players. In recent years, reinforcement learning methods have empowered AI with powerful decision-making capabilities, enabling impressive performance in gaming. However, these reinforcement learning approaches lack universal expressive power. For example, diverse gameplay and class types require the construction of unique AI combat models for each. This means that different gameplay styles require independent modeling schemes, and newly added classes require retraining the game AI. This consumes significant computing resources, is time-consuming and labor-intensive, and increases development and maintenance costs.

[0035] In response to at least one of the above-mentioned problems, the present disclosure proposes a method, device, electronic device and storage medium for controlling a virtual object, so as to provide a universal combat model solution for game AI.

[0036] First, the names involved in the embodiments of the present disclosure are introduced.

[0037] Terminal equipment:

[0038] The terminal device involved in the embodiments of the present disclosure mainly refers to an intelligent device for providing a game screen (such as a relevant setting / configuration interface in the game, an interface for presenting a game scene) and capable of controlling a virtual character. The terminal device may include, but is not limited to, any one of the following devices: a smart phone, a tablet computer, a portable computer, a desktop computer, a game console, a personal digital assistant (PDA), an e-book reader, an MP4 (Moving Picture Experts Group Audio Layer IV, Moving Picture Experts Group Audio Layer 4) player, etc. An application that supports game scenes is installed and running in the terminal device, such as an application that supports three-dimensional game scenes. The application may include, but is not limited to, any one of a virtual reality application, a three-dimensional map program, a military simulation program, a MOBA game, a multiplayer gun battle survival game, and a third-person shooter game (TPS, Third-Personal Shooting Game). Optionally, the application may be a stand-alone application, such as a stand-alone 3D game program, or a network-connected application.

[0039] Graphical User Interface:

[0040] It is an interface display format for communication between people and computers, allowing users to manipulate icons, logos or menu options on the screen using input devices such as a mouse, keyboard and / or game controller, and also allowing users to manipulate icons or menu options on the screen by performing touch operations on the touch screen of a touch terminal to select commands, start programs or perform other tasks. Exemplarily, the graphical user interface may include virtual objects such as game characters, NPC characters, and AI characters that execute game logic in the game scene. The interface includes any visible visual controls or elements, for example, game controls (such as skill controls, movement controls, function controls, etc.), indicator signs (such as direction indicator signs, character indicator signs, etc.), information display areas (such as number of people defeated, game time, etc.), or game setting controls (such as system settings, stores, gold coins, etc.), and may also include controls such as pictures, input boxes, and text boxes, some of which respond to user operations.

[0041] Game scene:

[0042] It is a virtual environment displayed (or provided) when the application is running on a terminal device or server. Optionally, the game scene is a simulation of the real world, or a semi-simulation and semi-fictitious virtual environment, or a purely fictitious virtual environment. The game scene can be any of a two-dimensional virtual environment, a 2.5-dimensional virtual environment, and a three-dimensional virtual environment. The virtual environment can be the sky, land, ocean, etc. Among them, the game scene is a scene in which the user controls the complete game logic of the virtual character. Optionally, the game scene is also used for virtual environment battles between at least two virtual characters, and the game scene has virtual resources that can be used by at least two virtual characters.

[0043] Virtual Objects:

[0044] Refers to a virtual object in a game scene, which may be a virtual character controlled by a player, including but not limited to at least one of a virtual person, a virtual animal, and an anime character, and may also be a virtual object (NPC) controlled by a non-player. Optionally, when the virtual environment is a three-dimensional virtual environment, the virtual object may be a three-dimensional virtual model, each virtual object having its own shape and volume in the three-dimensional virtual environment, and occupying a portion of the space in the three-dimensional virtual environment. Optionally, the virtual object is a three-dimensional character constructed based on three-dimensional human skeleton technology, and the virtual object achieves different external images by wearing different skins. In some implementations, the virtual object may also be implemented using a 2.5-dimensional or 2-dimensional model, which is not limited in the embodiments of the present disclosure.

[0045] There may be multiple virtual objects in a virtual scene, which may be virtual characters controlled by players (i.e., characters controlled by players through input devices) and / or artificial intelligence (AI) trained to compete in a game scene. Optionally, the virtual objects are virtual characters competing in the game scene. Optionally, the number of virtual characters in the game scene is preset or dynamically determined based on the number of terminal devices participating in the virtual game, which is not limited in the embodiments of the present disclosure.

[0046] In an optional embodiment, the terminal device may be a local terminal device. Taking a game as an example, the local terminal device stores the game program and is used to present the game screen. The local terminal device is used to interact with the player through a graphical user interface (GUI), i.e., conventionally downloading and installing the game program through an electronic device and running it. The local terminal device may provide the GUI to the player in a variety of ways, such as rendering it on the terminal device's display screen or providing it to the player through holographic projection. For example, the local terminal device may include a display screen and a processor, the display screen being used to present the GUI, which includes the game screen, and the processor being used to run the game, generate the GUI, and control the display of the GUI on the display screen.

[0047] The present disclosure is applicable to the following application scenarios: The present disclosure can be applied to the field of gaming technology, where multiple players participate in a virtual game together.

[0048] Before entering a virtual game, players can select different character attributes, such as identity attributes, for their virtual objects in the virtual game. By assigning different character attributes to determine different camps, players can win the game by completing tasks assigned by the game during different stages of the virtual game. For example, multiple virtual objects with character attribute A can "eliminate" virtual objects with character attribute B during the battle stage to win the game. Alternatively, character attributes can be randomly assigned to each virtual object participating in the virtual game upon entering the virtual game.

[0049] An implementation environment provided in one embodiment of the present disclosure may include: a first terminal device, a server, and a second terminal device. The first terminal device and the second terminal device each communicate with the server to implement data communication. In this embodiment, the first terminal device and the second terminal device are each installed with an application program that executes the virtual character control method provided by the present disclosure, and the server is a server-side device that executes the virtual character control method provided by the present disclosure. Through the application program, the first terminal device and the second terminal device can each communicate with the server.

[0050] Taking the first terminal device as an example, the first terminal device establishes communication with the server by running an application. In an optional embodiment, the server establishes a virtual game based on the game request of the application. The parameters of the virtual game can be determined based on the parameters in the received game request. For example, the parameters of the virtual game may include the number of players participating in the virtual game, the levels of the characters participating in the virtual game, etc. When the first terminal device receives a response from the game server, the game scene corresponding to the virtual game is displayed through the graphical user interface of the first terminal device. The first terminal device is a device controlled by the first user, and the virtual object displayed in the graphical user interface of the first terminal device is the player character controlled by the first user. The first user inputs operation instructions through the graphical user interface to control the virtual object to perform corresponding operations in the game scene.

[0051] Taking the second terminal device as an example, the second terminal device establishes communication with the server by running an application. In an optional embodiment, the server establishes a virtual game based on the game request of the application. The parameters of the virtual game can be determined based on the parameters in the received game request. For example, the parameters of the virtual game may include the number of participants in the virtual game, the level of the characters participating in the virtual game, etc. When the second terminal device receives a response from the server, the game scene corresponding to the virtual game is displayed through the graphical user interface of the second terminal device. The second terminal device is a device controlled by a second user. The virtual object displayed in the graphical user interface of the second terminal device is the player character controlled by the second user. The second user inputs operation instructions through the graphical user interface to control the virtual object to perform corresponding operations in the virtual scene.

[0052] The server performs data calculation based on the game data reported by the first terminal device and the second terminal device, and synchronizes the calculated game data to the first terminal device and the second terminal device, so that the first terminal device and the second terminal device control the graphical user interface to render the corresponding game scene and / or virtual object according to the synchronization data sent by the game server.

[0053] In this embodiment, the virtual object controlled by the first terminal device and the virtual object controlled by the second terminal device are virtual objects in the same virtual game. The virtual object controlled by the first terminal device and the virtual object controlled by the second terminal device may have the same character attributes or different character attributes. The virtual object controlled by the first terminal device and the virtual object controlled by the second terminal device may belong to the same camp or different camps with hostile relations. The game AI virtual object participating in the virtual game may have a camp to which it belongs, for example, it may belong to the camp to which the virtual object controlled by the first terminal device belongs, or the camp to which the virtual object controlled by the second terminal device belongs. The game AI virtual object may also not belong to a camp and may interact with other virtual objects in the virtual game.

[0054] It should be noted that a virtual game may include two or more virtual objects, and different virtual objects may correspond to different terminal devices. That is to say, in a virtual game, there are more than two terminal devices that send and synchronize game data with the game server respectively.

[0055] The control method of the virtual character provided by the embodiments of the present disclosure can be applied to any of virtual reality applications, three-dimensional map programs, military simulation programs, multiplayer online tactical arena games (MOBAs), multiplayer gun battle survival games, third-person combat games, and first-person combat games.

[0056] The control method of the virtual character in one embodiment of the present disclosure can be run on a local terminal device or a server. When the control method is run on a server, the method can be implemented and executed based on a cloud interaction system, wherein the cloud interaction system includes a server and a client device.

[0057] In an optional embodiment, various cloud applications, such as cloud games, can be run under the cloud interaction system. Taking cloud games as an example, cloud games refer to a gaming method based on cloud computing. In the cloud gaming operation mode, the operating body of the game program and the main body of the game screen presentation are separated. The storage and operation of the control method are completed on the cloud gaming server. The role of the client device is to receive and send data and present the game screen. For example, the client device can be a display device with data transmission function close to the user side, such as a mobile terminal, TV, computer, PDA, etc.; but the cloud gaming server in the cloud is responsible for information processing. When playing the game, the player operates the client device to send operation instructions to the cloud gaming server. The cloud gaming server runs the game according to the operation instructions, encodes and compresses the game screen and other data, and returns it to the client device through the network. Finally, the client device decodes and outputs the game screen.

[0058] In an optional embodiment, taking a game as an example, a local terminal device stores a game program and is used to present the game screen. The local terminal device is used to interact with the player through a graphical user interface, that is, conventionally downloading and installing the game program through an electronic device and running it. The local terminal device can provide the graphical user interface to the player in a variety of ways, for example, it can be rendered and displayed on the terminal's display screen, or provided to the player through holographic projection. For example, the local terminal device may include a display screen and a processor, the display screen is used to present the graphical user interface, the graphical user interface includes the game screen, and the processor is used to run the game, generate the graphical user interface, and control the display of the graphical user interface on the display screen.

[0059] In a possible implementation, an embodiment of the present invention provides a method for controlling a virtual character, providing a graphical user interface through a terminal device, wherein the terminal device can be the local terminal device mentioned above, or a client device in the cloud interaction system mentioned above.

[0060] To facilitate understanding of the present disclosure, the virtual character control method, device, electronic device, and storage medium provided by the embodiments of the present disclosure are described in detail below.

[0061] Please refer to Figure 1, which is a flowchart of a virtual object control method provided by an exemplary embodiment of the present disclosure, which is generally applied to a game server, such as the cloud game server described above, but the present disclosure is not limited thereto.

[0062] A graphical user interface is provided through the terminal device, and the graphical user interface displays at least a portion of the game scene of the current virtual game, wherein the game scene includes a first virtual object that is not controlled by the player. Exemplarily, the processing steps shown in FIG1 can be executed at a preset decision point in the progress of the current virtual game.

[0063] In the disclosed embodiment, the first virtual object is an artificial intelligence character controlled by a non-player participating in the virtual game. The first virtual object can be configured to move in the game scene according to an action strategy, for example, controlling the first virtual object to run, jump, crawl, etc., and can also be controlled to use the virtual skills, virtual props, etc. provided by the virtual game to interact with other virtual objects in the game.

[0064] Exemplarily, multiple decision points will appear sequentially during the course of this virtual game. The processing mechanism of Figure 1 can be executed at each decision point in the course of this virtual game. For example, each time a decision point is triggered, the virtual game can automatically execute the virtual object control method disclosed herein to reduce the intervention of game players. The decision point can be pre-embedded in the virtual game, and when the game process meets the triggering conditions of the decision point, a decision is triggered. Alternatively, the decision point can also be automatically triggered at pre-set time intervals during the game process. In addition, the processing mechanism of Figure 1 can also be executed only at some decision points, and other mechanisms can be executed for other decision points. This disclosure does not limit this.

[0065] As shown in FIG1 , the control method of a virtual object according to an exemplary embodiment of the present disclosure specifically includes:

[0066] Step S101: extracting a first attribute state feature of a first preset output in this virtual game.

[0067] In the embodiment of the present disclosure, the action space for the virtual object is divided into two links: skill output and target output, and the first preset output is one of the skill output and the target output.

[0068] Here, skill outputs may include, but are not limited to, professional skills. In the disclosed embodiment, for less universal professional skills, a universal skill description representation is used to express them. For example, professional skills may be expressed using at least one of the following skill description representations: skill positioning, skill release shape, skill size used to describe the skill release shape, skill release range, skill release target, skill cooldown time, and skill segment number.

[0069] See Table 1 below, which shows the classification of skill positioning. For example, the skill positioning in the embodiment of the present disclosure includes 22 types, which can cover all skills of all occupations. Each skill can be composed of two positionings in the table.

[0070] Table 1

[0071] For example, skill release shapes may include but are not limited to the following six categories: self, rectangle, circle, fan, single body and trapezoid, and each skill release shape corresponds to its own skill size, as shown in FIG2 .

[0072] The skill's target is used to identify the target of a skill, specifically the type of target the skill is targeting. This includes, but is not limited to, at least one of the following: self, teammates, self and teammates, enemies, or all. The skill's range indicates the maximum distance a skill can be cast, and can be represented by a single continuous scalar value. In addition to the general skill attributes described above, class skills may also include a cooldown, usability, and skill stage count.

[0073] Here, in addition to professional skills, skill output may also include displacement skills. For example, displacement skills may include but are not limited to displacement skill types and flag words, and the flag words are used to indicate the available status of the displacement skills at the current decision point, that is, whether the displacement skills are available is indicated by assigning the value of the flag word.

[0074] The target output is used to indicate the target to which the skill output corresponds. The target output includes at least one of the following objects: the first virtual object itself, the second virtual object belonging to the same team as the first virtual object, and the third virtual object belonging to the enemy team as the first virtual object.

[0075] In the disclosed embodiment, skill description representations are used to express occupation-related skills, so that the game AI can fully understand and abstract the skill characteristics it carries.

[0076] Step S102: extracting game state features of this virtual game.

[0077] Here, the game state features include various features that can be used to characterize the game state of the virtual game. The process of obtaining the game state features is described below with reference to FIG. 3 .

[0078] FIG3 shows a flowchart of steps for obtaining game status features of a current virtual game, provided by an exemplary embodiment of the present disclosure.

[0079] 3 , in step S1021 , game state information at the current decision point in the current virtual game is obtained.

[0080] Here, when the decision point is triggered, the game status information of this virtual game at the moment of triggering can be obtained. For example, the game status information may include but is not limited to multiple attribute fields used to characterize the game status of this virtual game, and the multiple attribute fields include but are not limited to at least one of the following items: skill status information, radar map information, terrain information, and team status information.

[0081] For example, skill state information includes information representing various professional skills and displacement skills included in the current virtual game. Terrain information may include terrain information within a preset range of the first virtual object's position in the game scene at the current decision point. Exemplarily, this preset range may be the range corresponding to the game screen presented on the graphical user interface, or the maximum skill release range corresponding to the first virtual object. This terrain information represents the terrain used to construct the game scene, such as raised peaks, sunken canyons, hillsides, and walls of different materials.

[0082] The radar chart information includes a radar game map, which may include, but is not limited to, at least one of the following: the position of the first virtual object and its corresponding perspective, the first virtual object, the first virtual object's teammates, and the first virtual object's enemy NPCs. For example, the radar game map may also display the positional relationship between the first virtual object and its teammates and / or enemies. Team status information includes the status of all virtual objects participating in the virtual game, such as health, position, speed, damage, etc.

[0083] In step S1022, feature extraction is performed on the multiple attribute fields respectively to obtain multiple game attribute features.

[0084] Here, feature extraction can be performed on multiple attribute fields of the skill status information to obtain multiple game attribute features corresponding to the skill information. Similarly, radar map information, terrain information, and team status information can be extracted separately to obtain game attribute features corresponding to each information.

[0085] It should be understood that various existing feature extraction methods can be used to perform feature extraction, and the present disclosure does not limit the content of this part.

[0086] In step S1023, all game attribute features are spliced ​​together to obtain the game state features of this virtual game.

[0087] In a preferred embodiment, the game state features may comprise a game state vector, where each element of the game state vector corresponds to a pair of game attribute features. In other words, feature concatenation may involve arranging the game attribute features in a predetermined order to form the game state vector.

[0088] Preferably, the above-mentioned multiple attribute fields may also include situation information and / or self-category information. For example, the situation information is used to indicate whether the first virtual object can win at the current moment. For example, when the situation information is configured as a value of 1, it means: the current number of survivors on the own side is greater than the number of survivors on the enemy side, or the current number of survivors on the own side is equal to the number of survivors on the enemy side and the total damage of the own side is greater than the total damage of the enemy side. The value of 0 indicates the opposite. The self-category information can be used to indicate the game level of the virtual object or the type of the virtual object (such as attack type or auxiliary type).

[0089] In this case, the game state features of this virtual game can be obtained in the following manner: the game attribute features corresponding to the skill state information, radar map information, terrain information, and team state information are input into at least one fully connected layer to obtain attribute combination features, and the game attribute features corresponding to the situation information, the game attribute features corresponding to the own category information, and the attribute combination features are spliced ​​to obtain the game state features of this virtual game. This helps to improve the flexibility of obtaining game state features, that is, when transplanting and using between different games and different skills, it can be flexibly determined whether to introduce own category information and / or situation information. In addition, the above feature processing method also helps to improve the winning rate of virtual objects.

[0090] In this disclosed embodiment, situational information is introduced. This information, which indicates whether the team's victory is certain at the current moment, guides the game AI to adopt different strategies, such as aggressive or conservative, helping to improve the AI's combat effectiveness. Testing has verified that the win rate with and without situational information is 59.3% to 40.7%, respectively. Compared to existing behavior tree solutions, the win rate can be maintained at over 99%.

[0091] By extracting the above-mentioned attribute state features and game state features, the versatility of game AI can be improved and the development time and maintenance cost of game AI can be reduced.

[0092] Returning to FIG. 1 , step S103 : determining a first output action from the first preset output according to the extracted first attribute state feature and the game state feature.

[0093] In an embodiment of the present disclosure, the first preset output may include multiple first actions. In this case, the first attribute state feature may include at least one attribute feature corresponding to each first action, and the first output action is one determined from the multiple first actions.

[0094] Exemplarily, the first output action can be determined in the following manner: for each first action, the game state feature is fused with at least one attribute feature corresponding to the first action to obtain the fusion feature corresponding to the first action; based on the fusion feature corresponding to each first action, the first output action is determined from multiple first actions.

[0095] In a preferred embodiment, the first attribute state feature can be represented in matrix form. For example, the first attribute state feature can include a first preset feature matrix, each row of the first preset feature matrix corresponds to a first action, and each element in each row is used to represent an attribute feature corresponding to the first action.

[0096] In this case, the fused feature corresponding to each first action can be obtained by performing a dot product between the first preset feature matrix and the game state vector to obtain a first fused feature vector, where each element in the first fused feature vector represents the fused feature of the corresponding first action. Here, the fused feature corresponding to each first action represents the probability of that action being selected.

[0097] Here, the game state vector can be an M-dimensional column vector. Each row in the first preset feature matrix is ​​dot-producted with the game state vector to obtain the fused features of the first action corresponding to the row. The fused features of each first action form an M-dimensional first fused feature column vector. It should be understood that in order to perform a dot product operation on the matrix and vector, it is necessary to ensure that the number of columns in the matrix is ​​consistent with the dimension of the vector. Exemplarily, various methods can be used to make the two computable, such as through processing of a fully connected layer.

[0098] In a preferred embodiment, the first action can be screened by introducing a first available action mask, which can refer to an available mask related to the first preset output, that is, the available mask is used to characterize the first action that can be used by the first virtual object at the current decision point. Based on this, the first available action mask is used to screen out the available first action from multiple first actions. If the available first action is not unique, the screening can be continued from the available first actions based on the fusion feature. For example, the first action with the largest value of the fusion feature is determined as the first output action.

[0099] Step S104: Based on the second attribute state characteristics of the second preset output in the current virtual game and the attribute state characteristics of the determined first output action, an action strategy for the first virtual object is formed to control the first virtual object to execute the action strategy in the game scene. Here, the second preset output is the other of the skill output and the target output.

[0100] In an embodiment of the present disclosure, the output action of the first virtual object is divided into two links: skill output and target output. In order to increase the correlation between skill output and target output, the present disclosure adopts an autoregressive operation, that is, one of the output actions is used as the input of another output action, so that the output action can be adjusted and optimized according to the previous output action.

[0101] In addition, based on the correlation between the two actions, it is also possible to reduce the mismatching of output skills and targets. Tests have shown that the mismatching rate is reduced by 62.5%. By introducing a skill-related target output legal matrix (such as the flag words in the displacement skill), the mismatching situation can be completely eliminated, thereby improving the training efficiency of the game.

[0102] The process of forming an action strategy for the first virtual object will be described below with reference to FIG. 4 .

[0103] FIG4 shows a flowchart of steps for forming an action strategy for a first virtual object provided by an exemplary embodiment of the present disclosure.

[0104] 4 , in step S1041 , the game state feature is superimposed on the attribute state feature of the first output action to obtain the superimposed feature of the first output action;

[0105] In the case where the first attribute state feature includes the first preset feature matrix, the attribute state feature of the first output action includes each element in the row vector corresponding to the first output action extracted from the first preset feature matrix.

[0106] When the game state feature is a game state vector, the superposition feature of the first output action can be obtained by vector multiplication. For example, each element in the vector corresponding to the first output action can be multiplied with each element in the game state vector bit by bit to obtain the superposition feature of the first output action.

[0107] In step S1042 , the second attribute state feature of the second preset output is fused with the superimposed feature of the first output action to form an action strategy for the first virtual object.

[0108] In the embodiment of the present disclosure, the second preset output may include multiple second actions, and the second attribute state characteristics include at least one attribute characteristic corresponding to each second action.

[0109] Exemplarily, an action strategy for the first virtual object can be formed in the following manner: for each second action, the attribute characteristics corresponding to the second action are fused with the superimposed characteristics of the first output action to obtain the fusion characteristics corresponding to the second action; based on the fusion characteristics corresponding to each second action, the second output action is determined from multiple second actions; based on the superimposed characteristics of the first output action and the attribute state characteristics of the second output action, an action strategy for the first virtual object is formed.

[0110] In a preferred embodiment, the superimposed feature of the first output action is an action superimposed vector, which can represent the second attribute state feature in matrix form. For example, the second attribute state feature can include a second preset feature matrix, each row of the second preset feature matrix corresponds to a second action, and each element in each row is used to represent an attribute feature corresponding to the second action.

[0111] In this case, the fusion feature corresponding to each second action is obtained in the following manner: dot product is performed on the second preset feature matrix and the action superposition vector to obtain a second fusion feature vector, and each element in the second fusion feature vector is used to characterize the fusion feature of a corresponding second action.

[0112] Here, the action superposition vector can be an M-dimensional column vector. A dot product is performed on each row of the second preset feature matrix and the action superposition vector to obtain a fused feature of the second action corresponding to that row. The fused features of each second action form an M-dimensional second fused feature column vector. It should be understood that in order for the matrix and vector to perform a dot product operation, the number of columns in the matrix must be consistent with the dimension of the vector.

[0113] In a preferred embodiment, the second action can be screened by introducing a second available action mask, which can refer to an available mask related to the first preset output, that is, the available mask is used to characterize the available second action associated with the first virtual object and the first preset output at the current decision point. Based on this, the available second action is screened out from multiple second actions through the second available action mask. If the available second action is not unique, the screening can be continued from the available second actions based on the fusion feature. For example, the second action with the largest value of the fusion feature is determined as the second output action.

[0114] The action strategy for the first virtual object includes a combination of a first output action and a second output action, that is, a combination of skill output and target output, wherein the superimposed characteristics of the first output action cover the state characteristics of the action itself and the game state characteristics to capture the correlation and dependency between the action and the game state, and the first output action is used as the input of the second output action, thereby increasing the correlation between the two actions. Compared with related technical solutions, the combat AI disclosed in the present invention has been greatly improved in combat intensity and versatility.

[0115] In one example, the present disclosure utilizes an autoregressive neural network to process game state information, and designs two autoregressive structures for two categories of action space (skill output and target output).

[0116] In the first processing scheme, the skill output and skill feature matrix can be obtained based on neural network reasoning, and then the skill feature corresponding to the skill is selected, and after being fused with the feature of the game state information, the target is output.

[0117] The following describes the processing process of the first solution with reference to FIG5 and FIG6 for the case where the first preset output is a skill output.

[0118] FIG5 is a schematic diagram illustrating a first processing solution for forming an action strategy for a first virtual object provided by an exemplary embodiment of the present disclosure.

[0119] In this example, the first preset output is a skill output, which includes professional skills and displacement skills, and the first preset feature matrix includes a skill feature matrix.

[0120] As shown in FIG5 , the skill output information in this virtual game is obtained. The skill output information may include action information of various professional skills and displacement skills in this virtual game. For example, the displacement skills may include but are not limited to at least one of the following items: sprinting, jumping, dodging, and walking.

[0121] Feature extraction is performed on the acquired skill output information to obtain a plurality of skill attribute features, and a skill feature matrix is ​​formed using the acquired plurality of skill attribute features.

[0122] Referring to the dashed box in Figure 6 , the specific method for forming the skill feature matrix includes: extracting features from each class skill in this virtual game to form a class skill feature matrix, where Skill_desc represents multiple class skills, and Scope, CD, etc. include the corresponding attribute characteristics of the skill, such as the skill's release range and cooldown. Extracting features from each displacement skill in this virtual game to form a displacement skill feature matrix, where Action Id represents multiple types of displacement skills, and Legal represents a flag word.

[0123] For example, for displacement skills, the ID of the displacement skill is first embedded and then fused with the remaining features. For professional skills, the general skill representation is used, the skill positioning numbers in Table 1 are embedded, and the skill release shape and skill release target are encoded using a one-hot encoding operation, and then the remaining features are fused.

[0124] The formed occupational skill feature matrix and displacement skill feature matrix are subjected to feature shape alignment. For example, the occupational skill feature matrix and displacement skill feature matrix can be subjected to FC (Fully Connected Layers) processing to perform feature shape alignment so that the number of columns of the two matrices is consistent.

[0125] The occupational skill feature matrix and the displacement skill feature matrix, after feature shape alignment, are concatenated to obtain a skill feature matrix. For example, the above matrix concatenation may refer to placing each row of the occupational skill feature matrix below the last row of the displacement skill feature matrix, or placing each row of the displacement skill feature matrix below the last row of the occupational skill feature matrix, or arranging the rows of the two matrices in a certain order or randomly.

[0126] In the embodiment of the present disclosure, a deep reinforcement learning algorithm is used to train combat AI, and the state s, action a and reward r are first defined.

[0127] For example, the state S used in this disclosure includes the following eight components: occupational skill information, displacement skill information, radar map information, terrain information, team status information, buff information, situation information, and self-category information. For occupational skill information with weak universality, this disclosure uses universal skill representations to express it. Universal skill representations include, but are not limited to, skill positioning, skill release shape, skill size, skill release target, and skill release range.

[0128] Action A in this disclosure uses a combination of two actions: skills and targets. Skill outputs include displacement skills and professional skills, and targets include the player, teammates, and opponents. When executing a decision, a skill is selected based on the neural network, and then a target is selected based on that skill, ultimately executing the combined action.

[0129] The reward function r disclosed herein combines individual rewards and team rewards. Individual rewards include killing opponents, causing damage, sustaining injuries, and healing. Team rewards include the number of team deaths and the final victory or defeat. Optionally, in order to encourage the game AI to actively fight, the reward function also introduces a time-related penalty, that is, a penalty is executed every time a decision is made. Exemplarily, the above reward function can be used as an evaluation indicator for game AI training. After each training is completed, the various parameters in the autoregressive network are adjusted based on the reward function to change the training strategy so that the strategy ends the battle faster.

[0130] In a preferred embodiment of the present disclosure, the multiple attribute fields further include gain status information, and the gain status information includes status attributes corresponding to all virtual objects participating in the current virtual game.

[0131] In this case, the game attribute features corresponding to the team status information can be obtained in the following ways: feature extraction is performed on the attribute field used to represent the gain status information to obtain multiple gain status features, the multiple gain status features are compressed to obtain a fused gain feature, feature extraction is performed on the attribute field used to represent the team status information to obtain the team status features corresponding to each virtual object participating in this virtual game, and the gain features are respectively spliced ​​with the team status features corresponding to each virtual object to form the game attribute features corresponding to the team status information.

[0132] In the network structure shown in Figure 6, an autoregressive deep neural network is designed to process game state information and output actions. This includes using a multi-head attention structure to process skill information and team state information, thereby capturing the correlation and dependency between skills and team members.

[0133] For the buff information, its corresponding ID is embedded and encoded, and then fused with the remaining features. After that, the features are compressed through two layers of FC and one layer of Maxpooling to obtain a fused gain feature.

[0134] For example, for each virtual object, a fused gain feature can be added after, before, or at other locations to the corresponding team status features. The team status features of each virtual object after the splicing are directly processed using Embedding and Multi-Head Attention.

[0135] For displacement skills, the displacement skill ID is first embedded, then fused with the remaining features, and finally processed by Embedding and Multi-Head Attention.

[0136] For professional skills, a general skill representation is used to perform an embedding operation on the skill location numbers in Table 1. A one-hot encoding operation is performed on the skill release shape and skill release target. The remaining features are then integrated and finally processed through embedding and multi-head attention.

[0137] Two layers of FC are used to process other scalar states (such as radar map information, terrain information, self-category information, and situation information).

[0138] In this example, the game attribute features corresponding to displacement skills, professional skills, radar maps, terrain information, and team status information are input into two layers of FC to obtain attribute combination features. Then, the game attribute features corresponding to the situation information, the game attribute features corresponding to the own category information, and the attribute combination features are concatenated to obtain the game status features of this virtual game.

[0139] The above-mentioned game state features are processed by embedding and then dot-producted with the skill feature matrix. The dot product results are processed by the available skill mask to determine the output skill output. The skill feature matrix is ​​subjected to gather+sigmoid processing to obtain a row of attribute features corresponding to the skill output. The attribute features corresponding to the skill output are then superimposed (× multiplied) with the game state features after embedding processing to obtain the superimposed features of the skill output. The superimposed features of the skill output are dot-producted (dot) with the target feature matrix to obtain the fusion features corresponding to each target. The dot product results are processed by the available target mask to determine the output target output.

[0140] Each row of the target feature matrix corresponds to a target, and each element in each row of the target feature matrix corresponds to an attribute feature of the target. Here, the targets forming the target feature matrix may include, but are not limited to, all virtual objects participating in the virtual game, for example, the first virtual object itself, at least one second virtual object that is a teammate of the first virtual object (which may be a virtual object controlled by another terminal device or an NPC), and at least one third virtual object that is an enemy of the first virtual object (which may be a virtual object controlled by another terminal device or an NPC).

[0141] In one example, a row of attribute features corresponding to a target may be the game attribute features obtained by extracting the team status information as described above, which may include, but are not limited to: the health value, position, speed value, and damage value corresponding to the target.

[0142] In another example, a row of attribute features corresponding to a target can also be the match attribute features obtained by extracting the team state information and gain state information. Exemplarily, multiple gain state features are compressed to obtain a fused gain feature, which is then concatenated with the team state feature corresponding to the target to form the match attribute feature for the target.

[0143] For output actions, the autoregressive network uses the first processing scheme shown in Figure 6. The network first outputs the skill output, then selects the corresponding skill feature based on the selected skill output. This skill feature is then dot-producted with the game state feature to associate the target selection with the skill output. Before the target output, a skill-related available target mask is introduced to avoid mismatches between the skill output and the target output, thereby accelerating the policy learning process.

[0144] In the second processing scheme, the target output and target feature matrix can be obtained based on neural network reasoning, and then the target feature corresponding to the target is selected and fused with the feature of the game state information to output the skill (as shown in Figure 7).

[0145] The difference between the second processing scheme and the first processing scheme is that in the first processing scheme, the skills are output first and then the target is selected, while in the second processing scheme, the target is output first and then the skills are selected. The two schemes are similar in terms of feature extraction, processing, superposition, and fusion methods in the processing process, and the content of this part will not be repeated in this disclosure.

[0146] Figures 5 and 7 show two different autoregressive structures. The essence of these two structures is to strengthen the connection between outputs, thereby reducing the mismatch between skills and target selection. After testing and verification, the winning rate of the first processing scheme is approximately 60%:40% compared with the second processing scheme.

[0147] In a preferred embodiment of the present disclosure, the aforementioned virtual object control method may further include: executing a reward strategy after executing the action strategy for the first virtual object. Exemplarily, the reward strategy includes individual rewards and / or team rewards, with individual rewards including positive impact rewards representing feedback results of the action strategy and negative impact rewards for settlement at decision points.

[0148] The control method of the virtual object disclosed above can achieve at least the following effective effects:

[0149] To reduce development time, this paper trains combat AI in a martial arts scenario, and subsequently integrates it into the Imperial City 1v1 and intelligent NPC scenarios. Compared with related reinforcement learning solutions, it effectively shortens the development cycle of integration.

[0150] This solution reduces maintenance costs and offers scalability for new professions. Previous reinforcement learning solutions require retraining for each new profession, requiring approximately 650,000 rounds. This proposed approach only requires 50,000 rounds of fine-tuning based on the existing model, surpassing the previous solution in win rate and increasing efficiency by 13 times.

[0151] High strength and anthropomorphism: Compared with related behavior tree solutions, the win rate of this disclosure remains above 99%, and the AI ​​trained through reinforcement learning is closer to humans in combat, and can dodge skills, cooperate with teammates to focus fire on targets, escape with low health, and other behaviors.

[0152] To address the poor universality of current game AI combat models, this solution uses a deep reinforcement learning algorithm. It employs universal skill descriptions to represent less universal professional skill states. It also incorporates embedding technology to design an autoregressive network structure, introducing situational states to enhance AI expressiveness. This allows the combat model to be well compatible with different types of intelligent agents and gameplay, reducing the time and cost of game AI development.

[0153] Based on the same application concept, the embodiments of the present disclosure also provide a control device for a virtual object corresponding to the method provided in the above embodiments. Since the principle of solving the problem by the device in the embodiments of the present disclosure is similar to the control method for a virtual object in the above embodiments of the present disclosure, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0154] FIG8 is a schematic diagram of the structure of a virtual object control device provided by an exemplary embodiment of the present disclosure. As shown in FIG8 , a graphical user interface is provided by a terminal device, wherein the graphical user interface displays at least a portion of the game scene of the current virtual game, wherein the game scene includes a first virtual object that is not controlled by the player. The control device 200 of the virtual object includes:

[0155] The attribute feature extraction module 210 is configured to extract a first attribute state feature of a first preset output in the current virtual game, where the first preset output is one of a skill output and a target output;

[0156] The game feature extraction module 220 is configured to extract game state features of the current virtual game;

[0157] The action output module 230 is configured to determine a first output action from the first preset output according to the extracted first attribute state feature and game state feature;

[0158] The strategy execution module 240 is configured to execute the second attribute state characteristics of the second preset output in this virtual game and the attribute state characteristics of the determined first output action to form an action strategy for the first virtual object to control the first virtual object to execute the action strategy in the game scene, where the second preset output is the other of the skill output and the target output.

[0159] In one possible implementation of the present disclosure, the first preset output includes multiple first actions, and the first attribute state feature includes at least one attribute feature corresponding to each first action, wherein the action output module 230 is configured to determine the first output action in the following manner: for each first action, the game state feature is fused with at least one attribute feature corresponding to the first action to obtain the fusion feature corresponding to the first action, and based on the fusion feature corresponding to each first action, the first output action is determined from the multiple first actions.

[0160] In one possible implementation of the present disclosure, the first attribute state feature includes a first preset feature matrix, each row of the first preset feature matrix corresponds to a first action, and each element in each row is used to characterize an attribute feature corresponding to the first action, and the game state feature includes a game state vector, wherein the action output module 230 is configured to execute the following method to obtain the fusion feature corresponding to each first action: dot product the first preset feature matrix with the game state vector to obtain a first fusion feature vector, and each element in the first fusion feature vector is used to characterize the fusion feature of the corresponding first action.

[0161] In a possible implementation of the present disclosure, the first preset output includes skill output, the skill output includes professional skills and displacement skills, and the first preset feature matrix includes a skill feature matrix, wherein the attribute feature extraction module 210 is configured to execute the formation of the skill feature matrix in the following manner: feature extraction is performed on each professional skill in this virtual game to form a professional skill feature matrix, feature extraction is performed on each displacement skill in this virtual game to form a displacement skill feature matrix, feature shape alignment processing is performed on the formed professional skill feature matrix and displacement skill feature matrix, and the professional skill feature matrix and displacement skill feature matrix after the feature shape alignment processing are spliced ​​to obtain a skill feature matrix.

[0162] In a possible implementation manner of the present disclosure, the attribute state feature of the first output action includes each element in a row vector corresponding to the first output action extracted from a first preset feature matrix.

[0163] In one possible implementation of the present disclosure, the game feature extraction module 220 is further configured to obtain game status information at a current decision point in this virtual game, where the game status information includes multiple attribute fields for characterizing the game status of this virtual game, and the multiple attribute fields include at least one of the following items: skill status information, radar chart information, terrain information, and team status information; perform feature extraction on the multiple attribute fields respectively to obtain multiple game attribute features; and splice all game attribute features to obtain the game status features of this virtual game.

[0164] In a possible implementation manner of the present disclosure, the multiple attribute fields further include situation information and / or self-category information.

[0165] Among them, the game feature extraction module 220 is also configured to input each game attribute feature corresponding to the skill status information, radar map information, terrain information, and team status information into at least one fully connected layer to obtain attribute combination features, and splice the game attribute features corresponding to the situation information, the game attribute features corresponding to the own category information, and the attribute combination features to obtain the game status features of this virtual game.

[0166] In one possible implementation of the present disclosure, the multiple attribute fields also include gain status information, wherein the game feature extraction module 220 is configured to execute the following method to obtain the game attribute features corresponding to the team status information: feature extraction of the attribute field used to characterize the gain status information to obtain multiple gain status features, compression processing of the multiple gain status features to obtain a one-dimensional gain feature, feature extraction of the attribute field used to characterize the team status information to obtain the team status feature corresponding to each virtual object participating in this virtual game, and splicing the gain feature with the team status feature corresponding to each virtual object to form the game attribute feature corresponding to the team status information.

[0167] In one possible implementation of the present disclosure, the strategy execution module 240 is configured to execute an action strategy for the first virtual object in the following manner: superimposing the game state feature onto the attribute state feature of the first output action to obtain the superimposed feature of the first output action; and fusing the second attribute state feature of the second preset output with the superimposed feature of the first output action to form an action strategy for the first virtual object.

[0168] In one possible embodiment of the present disclosure, the second preset output includes multiple second actions, and the second attribute state characteristics include at least one attribute feature corresponding to each second action, wherein the strategy execution module 240 is configured to execute an action strategy for the first virtual object in the following manner: for each second action, the attribute feature corresponding to the second action is fused with the superimposed feature of the first output action to obtain the fusion feature corresponding to the second action; based on the fusion feature corresponding to each second action, the second output action is determined from the multiple second actions; and the action strategy is formed based on the superimposed feature of the first output action and the attribute state characteristics of the second output action.

[0169] In one possible embodiment of the present disclosure, the second attribute state feature includes a second preset feature matrix, each row of the second preset feature matrix corresponds to a second action, each element in each row is used to characterize an attribute feature corresponding to the second action, and the superposition feature of the first output action includes an action superposition vector, wherein the strategy execution module 240 is configured to execute the following method to obtain the fusion feature corresponding to each second action: perform a dot product between the second preset feature matrix and the action superposition vector to obtain a second fusion feature vector, and each element in the second fusion feature vector is used to characterize the fusion feature of a corresponding second action.

[0170] In one possible implementation of the present disclosure, the strategy execution module 240 is further configured to execute a reward strategy after executing the action strategy, wherein the reward strategy includes individual rewards and / or team rewards, and the individual rewards include positive impact rewards for characterizing the feedback results of the action strategy and negative impact rewards for settlement at decision points.

[0171] In one possible embodiment of the present disclosure, the skill output includes professional skills, and the professional skills are expressed using at least one skill description representation of the following items: skill positioning, skill release shape, skill size used to describe the skill release shape, skill release range, skill release target, skill cooling time, skill segment number, and / or, the target output is used to indicate the application target corresponding to the skill output, and the target output includes one of the following objects: the first virtual object itself, the second virtual object belonging to the same team as the first virtual object, and the third virtual object belonging to the enemy team as the first virtual object.

[0172] In a possible implementation manner of the present disclosure, the skill output further includes a displacement skill, and the displacement skill includes a displacement skill type and a flag word, and the flag word is used to indicate the available status of the displacement skill at the current decision point.

[0173] The virtual object control device of the disclosed embodiment improves the versatility of game AI and reduces development and maintenance costs.

[0174] Please refer to Figure 9 , which is a schematic diagram of the structure of an electronic device provided by an exemplary embodiment of the present disclosure. As shown in Figure 9 , the electronic device 300 includes a processor 310 , a memory 320 , and a bus 330 .

[0175] The memory 320 stores machine-readable instructions executable by the processor 310. When the electronic device 300 is running, the processor 310 communicates with the memory 320 via the bus 330. When the machine-readable instructions are executed by the processor 310, the steps of the virtual object control method in any of the above embodiments can be performed, which are specifically as follows:

[0176] A graphical user interface is provided through a terminal device, and at least a portion of the game scene of this virtual game is displayed on the graphical user interface, wherein the game scene includes a first virtual object that is not controlled by the player, and the following processing is performed at each decision point in the process of this virtual game: extracting a first attribute state feature of a first preset output in this virtual game, wherein the first preset output is one of a skill output and a target output; extracting a game state feature of this virtual game; determining a first output action from the first preset output based on the extracted first attribute state feature and the game state feature; forming an action strategy for the first virtual object based on a second attribute state feature of a second preset output in this virtual game and the attribute state feature of the determined first output action, so as to control the first virtual object to execute the action strategy in the game scene, wherein the second preset output is the other of the skill output and the target output.

[0177] The electronic device according to the embodiment of the present disclosure improves the versatility of game AI and reduces development and maintenance costs.

[0178] The present disclosure also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the method for controlling a virtual object in any of the above embodiments are performed, specifically as follows:

[0179] A graphical user interface is provided through a terminal device, and at least a portion of the game scene of this virtual game is displayed on the graphical user interface, wherein the game scene includes a first virtual object that is not controlled by the player, and the following processing is performed at each decision point in the process of this virtual game: extracting a first attribute state feature of a first preset output in this virtual game, wherein the first preset output is one of a skill output and a target output; extracting a game state feature of this virtual game; determining a first output action from the first preset output based on the extracted first attribute state feature and the game state feature; forming an action strategy for the first virtual object based on a second attribute state feature of a second preset output in this virtual game and the attribute state feature of the determined first output action, so as to control the first virtual object to execute the action strategy in the game scene, wherein the second preset output is the other of the skill output and the target output.

[0180] The computer-readable storage medium of the embodiment of the present disclosure improves the versatility of game AI and reduces development and maintenance costs.

[0181] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system and device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. In the several embodiments provided in the present disclosure, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0182] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0183] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0184] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the relevant technology or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0185] The above are only specific embodiments of the present disclosure, but the scope of protection of the present disclosure is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this disclosure should be included in the scope of protection of the present disclosure. Therefore, the scope of protection of the present disclosure should be based on the scope of protection of the claims.

Claims

1. A method for controlling a virtual object, wherein a terminal device provides a graphical user interface, wherein the graphical user interface displays at least a portion of a game scene of a current virtual game, wherein the game scene includes a first virtual object that is not controlled by a player, and the method comprises: Extracting a first attribute state feature of a first preset output in the current virtual game, where the first preset output is one of a skill output and a target output; Extract the game status features of this virtual game; Determining a first output action from the first preset output according to the extracted first attribute state feature and game state feature; Based on the second attribute state characteristics of the second preset output in this virtual game and the attribute state characteristics of the determined first output action, an action strategy for the first virtual object is formed to control the first virtual object to execute the action strategy in the game scene, and the second preset output is the other of the skill output and the target output.

2. The method according to claim 1, wherein: The first preset output includes a plurality of first actions, and the first attribute state feature includes at least one attribute feature corresponding to each first action; The first output action is determined in the following manner: For each first action, the game state feature is fused with at least one attribute feature corresponding to the first action to obtain a fused feature corresponding to the first action; Based on the fusion feature corresponding to each first action, a first output action is determined from the multiple first actions.

3. The method according to claim 2, wherein: The first attribute state feature comprises a first preset feature matrix, each row of the first preset feature matrix corresponds to a first action, each element in each row is used to characterize an attribute feature corresponding to the first action, and the game state feature comprises a game state vector; The fusion feature corresponding to each first action is obtained in the following way: A dot product is performed on the first preset feature matrix and the game state vector to obtain a first fused feature vector, wherein each element in the first fused feature vector is used to characterize a fused feature of a corresponding first action.

4. The method according to claim 3, wherein: The first preset output includes a skill output, the skill output includes a professional skill and a displacement skill, and the first preset feature matrix includes a skill feature matrix; Among them, the skill feature matrix is ​​formed in the following way: Extract features of each professional skill in this virtual game to form a professional skill feature matrix; Extract features of each displacement skill in this virtual game to form a displacement skill feature matrix; Performing feature shape alignment processing on the formed occupational skill feature matrix and displacement skill feature matrix; The occupational skill feature matrix and the displacement skill feature matrix after feature shape alignment are spliced ​​to obtain a skill feature matrix.

5. The method according to claim 3, wherein: The attribute state feature of the first output action includes each element in the row vector corresponding to the first output action extracted from the first preset feature matrix.

6. The method according to claim 1, wherein: The steps of extracting the game state features of this virtual game include: Obtaining game state information at a current decision point in the current virtual game, the game state information comprising a plurality of attribute fields for characterizing the game state of the current virtual game, the plurality of attribute fields comprising at least one of the following items: skill state information, radar map information, terrain information, and team state information; Performing feature extraction on the multiple attribute fields respectively to obtain multiple game attribute features; All game attribute features are spliced ​​together to obtain the game state features of this virtual game.

7. The method according to claim 6, wherein: The plurality of attribute fields also include situation information and / or own category information; The step of combining all game attribute features to obtain the game state features of this virtual game includes: Input the attribute characteristics of each game corresponding to the skill status information, radar map information, terrain information, and team status information Enter into at least one fully connected layer to obtain attribute combination features; The game attribute features corresponding to the situation information, the game attribute features corresponding to the category information itself, and the attribute combination features are spliced ​​together to obtain the game state features of this virtual game.

8. The method according to claim 6, wherein: The plurality of attribute fields also include gain status information; Among them, the game attribute characteristics corresponding to the team status information are obtained in the following way: Extracting features from attribute fields used to characterize gain state information to obtain multiple gain state features; Performing compression processing on the plurality of gain state characteristics to obtain a gain characteristic; Extract features from the attribute fields used to represent team status information, and obtain team status features corresponding to each virtual object participating in this virtual game; The gain features are respectively concatenated with the team status features corresponding to each virtual object to form game attribute features corresponding to the team status information.

9. The method according to claim 1, wherein: The action strategy for the first virtual object is formed in the following way: Superimposing the game state feature onto the attribute state feature of the first output action to obtain the superimposed feature of the first output action; The second attribute state feature of the second preset output is fused with the superimposed feature of the first output action to form an action strategy for the first virtual object.

10. The method according to claim 9, wherein: The second preset output includes a plurality of second actions, and the second attribute state feature includes at least one attribute feature corresponding to each second action; The action strategy for the first virtual object is formed in the following manner: For each second action, the attribute feature corresponding to the second action is fused with the superposition feature of the first output action to obtain a fused feature corresponding to the second action; Determine a second output action from the plurality of second actions based on the fusion feature corresponding to each second action; The action strategy is formed based on the superposition feature of the first output action and the attribute state feature of the second output action.

11. The method according to claim 10, wherein: The second attribute state feature includes a second preset feature matrix, each row of the second preset feature matrix corresponds to a second action, each element in each row is used to characterize an attribute feature corresponding to the second action, and the superposition feature of the first output action includes an action superposition vector; The fusion feature corresponding to each second action is obtained in the following way: A dot product is performed on the second preset feature matrix and the action superposition vector to obtain a second fused feature vector, wherein each element in the second fused feature vector is used to represent a fused feature of a corresponding second action.

12. The method according to claim 1, wherein: Also includes: After executing the action strategy, executing the reward strategy; The reward strategy includes individual rewards and / or team rewards, and the individual rewards include positive impact rewards for characterizing the feedback results of the action strategy and negative impact rewards for settlement at decision points.

13. The method according to claim 1, wherein: The skill output includes a professional skill, and the professional skill is expressed using at least one of the following skill description representations: skill positioning, skill release shape, skill size for describing the skill release shape, skill release range, skill release target, skill cooldown time, and skill segment number; And / or, the target output is used to indicate the target to which the skill output corresponds, and the target output includes at least one of the following objects: the first virtual object itself, a second virtual object belonging to the same team as the first virtual object, and a third virtual object belonging to the enemy team as the first virtual object.

14. The method according to claim 13, wherein: The skill output also includes a displacement skill, and the displacement skill includes a displacement skill type and a flag word, and the flag word is used to indicate the available status of the displacement skill at the current decision point.

15. A control device for a virtual object, wherein: A graphical user interface is provided through a terminal device, wherein the graphical user interface displays at least a portion of a game scene of the current virtual game, wherein the game scene includes a first virtual object that is not controlled by a player, and the apparatus includes: An attribute feature extraction module is configured to extract a first attribute state feature of a first preset output in the current virtual game, where the first preset output is one of a skill output and a target output; A game feature extraction module is configured to extract game state features of the current virtual game; an action output module, configured to determine a first output action from the first preset output according to the extracted first attribute state feature and game state feature; The strategy execution module is configured to execute an action strategy for the first virtual object based on a second attribute state characteristic of a second preset output in this virtual game and the attribute state characteristic of the determined first output action, so as to control the first virtual object to execute the action strategy in the game scene, wherein the second preset output is the other of the skill output and the target output.

16. An electronic device, comprising: A processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the storage medium communicate via the bus, and the processor executes the machine-readable instructions to perform the steps of any method as claimed in claims 1 to 14.

17. A computer-readable storage medium having a computer program stored thereon, wherein the computer program executes the steps of any one of the methods of claims 1 to 14 when executed by a processor.

Images