Action decision-making methods, devices, equipment, storage media, and program products for virtual objects

Abstract

This application provides a method, apparatus, device, storage medium, and program product for action decision-making of virtual objects. The method includes: acquiring first layout information and feature information of the virtual object in a virtual scene; performing probability prediction processing corresponding to the first layout information based on the first layout information and the feature information to obtain the selection probability of each action in a preset first action set; and selecting at least one action from the first action set based on the selection probability of each action as an action strategy to be executed by the virtual object. This application enables accurate diversification of action strategies for virtual objects.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method, apparatus, device, storage medium, and program product for making action decisions for virtual objects. Background Technology

[0002] Action decision-making methods for virtual objects in related technologies include schemes based on supervised learning (SL) and reinforcement learning (RL). These schemes predict the action strategies of virtual objects by training a unified model. However, this approach, which uses a unified model for prediction, cannot adapt to the increasing complexity of virtual scenes. As a result, the predicted action strategies are too homogeneous, leading to rigid and monotonous actions of virtual objects and affecting the intelligence level of human-computer interaction in virtual scenes.

[0003] The relevant technologies lack a solution for accurately predicting the action strategies of virtual objects. Summary of the Invention

[0004] This application provides a method, apparatus, device, storage medium, and program product for making action decisions for virtual objects, which can accurately realize the diversification of action strategies for virtual objects.

[0005] The technical solution of this application embodiment is implemented as follows:

[0006] This application provides a method for making action decisions for a virtual object, the method comprising:

[0007] Obtain the first layout information and feature information of the virtual objects in the virtual scene;

[0008] Based on the first layout information and the feature information, a probability prediction process corresponding to the first layout information is performed to obtain the selection probability of each action in the preset first action set.

[0009] Based on the selection probability of each action, at least one action is selected from the first set of actions as the action strategy to be executed by the virtual object.

[0010] This application provides an action decision device for a virtual object, including:

[0011] The data acquisition module is used to acquire the first layout information and feature information of the virtual objects in the virtual scene;

[0012] The action prediction module is used to perform probability prediction processing corresponding to the first layout information based on the first layout information and the feature information, so as to obtain the selection probability of each action in the preset first action set.

[0013] The action prediction module is further configured to select at least one action from the action set based on the selection probability of each action, as the action strategy to be executed by the virtual object.

[0014] This application provides an electronic device, the electronic device comprising:

[0015] Memory is used to store executable instructions or computer programs.

[0016] The processor, when executing computer-executable instructions or computer programs stored in the memory, implements the action decision method for virtual objects provided in the embodiments of this application.

[0017] This application provides a computer-readable storage medium storing a computer program or computer-executable instructions, which, when executed by a processor, implements the action decision method for virtual objects provided in this application.

[0018] This application provides a computer program product, including a computer program or computer executable instructions. When the computer program or computer executable instructions are executed by a processor, they implement the action decision method for virtual objects provided in this application.

[0019] The embodiments of this application have the following beneficial effects:

[0020] By performing probability prediction processing corresponding to the first layout information and feature information of virtual objects, differentiated probability prediction processing is achieved for virtual objects with different layout information. This takes into account the unique layout information and feature information of each virtual object, and can provide more targeted information for probability prediction. This allows the predicted action strategy to be adapted to the layout information and feature information of the virtual object, which not only ensures the accuracy of the action strategy, but also effectively realizes the diversification of action strategies for different virtual objects, thus better adapting to multi-virtual object systems. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of the architecture of the virtual object action decision system provided in the embodiments of this application;

[0022] Figure 2 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application;

[0023] Figure 3A This is a first flowchart illustrating the action decision-making method for virtual objects provided in this application embodiment;

[0024] Figure 3BThis is a second flowchart illustrating the action decision-making method for virtual objects provided in this application embodiment;

[0025] Figure 3C This is a third flowchart illustrating the action decision-making method for virtual objects provided in this application embodiment;

[0026] Figure 3D This is a schematic diagram of the fourth process of the action decision-making method for virtual objects provided in the embodiments of this application;

[0027] Figure 3E This is a fifth flowchart illustrating the action decision-making method for virtual objects provided in this application embodiment;

[0028] Figure 3F This is a sixth flowchart illustrating the action decision-making method for virtual objects provided in this application embodiment;

[0029] Figure 3G This is a seventh flowchart illustrating the action decision-making method for virtual objects provided in this application embodiment;

[0030] Figure 3H This is the eighth flowchart of the action decision-making method for virtual objects provided in the embodiments of this application;

[0031] Figure 3I This is the ninth flowchart of the action decision-making method for virtual objects provided in the embodiments of this application;

[0032] Figure 3J This is the tenth flowchart of the action decision-making method for virtual objects provided in the embodiments of this application;

[0033] Figure 3K This is a schematic diagram of the eleventh step of the action decision-making method for virtual objects provided in the embodiments of this application;

[0034] Figure 4 This is a schematic diagram of the application process of the action decision method for virtual objects provided in the embodiments of this application;

[0035] Figure 5A This is a first schematic diagram illustrating the principle of the virtual object action decision-making method provided in the embodiments of this application;

[0036] Figure 5B This is a second schematic diagram illustrating the principle of the virtual object action decision-making method provided in the embodiments of this application;

[0037] Figure 5C This is a third schematic diagram illustrating the principle of the virtual object action decision-making method provided in the embodiments of this application;

[0038] Figure 5D This is a fourth schematic diagram illustrating the principle of the virtual object action decision-making method provided in the embodiments of this application;

[0039] Figure 5E This is the fifth schematic diagram illustrating the principle of the virtual object action decision-making method provided in the embodiments of this application;

[0040] Figure 6 This is a schematic diagram illustrating the construction principle of the fine-tuning weight parameters provided in the embodiments of this application.

[0041] It should be noted that the terms "first" and "second" mentioned above are only used to distinguish between different options and do not represent the degree of superiority or inferiority of the options or their priority in the implementation process. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0043] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0044] In the following description, the terms "first, second, third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0045] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0046] Unless otherwise defined, all technical and scientific terms used in the embodiments of this application have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in the embodiments of this application is for the purpose of describing the embodiments of this application only and is not intended to limit this application.

[0047] In the implementation of this application, the collection and processing of relevant data should strictly comply with the requirements of relevant laws and regulations, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of laws and regulations and the authorization of the personal information subject.

[0048] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0049] 1) In response to, used to indicate the conditions or states on which the operation performed depends. When the conditions or states on which it depends are met, one or more operations performed may be performed in real time or with a set delay. Unless otherwise specified, there is no restriction on the order in which the multiple operations are performed.

[0050] 2) A virtual scene is a scene displayed (or provided) by an application while it is running on a terminal device. This virtual scene can be a simulation of the real world, a semi-simulated / semi-fictional virtual environment, or a purely fictional virtual environment. A virtual scene can be any of a two-dimensional, 2.5-dimensional, or three-dimensional virtual scene; this application does not limit the dimension of the virtual scene. Taking a game scene as an example, a virtual scene can include the sky, land, ocean, etc., and the land can include environmental elements such as deserts and cities. Players can control their character to move within this virtual scene.

[0051] 3) Virtual objects refer to objects that do not actually exist within a virtual, simulated, or specific environment (virtual environment). This environment can be any form of scene, such as a virtual reality scene or a network scene. Virtual objects are used to provide interactive experiences in games, simulations, or other types of scenarios, such as game characters in a game scene.

[0052] 4) Layout information, used to represent the layout of virtual objects in the virtual environment. For example, it may include the virtual object's position in the virtual scene, movement route, activity range, faction, and map grid. Regarding movement routes, it may specifically include top, middle, and bottom routes.

[0053] 5) Feature information, which is used to represent the characteristics of virtual objects. For example, it can include the attributes of virtual objects and the state of the virtual environment around virtual objects. Attributes can include basic attributes such as the name of virtual objects, health in the game, and skills. The state can include the distribution of virtual objects' teammates, opposing faction game characters, or obstacles.

[0054] 6) Action set refers to the set of possible actions that a pre-defined virtual object can perform, such as crawling, jumping, and moving forward.

[0055] 7) Action strategy refers to at least one action to be performed by a virtual object in a virtual environment. These actions are designed to help the character achieve specific goals in the game, such as defeating enemies, completing tasks, or exploring the environment.

[0056] A game character's action strategy can include a series of actions to be performed. Here are some common examples of action strategies:

[0057] Attack strategy:

[0058] Action to be executed: Choose an appropriate time to launch an attack.

[0059] Strategy Details: Use specific skills or weapons based on the weaknesses of the opponent's characters, and arrange the attack order reasonably to maximize damage output.

[0060] Defense strategy:

[0061] Action to be performed: Defend when attacked.

[0062] Strategy details: Use shields or defensive skills to reduce damage, or maintain distance during the match to avoid the attack range of the opposing team's characters.

[0063] Evasion strategy:

[0064] Action to be performed: Dodge when the attack is about to hit.

[0065] Strategy details: Observe the attack patterns of the opponent's characters, predict the timing of their attacks, and quickly move to the side or jump to avoid them.

[0066] Resource management strategy:

[0067] Actions to be performed: Allocate and use resources reasonably (such as health, mana, items, etc.).

[0068] Strategy details: Use healing items wisely during the game, or conserve resources at crucial moments to deal with subsequent challenges.

[0069] Exploration strategy:

[0070] Action to be performed: Explore unknown areas in the game.

[0071] Strategy details: Find hidden paths, items, or secret levels to obtain additional virtual resources or information.

[0072] Teamwork strategies:

[0073] Action to be performed: Collaborate with teammates to complete the task.

[0074] Strategy details: Based on the abilities of teammates and the division of roles in the game, conduct effective cooperation and communication.

[0075] Environmental utilization strategies:

[0076] Actions to be performed: Use elements in the game environment to play a game or complete a task.

[0077] Strategy details: Use terrain cover for ambushes, or use special skills in specific environments.

[0078] 8) The first neural network model refers to the model used to execute the action decision method of the virtual object provided in the embodiments of this application, including a feature encoding model and multiple feature mapping models, wherein the multiple feature mapping models correspond to multiple layout information respectively, that is, the feature mapping module corresponds one-to-one with the layout information.

[0079] 9) Feature coding model (or feature coding module) refers to the model in the first neural network model used to extract features from feature information in order to obtain the features of the object.

[0080] 10) Feature mapping model (or feature mapping module) refers to the model in the first neural network used to perform feature mapping on the feature vector of an object in order to obtain the selection probability of each action in the action set.

[0081] 11) Artificial Intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0082] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, pre-trained model technology, operating / interactive systems, and mechatronics. Among these, pre-trained models, also known as large-scale models or foundational models, can be widely applied to downstream tasks across various AI fields after fine-tuning. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0083] Action decision-making methods for virtual objects in related technologies include schemes based on supervised learning (SL) and reinforcement learning (RL). These schemes encode the features of virtual objects and decode them using a pre-designed action space to obtain the action strategy of the virtual objects.

[0084] The action decision-making methods for virtual objects in related technologies have the following limitations:

[0085] 1) Different virtual objects share the action space for decoding output, which can be understood as different virtual objects sharing the same model parameters for inference to obtain the action policy of each virtual object. During training and inference, they are prone to mutual interference, affecting the training efficiency of the model and the prediction accuracy of the action policy.

[0086] 2) When a new virtual object is added, the model needs to be retrained, which is inefficient. Furthermore, the action decisions of the old virtual objects may be affected by the retraining, requiring the training parameters to be adjusted again, resulting in high training and integration costs.

[0087] This application provides a method, apparatus, device, computer-readable storage medium, and computer program product for making action decisions of virtual objects, which can accurately predict the action strategies of virtual objects. The following describes exemplary applications of the electronic devices provided in this application. The electronic devices provided in this application can be implemented as various types of terminals such as laptops, tablets, desktop computers, set-top boxes, smartphones, smart speakers, smartwatches, smart TVs, and vehicle terminals, or as servers.

[0088] See Figure 1 , Figure 1 This is a schematic diagram of the architecture of the virtual object action decision system provided in the embodiments of this application. Figure 1 The system involves server 100, terminal device 200, and network 300. Terminal device 200 is connected to server 100 through network 300, which can be a wide area network (WAN), a local area network (LAN), or a combination of both.

[0089] In some embodiments, the present application embodiments can be implemented collaboratively by a server and a terminal device. For example, the terminal device 200 sends the first layout information and feature information of a virtual object to the server 100. The server 100 receives the first layout information and feature information, processes the first layout information and feature information based on the virtual object action decision method provided in the present application embodiments, obtains the action strategy to be executed by the virtual object, and sends the action strategy to be executed by the virtual object to the terminal device 200.

[0090] In other embodiments, the embodiments of this application can be implemented by a terminal device alone. Terminal device 200 sends a model acquisition request to server 100. Server 100 receives the model acquisition request and sends a first neural network model for the action decision method for virtual objects provided in the embodiments of this application to terminal device 200. Terminal device 200 receives the first neural network model sent by the server and downloads it locally. It then processes the first layout information and feature information of the virtual object using the action decision method for virtual objects provided in the embodiments of this application to obtain the action strategy to be executed by the virtual object.

[0091] Here, server 100 can be a single server. In this case, the action decision method for virtual objects and the training of the first neural network model provided in this application embodiment can be implemented by the same server. Server 100 can also be a cluster of servers. In the case that server 100 is a server cluster, the action decision method for virtual objects and the training of the first neural network model provided in this application embodiment can be implemented by different servers. This application embodiment does not impose any limitations on this.

[0092] See Figure 2 , Figure 2 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Figure 2 The electronic device 400 shown can be either the server 100 or the terminal device 200 mentioned above. Figure 2 The illustrated electronic device 400 includes at least one processor 410, a memory 430, and at least one network interface 420. The various components of the electronic device 400 are coupled together via a bus system 440. It is understood that the bus system 440 is used to implement communication between these components. In addition to a data bus, the bus system 440 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 2 The general labeled all buses as Bus System 440.

[0093] The processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0094] The memory 430 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard disk drives, optical disk drives, etc. The memory 430 may optionally include one or more storage devices physically located away from the processor 410.

[0095] The memory 430 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 430 described in this application embodiment is intended to include any suitable type of memory.

[0096] In some embodiments, memory 430 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, as illustrated below.

[0097] Operating system 431 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, driver layer, etc., for implementing various basic business functions and handling hardware-based tasks;

[0098] The network communication module 432 is used to reach other electronic devices via one or more (wired or wireless) network interfaces 420, exemplary network interfaces 420 including: Bluetooth, WiFi, and Universal Serial Bus (USB), etc.

[0099] In some embodiments, the apparatus provided in this application can be implemented in software. Figure 2 An action decision-making device 433 for a virtual object stored in memory 430 is shown. This device can be software in the form of programs or plug-ins, and includes the following software modules: a data acquisition module 4331 and an action prediction module 4332. These modules are logically linked and can therefore be arbitrarily combined or further separated according to their implemented functions. The functions of each module will be described below.

[0100] In some embodiments, the terminal device or server can implement the action decision-making method for virtual objects provided in this application by running various computer-executable instructions or computer programs. For example, computer-executable instructions can be microprogram-level commands, machine instructions, or software instructions. Computer programs can be native programs or software modules in an operating system; they can be native applications (APPs), i.e., programs that need to be installed in the operating system to run; or they can be applets that can be embedded in any APP, i.e., programs that only need to be downloaded to a browser environment to run. In summary, the aforementioned computer-executable instructions can be any form of instruction, and the aforementioned computer programs can be any form of application, module, or plugin.

[0101] In other embodiments, the apparatus provided in this application can be implemented in hardware. As an example, the apparatus provided in this application can be a processor in the form of a hardware decoding processor, which is programmed to execute the action decision method of the virtual object provided in this application. For example, the processor in the form of a hardware decoding processor can be one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0102] The following will describe the action decision-making method for virtual objects provided in this application embodiment, with the server as the execution subject, using exemplary applications and implementations of the server provided in the embodiments of this application. See also Figure 3A , Figure 3A This is a first flowchart illustrating the action decision-making method for virtual objects provided in this application embodiment, which will be combined with... Figure 3A The steps shown are explained.

[0103] In step 101, the first layout information and feature information of the virtual objects in the virtual scene are obtained.

[0104] In some embodiments, the first layout information includes any one of the following: the position of the virtual object in the virtual scene, the movement route of the virtual object in the virtual scene, the faction to which the virtual object belongs in the virtual scene, and the map grid of the virtual object in the virtual scene.

[0105] For example, the position of a virtual object in a virtual scene refers to its specific coordinates or location information, which can be represented by three-dimensional coordinates (x, y, z); the movement path of a virtual object in a virtual scene refers to the path the virtual object moves along in the virtual scene, which can be pre-set for the virtual object or selected when the virtual object enters the virtual scene; the faction to which a virtual object belongs in a virtual scene refers to the faction to which the object may belong, such as friendly, opposing, or neutral; and the map grid of a virtual object in a virtual scene refers to the map grid where the virtual object's activity range or location is located.

[0106] Here, the first layout information of a virtual object can be understood as the category label of the virtual object. For example, virtual objects with the same action route or located in the same map grid belong to the same category (that is, these virtual objects have the same first layout information). By obtaining the first layout information of a virtual object, the category of the current virtual object can be determined, and targeted probability prediction processing can be performed on the virtual objects of the current category (for specific implementation, please refer to the description of steps 1021 to 1022 below) to obtain the action selection probability of the virtual object carrying the first layout information, thereby realizing diversified probability prediction processing for virtual objects with different layout information, and achieving the beneficial effect of improving the accuracy of probability prediction.

[0107] Taking a game scene as an example, the position of a virtual object in a virtual scene can be the position of a game character (virtual object) on the game map, the movement route can be the lane to which the game character belongs (such as top lane, middle lane, bottom lane, etc.), the faction to which the virtual object belongs in the virtual scene can be the faction of the team to which the game character belongs (such as the red team and the blue team), and the map grid of the virtual object in the virtual scene can be the map grid in which the game character's activity range is located after the game map is divided into multiple grids.

[0108] In some embodiments, feature information is used to represent the attributes of the virtual object itself and the state of the virtual environment surrounding the virtual object. The attributes of the virtual object itself may include category information and attribute information, and the state of the virtual environment surrounding the virtual object may be represented by image information in the feature information.

[0109] Taking a game scene as an example, the attributes of a virtual object include the game character's name, class, etc. (corresponding category information); health, skills, movement direction, etc. in the game (corresponding attribute information); and the distribution of teammates, opposing team characters, or obstacles around the game character in the game (corresponding to the state of the virtual environment around the virtual object). Feature information can be obtained from game logs or game image frames automatically recorded by the game engine or specialized recording software when the user plays the game.

[0110] By acquiring the characteristic information of virtual objects, we obtain relevant information about the virtual object's own state (the attributes of the virtual object itself) and the state of the virtual environment (the state of the virtual environment surrounding the virtual object), which provides a basis for the virtual object to make action decisions in the current state.

[0111] In step 102, based on the first layout information and feature information, probability prediction processing corresponding to the first layout information is performed to obtain the selection probability of each action in the preset first action set.

[0112] In some embodiments, the first action set is generated by obtaining a pre-set first action set, wherein the first action set is specifically set for the first layout information, or the first action set is uniformly set for different layout information, and each action in the first action set includes at least one of the following attributes: action event, action direction, action end point, and target object.

[0113] Taking a game scene as an example, an action event can be an action of a game character standing, moving, releasing a skill, returning to town, or purchasing equipment. The direction of the action can be the direction the game character is moving towards or the direction the skill is released towards. The landing point of the action can be the landing point of the skill after it is released. The target of the action can be the target of the skill, such as the game characters, monsters, or defensive towers of the opposing faction.

[0114] By pre-setting the action space (first action set) of virtual objects, the action strategies that virtual objects need to consider are clarified, so that virtual objects will not execute meaningless action strategies and the computational complexity is reduced, achieving the beneficial effect of clarifying the scope of action decisions.

[0115] In some embodiments, see Figure 3B The first set of actions can be generated through the following steps 201 to 206, which are explained in detail below.

[0116] In step 201, interaction record data is acquired, wherein the interaction record data is used to record the interactions of the virtual object under the control of the real object.

[0117] In some embodiments, interaction record data of a virtual object under the control of a real object (user) is acquired, and the interaction record data includes interaction information between the real object and the virtual environment.

[0118] Taking game scenarios as an example, interaction recording data can be game screen recording data, game replay data, or game log data obtained after the user operates the game character (virtual object) to play the game. It can completely capture the user's operation sequence (such as clicking the control to release skills, controlling the game character to move, etc.), changes in the game screen, etc. Through the game engine or dedicated recording software, interaction recording data can be automatically recorded when the user plays the game. Alternatively, the game's built-in replay function can be used to automatically save the user's game process.

[0119] In step 202, at least one action event is detected from the interaction record data, wherein the at least one action event is performed by a virtual object.

[0120] In some embodiments, keyframes or key event logs are extracted from the interaction recording data, and specific action events are identified from the keyframes or key event logs using image recognition, pattern recognition, or natural language processing techniques.

[0121] Taking a game scene as an example, character movement can be identified by analyzing pixel changes in keyframes; skill release events can be identified by analyzing text information in game logs. For instance, assuming a game recording is obtained, the following action events can be detected using image recognition technology:

[0122] Event 1: The character begins to move at 00:12:34 and lasts for 5 seconds.

[0123] Event 2: The character used the level 1 skill "Fireball" at 00:13:45.

[0124] Event 3: The character returns to town at 00:15:00, lasting for 10 seconds.

[0125] Event 4: The character purchased the equipment "Armor" at 00:16:20.

[0126] The identified action events can be labeled, such as recording the event name (e.g., standing, moving, releasing a skill, returning to town, purchasing equipment, etc.), the time of occurrence, and the duration, and the detected action events can be stored in a database or file for subsequent analysis and use.

[0127] In step 203, the position sequence of the virtual object when it performs the action event is obtained, and the action direction corresponding to the action event is determined based on the position sequence.

[0128] In some embodiments, a position sequence refers to a record of a series of position points traversed by a virtual object during the execution of an action event. These position points contain coordinate information, such as x, y (and z) values ​​in two-dimensional or three-dimensional space. The coordinate information of the position points can be obtained through techniques such as image recognition, or from log files or databases. The direction of the virtual object's action is determined by comparing the displacement vectors between consecutive position points; for example, if the displacement vector points to the right, the direction of the action is "to the right." In more complex scenarios, the overall trend of the virtual object's action direction can be considered, rather than just a single displacement vector; for example, the overall action direction can be obtained by smoothing the position sequence.

[0129] Taking a game scene as an example, the game character controlled by the player moves on the game map (virtual scene) and performs a series of action events. By recording and analyzing the position sequence of these action events, the direction of the character's movement can be determined. For example, the position sequence corresponding to the movement of the game character can be represented as: [(x1, y1), (x2, y2), ..., (xn, yn)], and the displacement vector can be represented as: [(x2-x1, y2-y1), (x3-x2, y3-y2), ..., (xn-x(n-1), yn-y(n-1))]. Thus, based on the direction of the displacement vector, it can be determined whether the character is moving left, right, up, or down.

[0130] In step 204, the distance parameters pre-associated with the action event are queried, and the action end point of the action event is determined based on the distance parameters and the action direction.

[0131] In some embodiments, the distance parameter refers to a predefined distance value associated with a specific action event, such as the radius of the skill release range in a game scene.

[0132] Taking a game scene as an example, distance parameters are determined during the game design phase and stored in the game's data structure or configuration file. Each action event is associated with one or more distance parameters. For example, a ranged skill can correspond to a specific skill distance. The landing point of an action event can be calculated using the action direction and distance parameters. For example, if the character is facing east and the radius of the skill's range is 5 units, the landing point will be 5 units east of the character. When determining the landing point, environmental factors such as obstacles and terrain can also be considered. These factors will affect the landing point of the action event. For example, if the character is facing east and the radius of the skill's range is 5 units, but there is an obstacle 3 units away in that direction, the landing point will be 3 units east of the character, meaning the obstacle blocks the released skill.

[0133] In step 205, target identification is performed on the interaction record data to obtain the target object.

[0134] In some embodiments, video frames containing visual information are extracted from the interaction recording. Target recognition is then performed on the video frames to obtain the target object. For example, image processing techniques are used to extract key features from the frames, such as color, shape, and texture. Target recognition algorithms, such as Convolutional Neural Networks (CNNs) or other machine learning models, are then applied to the extracted features to identify and classify the objects in the frames. Here, the target object refers to the object that the virtual object refers to when performing the action, i.e., the object to which the action is applied.

[0135] Taking a game scene as an example, a series of frames in the game screen recording (interactive record data) record the player character's game process. Through image recognition technology, the system identifies the player character, enemy monsters, and defense towers in the frames. The player character's skill release action is directed at the enemy monsters, so the enemy monsters are the target of the skill release action.

[0136] In step 206, based on the action event, action direction, action end point, and target, a first action set corresponding to the first layout information is obtained.

[0137] In some implementation examples, information such as action events, action directions, action ending points, and target objects are integrated to obtain the first action set corresponding to the layout information. Taking a game scene as an example, the first action set can be represented as: {Action 1 (Release skill, East, 5 units away from the game character, monster 1), Action 2 (Return to town, game starting point), Action 3 (Stand)}.

[0138] Through steps 201 to 206, a first action set corresponding to the first layout information is constructed based on the interactive record data. By introducing technologies such as image recognition, pattern recognition, and natural language processing, the complexity of manual annotation and action set construction is reduced, and the speed and efficiency of interactive record data processing are improved to cope with the processing of large-scale interactive record data, reduce the possibility of human error, and thus improve the reliability of the first action set.

[0139] In some embodiments, the action decision method for virtual objects provided in this application is implemented through a first neural network model. The first neural network model includes a feature encoding model and multiple feature mapping models. The multiple feature mapping models correspond to multiple layout information respectively. That is, the feature mapping models and layout information are in one-to-one correspondence, and the multiple layout information includes the first layout information. Each feature mapping model is trained based on training data related to the corresponding layout information.

[0140] In some embodiments, see Figure 3C , Figure 3A Step 102 shown can be implemented through steps 1021 to 1022, which will be explained in detail below.

[0141] In step 1021, the feature information is encoded using a feature encoding model to obtain the object feature vector.

[0142] In some embodiments, feature encoding is implemented through a feature encoding model in a first neural network model. The embodiments of this application do not limit the specific network structure of the feature encoding model.

[0143] In some embodiments, see Figure 3D , Figure 3B Step 1021 shown can be implemented through steps 10211 to 10212, which will be explained in detail below.

[0144] In step 10211, the feature information is processed by data conversion to obtain numerical information.

[0145] In some embodiments, the feature information is processed using a feature encoding model using any of the following methods:

[0146] The system performs one-hot encoding on the category information in the feature information; embedding encoding on the attribute information in the feature information; network partitioning on the image information in the feature information to obtain multiple image grids, and binarization on each image grid; multiple rays are created using virtual objects in the image information as ray endpoints according to the preset number and direction of rays, and elements in the ray direction are detected through each ray to obtain the detection result, and the detection result in each ray direction is numerically processed.

[0147] Taking a game scene as an example, the category information in the feature information can include the game character's profession, gender, level, preset character path, etc. One-hot encoding of the category information in the feature information can be achieved in the following way: For each category, create a new binary feature matrix, where each category value corresponds to a new column. Each row in the matrix represents a sample. If the sample belongs to a certain category, the corresponding column value is 1, and the other column values ​​are 0. For example, the data obtained after one-hot encoding of the gender category can be represented as: male [1, 0] and female [0, 1].

[0148] Taking a game scene as an example, the attribute information in the feature information can include the game character's health, skills, defense, etc. Embedding encoding of the attribute information in the feature information can be achieved in the following way: Create an embedding space for each attribute. The size of this space depends on the number of attributes and the embedding dimension. The dimension of the embedding space can be preset based on experience or model requirements. Create a matrix whose size is the number of attributes multiplied by the embedding dimension. For example, if there are 3 skills (such as Fireball, Freeze, and Heal) and a 10-dimensional embedding space, the matrix size will be 3×10. Randomly initialize an embedding for each attribute. The embedding vectors, or pre-trained embedding vectors, can be used for embedding encoding. The skill embedding matrix obtained after embedding encoding can be represented as: {[0.1, 0.2, 0.3, ..., 0.1], [0.2, 0.3, 0.4, ..., 0.2], [0.3, 0.4, 0.5, ..., 0.3]}, where [0.1, 0.2, 0.3, ..., 0.1] corresponds to the embedding vector of Fireball, [0.2, 0.3, 0.4, ..., 0.2] corresponds to the embedding vector of Freeze, and [0.3, 0.4, 0.5, ..., 0.3] corresponds to the embedding vector of Heal.

[0149] Taking a game scene as an example, the image information in the feature information is divided into multiple image grids, and each image grid is binarized. This can be achieved in the following way: the game screen is divided into grids of a fixed size (or a square area with a length h and a width h centered on the location of the game character can be selected, and the square area can be divided into grids of size h / w with a length w; this application embodiment does not impose any restrictions). Each grid represents a region of the game map, which can be a square or a rectangle. Here, the size of the grid can be determined according to the complexity of the game scene and the required accuracy of the analysis. The smaller the grid, the higher the accuracy, but the greater the computational load. After the game screen is evenly divided into multiple grids, a unique identifier is set for each grid. For each grid, binarization is performed according to specific conditions, that is, the value of the grid is set to 0 or 1. For example, if an obstacle exists in the grid, the skill release range covers the grid, the grass is located in the grid, or a teammate or enemy character is located in the grid, then the value of the grid is set to 1. In this way, each grid is represented as a binary value, and the corresponding game screen is represented as a binary matrix.

[0150] Taking a game scene as an example, using virtual objects in the image information as ray endpoints, multiple rays are created according to a pre-set number and direction. Elements along the ray direction are detected by each ray to obtain the detection results. The detection results for each ray direction are then numerically processed. This can be achieved in the following way: using the position of the virtual object as the starting point of the ray, the number of rays and the direction of each ray are pre-set. The direction of the ray can be based on the hero's orientation, movement direction, or attack direction. Rays are emitted from the endpoints according to the set directions. Elements on the rays are detected, for example, by checking the collision between the ray and elements in the game world (such as obstacles, defensive towers, etc.). For each ray, if there is an element in the ray direction, the numerical result of the ray is set to 1; if there is no element, it is set to 0. Alternatively, the numerical result can be determined based on the number of elements in the ray direction. For example, if there are 3 elements in the ray direction, the numerical result of the ray is set to 3.

[0151] In other embodiments, an identifier of incomplete information in the feature information (or information contained in the feature information) is obtained. When the identifier indicates that the feature information is incomplete information (information is not visible), no data conversion processing is performed on the feature information. Incomplete information refers to the state information of an object that is beyond the perception range of the virtual object (such as beyond the field of view) or cannot be perceived.

[0152] For example, an "isvisible" field can be added to the feature information as an identifier for incomplete information. When the "isvisible" attribute is set to true, it means that the feature information is invisible and incomplete, and no data transformation processing will be performed on the feature information. That is, incomplete information will not participate in the subsequent probability prediction processing.

[0153] Taking a game scenario as an example, in a game match, the game character cannot perceive the specific location, attributes, behavior, etc. of the opposing team's game characters that are not within the field of vision. By marking the incomplete information in the feature information, the field of vision of the real player in the game client can be simulated.

[0154] By converting feature information into numerical information (data in numerical form), a more suitable data format is provided for the inference of feature coding models, which facilitates mathematical modeling and analysis of feature coding models.

[0155] In step 10212, feature extraction processing is performed on the numerical information to obtain the object feature vector.

[0156] In some embodiments, see Figure 3E , Figure 3C Step 10212 shown can be implemented through steps 10213 to 10215, which will be explained in detail below.

[0157] In step 10213, the numerical information is normalized to obtain normalized information.

[0158] In some embodiments, a target range for normalization is determined, such as [0, 1]. For each numerical feature in the numerical information, the original value is normalized to the specified range using a normalization formula. For example, the normalization formula can be expressed as: Normalized value = Original value - Minimum value / Maximum value - Minimum value. By normalizing the numerical information, the numerical range of all numerical information is unified, which helps the input layer of the feature encoding model to better process the data.

[0159] In step 10214, the normalized information is subjected to a linear transformation to obtain linear features.

[0160] In some embodiments, the normalized information is linearly transformed using the weight parameters of a pre-trained feature encoding model to obtain linear features, which can be represented by, for example, formula (1):

[0161] Y = W*X + b (1)

[0162] Where W is the weight parameter from the input layer to the hidden layer (e.g., ... Figure 5C The feature encoding model shown in the figure has an input layer and a hidden layer. X represents normalized information, Y represents linear features, and b is a bias parameter. By weighting the normalized information with the weight parameter W, the feature importance of the normalized information is adjusted, thereby improving the generalization ability of the feature encoding model.

[0163] In step 10215, the linear features are subjected to nonlinear transformation to obtain the object feature vector.

[0164] In some embodiments, linear features are subjected to nonlinear transformation using a pre-set activation function (e.g., the tanh function) to obtain the object feature vector. Nonlinear transformation can capture nonlinear relationships in the data, enabling the feature encoding model to better fit complex data distributions and improve its generalization ability on unseen data, thereby reducing the risk of overfitting.

[0165] See also Figure 3C In step 1022, the feature vector of the object is mapped by the feature mapping model corresponding to the first layout information to obtain the selection probability of each action in the first action set.

[0166] In some embodiments, when multiple feature mapping models share the same input layer, the object feature vector is input into the input layer to perform feature mapping through multiple feature mapping models respectively, thereby obtaining the probability distributions corresponding to multiple layout information respectively, and the probability distribution corresponding to the second layout information in the probability distributions corresponding to the multiple layout information is masked to obtain the probability distribution corresponding to the first layout information. Here, the second layout information is the layout information that is different from the first layout information among the multiple layout information, and the probability distribution corresponding to the first layout information includes the selection probability of each action in the first action set.

[0167] By using different feature mapping models to perform feature mapping on virtual objects with different layout information, the unique layout and feature information of each virtual object is taken into account, thereby providing more targeted information for probability prediction, so that the predicted action strategy can be adapted to the layout and feature information of the virtual object.

[0168] For example, see Figure 5A When multiple feature mapping models share the same input layer, the object feature vector is input into the input layer, and feature mapping is performed by multiple feature mapping models (each feature mapping model includes an input layer, an intermediate layer and an output layer, and multiple feature mapping models only share the input layer) to obtain the probability distributions corresponding to multiple layout information (the feature mapping model and the layout information correspond one-to-one). By performing the dot product (bitwise multiplication) of the probability distributions corresponding to the multiple layout information with the layout information mask, the probability distribution corresponding to the first layout information is obtained.

[0169] For example, feature mapping can be implemented using fully connected layers (FC). The dimension of the fully connected layer is set to the number of actions in the first action set. The probability distribution of each action is output through the Softmax function, thereby mapping the object feature vector to the selection probability of each action in the first action set.

[0170] In some embodiments, when the first action set is specifically set for the first layout information, the probability distribution corresponding to the first layout information includes the execution probability of each action in the first action set, and the probability distribution corresponding to the second layout information (layout information that is different from the first layout information among multiple layout information) includes the execution probability of each action in the second action set (action set that is different from the first action set among multiple action sets); when the first action set is an action set uniformly set for multiple layout information (including the first layout information), the probability distribution corresponding to the first layout information includes the execution probability of each action in the first action set, and the probability distribution corresponding to the second layout information (layout information that is different from the first layout information among multiple layout information) includes the execution probability of each action in the first action set.

[0171] For example, see Figure 5C The input layer in Figure 5C The diagram illustrates the network framework of the first neural network model during the training phase (the forward pass during the inference phase is the same as in the training phase). Different feature vectors are input to multiple feature mapping models via the same input layer. Each feature mapping model performs feature mapping to obtain probability distributions corresponding to multiple layout information. By masking the probability distribution corresponding to the second layout information among the probability distributions corresponding to the multiple layout information, the probability distribution corresponding to the first layout information is obtained. For example, after the object feature vector -1 corresponding to layout information -1 is input to the input layer, it is subjected to feature mapping by multiple feature mapping models such as feature mapping model -1, feature mapping model -2, and feature mapping model -N to obtain probability distributions corresponding to multiple layout information (the probability of each action in the action set corresponding to the layout information). By performing a dot product (bitwise multiplication) between the probability distributions corresponding to the multiple layout information and the layout information mask, the probability distribution corresponding to the first layout information is obtained. For example, the probability distributions corresponding to the multiple layout information are the probability distributions corresponding to layout information -1 to layout information -N respectively. The layout information mask is represented as [1,0,...,0]. After performing a dot product with the probability distributions corresponding to the multiple layout information, the probability distribution corresponding to the first layout information (layout information -1) is output.

[0172] In some embodiments, when multiple feature mapping models each have an independent input layer, the object feature vector is input into the input layer of the feature mapping model corresponding to the first layout information, so as to perform feature mapping through the feature mapping model corresponding to the first layout information and obtain the probability distribution corresponding to the first layout information.

[0173] For example, see Figure 5B When multiple feature mapping models each have an independent input layer, the object feature vector is input into the input layer-1 of the feature mapping model corresponding to the first layout information, so that feature mapping is performed through the intermediate layer-1 of the feature mapping model-1 corresponding to the first layout information, and the probability distribution corresponding to the first layout information is output through the output layer-1.

[0174] For example, see Figure 5D The probability prediction stage shown in the figure has multiple input layers (input layer-1, input layer-2 to input layer-N). Each feature mapping model has a corresponding input layer. After obtaining the object feature vector input by the input layer, feature mapping is performed through the corresponding feature mapping model. For example, input layer-1 inputs the object feature vector-1 into feature mapping model-1 to obtain the selection probability of each action in action-several-1 corresponding to layout information-1 (first layout information).

[0175] See also Figure 3A In step 103, based on the selection probability of each action, at least one action is selected from the first action set as the action strategy to be executed by the virtual object.

[0176] In some embodiments, the action with the highest probability is selected as the action strategy to be executed, or the top N actions with the highest probability are selected as the action strategy to be executed for the virtual object.

[0177] Taking a game scene as an example, when there are multiple action strategies to be executed, the action strategy to be executed by the virtual object can be represented as (move, stand, return to town). That is, the game character first executes the movement action, then executes the stand action after moving to the target position, and finally executes the return to town action.

[0178] Through steps 101 to 103, probability prediction processing corresponding to the first layout information is achieved based on the first layout information and feature information of the virtual object. Differentiated probability prediction processing is achieved for virtual objects with different layout information. By considering the unique layout information and feature information of each virtual object, more targeted information can be used for probability prediction, so that the predicted action strategy can be adapted to the layout information and feature information of the virtual object. This not only ensures the accuracy of the action strategy, but also effectively realizes the diversification of action strategies for different virtual objects, thereby better adapting to multi-virtual object systems.

[0179] In some embodiments, see Figure 3F After controlling the virtual object to execute the action strategy to be executed, the following steps 104 to 105 can also be executed, which are explained in detail below.

[0180] In step 104, the decision performance of the first neural network model is tested to obtain the first decision performance parameters.

[0181] In some embodiments, see Figure 3G , Figure 3F Step 104 shown can be implemented through steps 1041 to 1042, as explained in detail below.

[0182] In step 1041, the prediction action strategy of the virtual object is determined by the first neural network model, and the virtual object is controlled to execute the prediction action strategy in the virtual scene.

[0183] In some embodiments, the actions of virtual objects in a virtual scene are controlled according to a predicted action strategy. The method for determining the predicted action strategy of virtual objects can be found in the description of steps 101 to 103 above, and will not be repeated here.

[0184] In step 1042, a first prediction performance parameter is determined based on the interaction results achieved by the virtual object executing the prediction action strategy. The first prediction performance parameter includes at least one of the following: the win rate of the virtual object, and an anthropomorphic index, wherein the anthropomorphic index is used to characterize the degree to which the actions of the virtual object match those of the real object.

[0185] Using a game scenario as an example, 10 game characters are randomly selected to form a match lineup. Here, it is ensured that the action strategy of one game character (virtual object) is controlled by the first neural network model, while the action strategies of the remaining game characters can be controlled by the player or by the first neural network model. By having the 10 game characters play against each other to generate match data (interaction results), the match data is analyzed to evaluate the win rate (average win rate) and anthropomorphic indicators (such as the number of defeats, the number of times defeated, the number of times returning to base, the number of times key items are obtained, the number of times excellent operations are performed, etc.) of the game character controlled by the first neural network model. 3000 matches can be played to ensure the reliability of the data.

[0186] Steps 1041 to 1042 are used to obtain the first decision performance parameters of the first neural network model. This allows for a more accurate evaluation of the predictive performance of the action strategy of the first neural network model based on the quantified first decision performance parameters (such as win rate and anthropomorphic indicators), so as to optimize the first neural network model in the future.

[0187] See also Figure 3F In step 105, in response to the first decision performance parameter being less than a preset decision performance parameter threshold, the first neural network model is updated, wherein the updated first neural network model is used to replace the unupdated first neural network model for probability prediction.

[0188] Taking a game scenario as an example, the pre-set decision performance parameter thresholds may include at least one of the following: win rate threshold, anthropomorphic indicator threshold. For example, the win rate threshold may be 65%, and the anthropomorphic indicator threshold may be an average number of defeats per game greater than or equal to 6, an average number of defeats per game less than or equal to 5, etc. The embodiments of this application do not limit the specific win rate threshold and anthropomorphic indicator threshold.

[0189] In some embodiments, the action decision method for virtual objects provided in this application is implemented by a first neural network model. The first neural network model includes a feature encoding model and multiple feature mapping models. The multiple feature mapping models correspond to multiple layout information (one-to-one correspondence), and the multiple layout information includes first layout information. Each feature mapping model is trained based on training data related to the corresponding layout information.

[0190] In some embodiments, see Figure 3H , Figure 3FThe update of the first neural network model in step 105 shown can be achieved through the following steps 301 to 307, which are explained in detail below.

[0191] In step 301, an action set sample is created, wherein the action set sample includes multiple action samples adapted to the virtual object.

[0192] In some embodiments, the action set sample may be the same as the first action set sample, or actions may be added or removed based on the first action set to better adapt to the virtual object.

[0193] In step 302, a sample of feature information of the virtual object under the control of the real object is obtained.

[0194] In some embodiments, feature information samples of virtual objects under the control of real objects (users) are obtained. For specific implementation methods of obtaining feature information samples, please refer to the description of step 101 above, which will not be repeated here.

[0195] In step 303, a feature mapping model corresponding to the virtual object is added to the first neural network model to obtain a second neural network model.

[0196] In some embodiments, see Figure 5E In the probability prediction stage, add a feature mapping model corresponding to the virtual object (e.g., Figure 5E The second neural network model is obtained by using the feature mapping model corresponding to virtual object-1 (N+1).

[0197] In step 304, the second neural network model is used to perform probability prediction based on feature information samples to obtain the selection probability of each action sample in the action set sample.

[0198] In some embodiments, the feature information sample is feature-encoded by the feature encoding model in the second neural network model to obtain the object feature vector (see the description of step 1021 above); the object feature vector is feature-mapped by the feature mapping model corresponding to the virtual object to obtain the selection probability of each action in the action set sample (see the description of step 1022 above).

[0199] In some embodiments, feature encoding is performed on feature information samples through the feature encoding model in the second neural network model to obtain object feature vectors. This can be achieved by: performing data conversion processing on feature information samples to obtain numerical information samples (see the description of step 10211 above); and performing feature extraction processing on numerical information samples to obtain object feature vectors (see the description of step 10212 above).

[0200] In some embodiments, when multiple feature mapping models share the same input layer, the object feature vector is input into the input layer to perform feature mapping through multiple feature mapping models respectively, thereby obtaining multiple layout information and probability distributions corresponding to the feature mapping models corresponding to the virtual objects respectively, and masking processing is performed on the layout information and / or probability distributions corresponding to other virtual objects in the multiple probability distributions to obtain the probability distribution corresponding to the virtual objects.

[0201] For example, see Figure 5E After the object feature vector of virtual object-1 is input into the input layer, it is used for feature mapping through multiple feature mapping models such as feature mapping model-1, feature mapping model-N+1 (a feature mapping model added for virtual object-1), and feature mapping model-N+a to obtain multiple probability distributions. By masking the layout information and the probability distributions corresponding to other virtual objects in the multiple probability distributions, the probability distribution corresponding to the virtual object is obtained. For example, the multiple probability distributions are the probability distributions corresponding to layout information-1 to layout information-N and virtual object-1 to virtual object-a respectively. The object identifier mask is represented as [0, ..., 1, ..., 0]. After performing a dot product with the multiple probability distributions, the probability distribution corresponding to virtual object-1 is output.

[0202] In some embodiments, when multiple feature mapping models each have an independent input layer, the object feature vector of the virtual object is input into the input layer of the added feature mapping model corresponding to the virtual object, so as to perform feature mapping through the added feature mapping model corresponding to the virtual object and obtain the probability distribution corresponding to the virtual object.

[0203] In step 305, based on the selection probability of each action sample, at least one action sample is selected from the action set sample as an action strategy sample to be executed by the virtual object.

[0204] In some embodiments, the action sample with the highest probability is selected as the action strategy sample to be executed, or the top N action samples with the highest probability are selected as the action strategy samples to be executed for the virtual object.

[0205] In step 306, a first loss value is determined based on the action policy samples and the preset labeled action policy.

[0206] In some embodiments, a first loss value is calculated between the action policy sample and the preset labeled action policy (the action performed by the real object) according to a preset loss function (e.g., cross-entropy loss function).

[0207] In step 307, based on the first loss value, the parameters of the feature mapping model corresponding to the virtual object are updated to obtain the updated second neural network model.

[0208] In some embodiments, the gradient of the loss function with respect to the parameters of the second neural network model is calculated based on the first loss value using the back propagation (BP) algorithm. The model parameters of multiple network layers of the second neural network model are updated according to the direction of the gradient using gradient descent methods (such as batch gradient descent (BGD), stochastic gradient descent (SGD), etc.) to obtain the updated second neural network model.

[0209] Through steps 301 to 307, when the first decision performance parameter is less than the preset decision performance parameter threshold, feature information samples of the virtual object under the control of the real object are obtained, thereby updating the first neural network model. While saving training costs (because there is no need to train the model from scratch), the updated second neural network model can better generalize to the new dataset (feature information samples), thus enhancing the generalization ability of the second neural network model.

[0210] In some embodiments, see Figure 3I , Figure 3F The update of the first neural network model in step 105 shown can be achieved through the following steps 401 to 405, which are explained in detail below.

[0211] In step 401, a sample of feature information of the virtual object under the control of the real object is obtained.

[0212] In some embodiments, feature information samples of virtual objects under the control of real objects (users) are obtained. For specific implementation methods of obtaining feature information samples, please refer to the description of step 101 above, which will not be repeated here.

[0213] In step 402, the first neural network model is used to perform probability prediction based on feature information samples to obtain the selection probability of each action sample in the action set sample.

[0214] In some embodiments, feature information samples are feature-encoded using a feature encoding model in the first neural network model to obtain object feature vectors (see the description of step 1021 above); the object feature vectors are feature-mapped using a feature mapping model corresponding to the first layout information to obtain the selection probability of each action sample in the action set sample (see the description of step 1022 above). Here, the action set sample can be the same as the first action set sample, or actions can be added or reduced based on the first action set to better adapt to the virtual object.

[0215] In some embodiments, when multiple feature mapping models share the same input layer, the object feature vector is input into the input layer to perform feature mapping through multiple feature mapping models respectively, thereby obtaining the probability distributions corresponding to multiple layout information. The probability distribution corresponding to the second layout information in the probability distributions corresponding to the multiple layout information is masked to obtain the probability distribution corresponding to the first layout information. The second layout information is the layout information that is different from the first layout information among the multiple layout information. The probability distribution corresponding to the first layout information includes the selection probability of each action in the action set sample.

[0216] In some embodiments, when multiple feature mapping models each have an independent input layer, the object feature vector is input into the input layer of the feature mapping model corresponding to the first layout information, so as to perform feature mapping through the feature mapping model corresponding to the first layout information and obtain the probability distribution corresponding to the first layout information.

[0217] In step 403, based on the selection probability of each action sample, at least one action sample is selected from the action set sample as an action strategy sample to be executed by the virtual object.

[0218] In some embodiments, the action sample with the highest probability is selected as the action strategy sample to be executed, or the top N action samples with the highest probability are selected as the action strategy samples to be executed for the virtual object, where N is an integer greater than or equal to 2.

[0219] In step 404, a second loss value is determined based on the action policy samples and the preset labeled action policy.

[0220] In some embodiments, a second loss value is calculated between the action policy sample and the preset labeled action policy (the action performed by the real object) according to a preset loss function (e.g., cross-entropy loss function).

[0221] In step 405, based on the second loss value, the parameters of the feature mapping model corresponding to the first layout information are updated to obtain the updated first neural network model.

[0222] In some embodiments, see Figure 3J , Figure 3I Step 405 shown can be implemented through steps 4051 to 4053, which will be explained in detail below.

[0223] In step 4051, a first low-rank matrix initialized based on a Gaussian distribution is generated, and a second low-rank matrix initialized based on all zeros is generated.

[0224] In some embodiments, random numbers are generated using a Gaussian distribution (normal distribution) to initialize the first low-rank matrix. The Gaussian distribution is defined by the mean and standard deviation (STD) parameters. All elements of the second low-rank matrix are initialized to zero, wherein the dimension of the second low-rank matrix is ​​the same as that of the transpose of the first low-rank matrix.

[0225] In step 4052, fine-tuning weight parameters are constructed based on the first low-rank matrix and the second low-rank matrix, and fine-tuning weight parameters are added to the parameters of the feature mapping model corresponding to the first layout information.

[0226] In some embodiments, fine-tuning weight parameters are constructed based on the product of the first low-rank matrix and the second low-rank matrix, see [link to relevant documentation]. Figure 6 , Figure 6 This is a schematic diagram illustrating the construction principle of the fine-tuning weight parameters provided in the embodiments of this application, as shown below. Figure 6 As shown, the product of the first low-rank matrix (A) and the second low-rank matrix (B) constitutes the fine-tuning weight parameter (ΔW), wherein the dimension of the second low-rank matrix is ​​the same as that of the transpose of the first low-rank matrix.

[0227] In step 4053, the fine-tuned weight parameters are updated using the second loss value to obtain the updated first neural network model.

[0228] In some embodiments, the original parameters of the feature mapping model corresponding to the first layout information are kept unchanged except for the first low-rank matrix and the second low-rank matrix, and the first low-rank matrix and the second low-rank matrix are updated by the second loss value.

[0229] For example, the original parameters (weight parameters) of the feature mapping model corresponding to the first layout information are denoted as W. A fine-tuning weight parameter (ΔW) is used to fine-tune the weight parameters of the feature mapping model corresponding to the first layout information after fine-tuning (i.e., updating), resulting in W + ΔW. Then, ΔW is decomposed into the product of two lower-rank matrices, ΔW = B × A, where B (corresponding to the second lower-rank matrix) and A (corresponding to the first lower-rank matrix), and the dimension of the second lower-rank matrix is ​​the same as the transpose of the first lower-rank matrix. During the reverse update process, W is frozen (W is not updated, corresponding to keeping the network parameters unchanged except for the first and second lower-rank matrices), and only the parameters of A and B are updated.

[0230] Steps 4051 to 4053 introduce low-rank matrices (first low-rank matrix and second low-rank matrix) to replace the complete weight update, which significantly reduces the number of parameters that need to be trained, thereby enabling the model to converge faster and achieving the beneficial effect of improving training efficiency.

[0231] In some embodiments, after updating the first neural network model, the following processes may also be performed: performing a decision performance test on the updated first neural network model to obtain a second decision performance parameter; in response to the second decision performance parameter being less than a preset decision performance parameter threshold, updating the updated first neural network model again until the updated first neural network model passes the decision performance test.

[0232] In some embodiments, a predicted action strategy for a virtual object is determined by an updated first neural network model, and the virtual object is controlled to execute the predicted action strategy in a virtual scene (see the description of step 1041 above); based on the interaction results achieved by the virtual object in executing the predicted action strategy, a first prediction performance parameter is determined, wherein the first prediction performance parameter includes at least one of the following: the win rate of the virtual object, an anthropomorphic index, wherein the anthropomorphic index is used to characterize the degree to which the actions of the virtual object match those of the real object (see the description of step 1042 above).

[0233] Through steps 104 and 105, after the virtual object executes the predicted action strategy obtained through the first neural network inference, the decision performance of the first neural network model is tested. If the decision performance test fails, a corresponding feature mapping model is added to the virtual object or a separate update is made based on the feature mapping model corresponding to the first layout information of the virtual object. This will not affect the parameters of other feature mapping models, thus avoiding the problems in related technologies where the model needs to be retrained when a new virtual object is connected, resulting in low training efficiency and the action decision of the old virtual object may be affected by retraining, requiring adjustment of training parameters again, leading to high training and connection costs.

[0234] In some embodiments, probability prediction is achieved through a first neural network model, see [link to relevant documentation]. Figure 3K Before performing probability prediction processing corresponding to the first layout information and the feature information to obtain the selection probability of each action in the preset first action set, the first neural network model can be trained through the following steps 501 to 505, which are explained in detail below.

[0235] In step 501, layout information samples and feature information samples of multiple virtual object samples in the virtual scene are obtained.

[0236] In some embodiments, see Figure 5C The layout information samples and feature information samples of multiple virtual object samples are obtained. Virtual object samples with the same layout information samples are classified into the same domain. Then, based on the feature information samples of virtual object samples in the domain corresponding to the layout information samples, the feature mapping model corresponding to the layout information samples is trained.

[0237] In step 502, the first neural network model to be trained performs probability prediction based on the layout information sample and feature information sample of each virtual object sample to obtain the selection probability of each action sample in the preset action set sample corresponding to each virtual object sample.

[0238] In some embodiments, the following processing is performed on each virtual object sample: the feature information sample is feature-encoded by the feature encoding model in the first neural network model to obtain the object feature vector (see the description of step 1021 above); the object feature vector is feature-mapped by the feature mapping model corresponding to the first layout information sample (the layout information sample corresponding to the virtual object) to obtain the selection probability of each action sample in the first action set sample (see the description of step 1022 above).

[0239] In some embodiments, the first action set sample is generated by obtaining a pre-set first action set sample, wherein the first action set sample is specifically set for the first layout information sample, or the first action set sample is uniformly set for different layout information samples, and each action sample in the first action set sample includes at least one of the following attributes: action event, action direction, action end point, and target object.

[0240] In some embodiments, the following processing is performed on each virtual object sample: when multiple feature mapping models share the same input layer, the object feature vector is input into the input layer to perform feature mapping through multiple feature mapping models respectively, so as to obtain the probability distributions corresponding to multiple layout information samples respectively, and the probability distribution corresponding to the second layout information sample in the probability distributions corresponding to the multiple layout information samples is masked to obtain the probability distribution corresponding to the first layout information sample, wherein the second layout information sample is the layout information that is different from the first layout information sample among the multiple layout information samples, and the probability distribution corresponding to the first layout information sample includes the selection probability of each action in the first action set sample.

[0241] In some embodiments, when multiple feature mapping models each have an independent input layer, the object feature vector is input into the input layer of the feature mapping model corresponding to the first layout information sample, so as to perform feature mapping through the feature mapping model corresponding to the first layout information sample and obtain the probability distribution corresponding to the first layout information sample.

[0242] In step 503, based on the selection probability of each action sample, at least one action sample is selected from the action set sample as the action strategy sample to be executed for each virtual object sample.

[0243] In some embodiments, the following processing is performed for each virtual object sample: the action sample with the highest probability is selected as the action policy sample to be executed, or the top N action samples with the highest probability are selected as the action policy samples to be executed for the virtual object.

[0244] In step 504, a third loss value is determined based on the action policy samples and the labeled action policies.

[0245] In some embodiments, a third loss value is calculated between the action policy sample and the preset labeled action policy (the action performed by the real object) based on a preset loss function (e.g., cross-entropy loss function).

[0246] In step 505, the parameters of the first neural network model are updated based on the third loss value to obtain the trained first neural network model.

[0247] In some embodiments, the gradient of the loss function with respect to the parameters of the first neural network model is calculated based on the third loss value using the backpropagation (BP) algorithm. The model parameters of multiple network layers of the first neural network model are updated according to the direction of the gradient using gradient descent methods (such as full gradient descent (BGD), stochastic gradient descent (SGD), etc.) to obtain the trained first neural network model.

[0248] The action decision-making method for virtual objects provided in this application can be applied to game scenarios, such as controlling the interaction between AI characters (e.g., non-player characters, NPCs) and player-controlled characters in the game, so that the AI ​​characters can interact with the player autonomously. Such interactions may include, but are not limited to, cooperation and competition.

[0249] The following will describe exemplary applications of the embodiments of this application in game scenarios. See also Figure 4 , Figure 4 This is a schematic diagram of the application flow of the action decision method for virtual objects provided in the embodiments of this application, which will be combined with Figure 4 The steps shown are explained.

[0250] In step 601, the character layout information and character feature information of the artificial intelligence character in the game scene are obtained.

[0251] In some embodiments, the character layout information (corresponding to the first layout information above) includes any one of the following: the position of the AI ​​character (corresponding to the virtual object above) in the game scene (corresponding to the virtual scene above) (such as the position in the game map), the movement route of the AI ​​character in the game scene (such as the lane to which the game character belongs (such as top lane, middle lane, bottom lane, etc.)), the faction to which the AI ​​character belongs in the game scene (such as the red team faction, blue team faction), and the map grid of the AI ​​character in the game scene (such as the map grid in which the game character's activity range is located after the game map is divided into multiple grids).

[0252] In some embodiments, character feature information (corresponding to the feature information above) is used to represent the attributes of the AI ​​character itself and the state of the game environment surrounding the AI ​​character. The attributes of the AI ​​character itself may include category information and attribute information, and the state of the game environment surrounding the AI ​​character can be represented by image information in the character feature information.

[0253] For example, the attributes of an AI character (or intelligent agent) include the game character's name, class, etc. (corresponding category information); health, skills, movement direction, etc. in the game (corresponding attribute information); and the distribution of teammates, opposing team game characters, or obstacles around the AI ​​character in the game (corresponding to the state of the virtual environment around the virtual object). Feature information can be obtained from game logs or game image frames automatically recorded by the game engine or specialized recording software when the user plays the game.

[0254] In step 602, based on the character layout information and character feature information, probability prediction processing corresponding to the character layout information is performed to obtain the selection probability of each action in the preset action set.

[0255] In some embodiments, the preset action set (corresponding to the first action set above) is specifically set for character layout information, or the action set is uniformly set for different character layout information. Each action in the action set includes at least one of the following attributes: action event, action direction, action end point, and target. The generation of the action set can be found in the description of steps 201 to 206 above, and will not be repeated here.

[0256] For example, an action event can be a game character's actions such as standing, moving, releasing a skill, returning to town, or purchasing equipment. The direction of the action can be the direction the game character is moving towards or releasing a skill towards. The landing point of the action can be the landing point of the skill after it is released. The target of the action can be the object that the skill is applied to, such as game characters, monsters, or defensive towers in the opposing faction.

[0257] In some embodiments, the character feature information is encoded by the feature encoding model in the first neural network model to obtain the character feature vector (corresponding to the object feature vector above, see the description of step 1021 above); the character feature vector is mapped by the feature mapping model corresponding to the character layout information to obtain the selection probability of each action in the action set (see the description of step 1022 above).

[0258] For example, feature mapping can be achieved using a fully connected layer (FC). The dimension of the fully connected layer is set to the number of actions in the action set, and the probability distribution of each action is output through the Softmax function, thereby mapping the character feature vector to the selection probability of each action in the action set.

[0259] In step 603, based on the selection probability of each action, at least one action is selected from the action set as the action strategy to be executed by the artificial intelligence character.

[0260] In some embodiments, the action with the highest probability is selected as the action strategy to be executed, or the top N actions with the highest probability are selected as the action strategies to be executed by the AI ​​character. For example, when there are multiple action strategies to be executed, the action strategy to be executed by the AI ​​character can be represented as (move, stand, return to town), that is, the AI ​​character first performs the move action, then performs the stand action after moving to the target location, and finally performs the return to town action.

[0261] In step 604, the AI ​​character is controlled to execute an action strategy to interact with the player-controlled character.

[0262] In some embodiments, these action strategies can be sent to the game world through the game engine's Application Programming Interface (API), enabling AI characters to exhibit corresponding behaviors in the game.

[0263] For example, the action strategy of an AI character may include a series of actions to be performed. Here are some common examples of action strategies:

[0264] Attack strategy:

[0265] Action to be executed: Choose an appropriate time to launch an attack.

[0266] Strategy Details: Use specific skills or weapons based on the weaknesses of the opponent's characters, and arrange the attack order reasonably to maximize damage output.

[0267] Defense strategy:

[0268] Action to be performed: Defend when attacked.

[0269] Strategy details: Use shields or defensive skills to reduce damage, or maintain distance during the match to avoid the attack range of the opposing team's characters.

[0270] Evasion strategy:

[0271] Action to be performed: Dodge when the attack is about to hit.

[0272] Strategy details: Observe the attack patterns of the opponent's characters, predict the timing of their attacks, and quickly move to the side or jump to avoid them.

[0273] Resource management strategy:

[0274] Actions to be performed: Allocate and use resources reasonably (such as health, mana, items, etc.).

[0275] Strategy details: Use healing items wisely during the game, or conserve resources at crucial moments to deal with subsequent challenges.

[0276] Exploration strategy:

[0277] Action to be performed: Explore unknown areas in the game.

[0278] Strategy details: Find hidden paths, items, or secret levels to obtain additional virtual resources or information.

[0279] Teamwork strategies:

[0280] Action to be performed: Collaborate with teammates to complete the task.

[0281] Strategy details: Based on the abilities of teammates and the division of roles in the game, conduct effective cooperation and communication.

[0282] Environmental utilization strategies:

[0283] Actions to be performed: Use elements in the game environment to play a game or complete a task.

[0284] Strategy details: Use terrain cover for ambushes, or use special skills in specific environments.

[0285] Through steps 601 to 604, probability prediction processing corresponding to the role layout information is achieved based on the role layout information and role feature information of the artificial intelligence role. Differentiated probability prediction processing is achieved for artificial intelligence roles with different role layout information. The unique role layout information and role feature information of each artificial intelligence role are considered. By setting a corresponding feature mapping model for each role layout information, the probability prediction of artificial intelligence roles corresponding to different role layout information does not interfere with each other, and more personalized probability prediction can be provided, making the prediction results more accurate and consistent with the characteristics of each artificial intelligence role.

[0286] By learning from the actions and decisions of high-level players, AI characters can serve as more challenging opponents, helping players improve their gaming skills and introducing new game modes such as player-versus-AI battles and AI-assisted practice, thereby enriching the player's gaming experience.

[0287] The following description continues to illustrate the exemplary structure of the virtual object action decision device 433 provided in the embodiments of this application as a software module. In some embodiments, such as Figure 2 As shown, the software module in the action decision device 433 of the virtual object stored in the memory 430 may include:

[0288] The data acquisition module 4331 is used to acquire the first layout information and feature information of the virtual object in the virtual scene.

[0289] The action prediction module 4332 is used to perform probability prediction processing corresponding to the first layout information based on the first layout information and the feature information, so as to obtain the selection probability of each action in the preset first action set.

[0290] In some embodiments, the action prediction module 4332 is further configured to select at least one action from the action set based on the selection probability of each action, as an action strategy to be executed by the virtual object.

[0291] In some embodiments, the action decision method for the virtual object is implemented through a first neural network model, which includes a feature encoding model and multiple feature mapping models. The multiple feature mapping models correspond to multiple layout information, and the multiple layout information includes the first layout information. The action prediction module 4332 is further configured to encode the feature information using the feature encoding model to obtain an object feature vector; and to perform feature mapping on the object feature vector using the feature mapping model corresponding to the first layout information to obtain the selection probability of each action in the first action set.

[0292] In some embodiments, the action prediction module 4332 is further configured to, when the multiple feature mapping models share the same input layer, input the object feature vector into the input layer to perform feature mapping through the multiple feature mapping models respectively, to obtain the probability distributions corresponding to the multiple layout information respectively, and to perform masking processing on the probability distribution corresponding to the second layout information in the probability distributions corresponding to the multiple layout information respectively, to obtain the probability distribution corresponding to the first layout information, wherein the second layout information is the layout information that is different from the first layout information among the multiple layout information, and the probability distribution corresponding to the first layout information includes the selection probability of each action in the first action set.

[0293] In some embodiments, the action prediction module 4332 is further configured to, when each of the plurality of feature mapping models has an independent input layer, input the object feature vector into the input layer of the feature mapping model corresponding to the first layout information, so as to perform feature mapping through the feature mapping model corresponding to the first layout information to obtain the probability distribution corresponding to the first layout information.

[0294] In some embodiments, the action prediction module 4332 is further configured to perform data conversion processing on the feature information to obtain numerical information; and to perform feature extraction processing on the numerical information to obtain the object feature vector.

[0295] In some embodiments, the action prediction module 4332 is further configured to perform any one of the following processing on the feature information through the feature encoding model: perform one-hot encoding on the category information in the feature information; perform embedding encoding on the attribute information in the feature information; perform network partitioning on the image information in the feature information to obtain multiple image grids, and perform binarization on each of the image grids; create multiple rays according to a preset number of rays and ray direction, using virtual objects in the image information as ray endpoints, and detect elements in the ray direction through each ray to obtain detection results, and perform numerical processing on the detection results in each ray direction.

[0296] In some embodiments, the action prediction module 4332 is further configured to normalize the numerical information to obtain normalized information; perform linear transformation on the normalized information to obtain linear features; and perform nonlinear transformation on the linear features to obtain the object feature vector.

[0297] In some embodiments, the data processing module 4331 is further configured to perform a decision performance test on the first neural network model to obtain a first decision performance parameter; and update the first neural network model in response to the first decision performance parameter being less than a preset decision performance parameter threshold, wherein the updated first neural network model is used to replace the unupdated first neural network model for the probability prediction.

[0298] In some embodiments, the data processing module 4331 is further configured to determine the prediction action strategy of the virtual object through the first neural network model, control the virtual object to execute the prediction action strategy in the virtual scene; and determine a first prediction performance parameter based on the interaction result achieved by the virtual object in executing the prediction action strategy, wherein the first prediction performance parameter includes at least one of the following: the win rate of the virtual object, an anthropomorphic index, wherein the anthropomorphic index is used to characterize the degree to which the action of the virtual object matches the real object.

[0299] In some embodiments, the action decision-making method for the virtual object is implemented through a first neural network model, which includes a feature encoding model and multiple feature mapping models. Each feature mapping model corresponds to multiple layout information, and the multiple layout information includes the first layout information. The data processing module 4331 is further configured to: create an action set sample, wherein the action set sample includes multiple action samples adapted to the virtual object; obtain feature information samples of the virtual object under the control of a real object; add a feature mapping model corresponding to the virtual object to the first neural network model to obtain a second neural network model; perform probability prediction based on the feature information samples using the second neural network model to obtain the selection probability of each action sample in the action set sample; select at least one action sample from the action set sample based on the selection probability of each action sample as an action strategy sample to be executed by the virtual object; determine a first loss value based on the action strategy sample and a preset labeled action strategy; and update the parameters of the feature mapping model corresponding to the virtual object based on the first loss value to obtain an updated second neural network model.

[0300] In some embodiments, the data processing module 4331 is further configured to: acquire feature information samples of the virtual object under the control of a real object; perform probability prediction based on the feature information samples using the first neural network model to obtain the selection probability of each action sample in the action set sample; select at least one action sample from the action set sample based on the selection probability of each action sample as an action strategy sample to be executed by the virtual object; determine a second loss value based on the action strategy sample and a preset labeled action strategy; and update the parameters of the feature mapping model corresponding to the first layout information based on the second loss value to obtain the updated first neural network model.

[0301] In some embodiments, the data processing module 4331 is further configured to generate a first low-rank matrix initialized based on a Gaussian distribution, and generate a second low-rank matrix initialized based on all zeros; construct fine-tuning weight parameters based on the first low-rank matrix and the second low-rank matrix, and add the fine-tuning weight parameters to the parameters of the feature mapping model corresponding to the first layout information; update the fine-tuning weight parameters through the second loss value to obtain the updated first neural network model.

[0302] In some embodiments, the data processing module 4331 is further configured to perform a decision performance test on the updated first neural network model to obtain a second decision performance parameter; in response to the second decision performance parameter being less than a preset decision performance parameter threshold, the updated first neural network model is updated again until the updated first neural network model passes the decision performance test.

[0303] In some embodiments, the data processing module 4331 is further configured to acquire layout information samples and feature information samples of multiple virtual object samples in the virtual scene; through the first neural network model to be trained, perform probability prediction corresponding to each layout information sample based on the layout information sample and feature information sample of each virtual object sample to obtain the selection probability of each action sample in the preset action set sample corresponding to each virtual object sample; based on the selection probability of each action sample, select at least one action sample from the action set sample as an action strategy sample to be executed for each virtual object sample; determine a third loss value based on the action strategy sample and the labeled action strategy; update the parameters of the first neural network model based on the third loss value to obtain the trained first neural network model.

[0304] In some embodiments, the data processing module 4331 is further configured to obtain a pre-set first action set, wherein the first action set is specifically set for the first layout information, or the first action set is uniformly set for different layout information, and each action in the first action set includes at least one of the following attributes: action event, action direction, action end point, and target object.

[0305] In some embodiments, the data processing module 4331 is further configured to acquire interaction record data, wherein the interaction record data is used to record the interaction of the virtual object under the control of a real object; detect at least one action event from the interaction record data, wherein the at least one action event is performed by the virtual object; acquire the position sequence of the virtual object when performing the action event, and determine the action direction corresponding to the action event based on the position sequence; query a distance parameter pre-associated with the action event, and determine the action end point of the action event based on the distance parameter and the action direction; perform target recognition on the interaction record data to obtain the target object; and combine the action event, the action direction, the action end point, and the target object to obtain a first action set corresponding to the first layout information.

[0306] This application provides a computer program product, which includes a computer program or computer-executable instructions stored in a computer-readable storage medium. The processor of an electronic device reads the computer-executable instructions from the computer-readable storage medium and executes the computer-executable instructions, causing the electronic device to perform the virtual object action decision-making method described above in this application.

[0307] This application provides a computer-readable storage medium storing computer-executable instructions or a computer program. When the computer-executable instructions or the computer program are executed by a processor, the processor will execute the action decision method for a virtual object provided in this application. For example, ... Figure 3A The action decision-making method for the virtual object is shown.

[0308] In some embodiments, the computer-readable storage medium may be a memory such as RAM, ROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0309] In some embodiments, computer-executable instructions may take the form of programs, software, software modules, scripts, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as stand-alone programs or as modules, components, subroutines, or other units suitable for use in a computing environment.

[0310] As an example, computer-executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple co-located files (e.g., files that store one or more modules, subroutines, or code sections).

[0311] As an example, computer-executable instructions can be deployed to execute on a single electronic device, or on multiple electronic devices located at one location, or on multiple electronic devices distributed across multiple locations and interconnected via a communication network.

[0312] In summary, through the embodiments of this application, based on the first layout information and feature information of virtual objects, probability prediction processing corresponding to the first layout information is performed. This achieves differentiated probability prediction processing for virtual objects with different layout information, taking into account the unique layout information and feature information of each virtual object. It can provide more targeted information for probability prediction, enabling the predicted action strategy to be adapted to the layout information and feature information of the virtual object. This not only ensures the accuracy of the action strategy but also effectively realizes the diversification of action strategies for different virtual objects, thereby better adapting to multi-virtual object systems.

[0313] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. A method for action decision-making of virtual objects, characterized in that, The method includes: Obtain the first layout information and feature information of the virtual objects in the virtual scene; Based on the first layout information and the feature information, a probability prediction process corresponding to the first layout information is performed to obtain the selection probability of each action in the preset first action set. Based on the selection probability of each action, at least one action is selected from the first set of actions as the action strategy to be executed by the virtual object.

2. The method according to claim 1, characterized in that, The action decision method is implemented through a first neural network model, which includes a feature encoding model and multiple feature mapping models. The multiple feature mapping models correspond to multiple layout information, and the multiple layout information includes the first layout information. The step of performing probability prediction based on the first layout information and the feature information to obtain the selection probability of each action in the preset first action set includes: The feature information is encoded using the feature encoding model to obtain the object feature vector; By using the feature mapping model corresponding to the first layout information, feature mapping is performed on the object feature vector to obtain the selection probability of each action in the first action set.

3. The method according to claim 2, characterized in that, The step of performing feature mapping on the object feature vector using the feature mapping model corresponding to the first layout information to obtain the selection probability of each action in the first action set includes: When multiple feature mapping models share the same input layer, the object feature vector is input into the input layer so that feature mapping is performed by the multiple feature mapping models respectively, resulting in probability distributions corresponding to the multiple layout information. The probability distribution corresponding to the second layout information in the probability distribution corresponding to the plurality of layout information is masked to obtain the probability distribution corresponding to the first layout information. The second layout information is the layout information that is different from the first layout information among the plurality of layout information. The probability distribution corresponding to the first layout information includes the selection probability of each action in the first action set.

4. The method according to claim 2, characterized in that, The step of performing feature mapping on the object feature vector using the feature mapping model corresponding to the first layout information to obtain the selection probability of each action in the first action set includes: When each of the multiple feature mapping models has an independent input layer, the object feature vector is input into the input layer of the feature mapping model corresponding to the first layout information, so as to perform feature mapping through the feature mapping model corresponding to the first layout information and obtain the probability distribution corresponding to the first layout information.

5. The method according to claim 2, characterized in that, The step of encoding the feature information to obtain the object feature vector includes: The feature information is processed by data conversion to obtain numerical information; The numerical information is subjected to feature extraction processing to obtain the object feature vector.

6. The method according to claim 5, characterized in that, The step of performing data conversion processing on the feature information through the feature encoding model to obtain numerical information includes: The feature information is processed using the feature encoding model using any of the following methods: The category information in the feature information is processed by one-hot encoding; The attribute information in the feature information is embedded and encoded. The image information in the feature information is divided into multiple image grids by network partitioning, and each image grid is binarized. Using the virtual object in the image information as the endpoint of the ray, multiple rays are created according to the preset number of rays and ray direction. Elements in the ray direction are detected through each ray to obtain the detection result. The detection result of each ray direction is then numerically processed.

7. The method according to claim 5, characterized in that, The step of performing feature extraction processing on the numerical information to obtain the object feature vector includes: The numerical information is normalized to obtain normalized information; The normalized information is subjected to a linear transformation to obtain linear features; The linear features are subjected to a nonlinear transformation to obtain the object feature vector.

8. The method according to claim 1, characterized in that, The probability prediction is achieved through a first neural network model. After controlling the virtual object to execute the action strategy to be executed, the method further includes: The decision performance of the first neural network model is tested to obtain the first decision performance parameters; In response to the first decision performance parameter being less than a preset decision performance parameter threshold, the first neural network model is updated, wherein the updated first neural network model is used to replace the unupdated first neural network model for the probability prediction.

9. The method according to claim 8, characterized in that, The step of testing the decision performance of the first neural network model to obtain the first decision performance parameters includes: The first neural network model is used to determine the prediction action strategy of the virtual object, and the virtual object is controlled to execute the prediction action strategy in the virtual scene. Based on the interaction results achieved by the virtual object in executing the prediction action strategy, a first prediction performance parameter is determined, wherein the first prediction performance parameter includes at least one of the following: the win rate of the virtual object, and an anthropomorphic index, wherein the anthropomorphic index is used to characterize the degree to which the action of the virtual object matches that of the real object.

10. The method according to claim 8, characterized in that, The action decision method is implemented through a first neural network model, which includes a feature encoding model and multiple feature mapping models. The multiple feature mapping models correspond to multiple layout information, and the multiple layout information includes the first layout information. The step of updating the first neural network model includes: Create an action set sample, wherein the action set sample includes multiple action samples adapted to the virtual object; Obtain feature information samples of the virtual object when it is controlled by a real object; In the first neural network model, a feature mapping model corresponding to the virtual object is added to obtain a second neural network model; The second neural network model is used to perform probability prediction based on the feature information samples to obtain the selection probability of each action sample in the action set sample. Based on the selection probability of each action sample, at least one action sample is selected from the action set sample as the action strategy sample to be executed by the virtual object; Based on the action strategy samples and the preset labeled action strategy, a first loss value is determined; Based on the first loss value, the parameters of the feature mapping model corresponding to the virtual object are updated to obtain the updated second neural network model.

11. The method according to claim 8, characterized in that, The action decision method is implemented through a first neural network model, which includes a feature encoding model and multiple feature mapping models. The multiple feature mapping models correspond to multiple layout information, and the multiple layout information includes the first layout information. The step of updating the first neural network model includes: Obtain feature information samples of the virtual object when it is controlled by a real object; The first neural network model is used to perform probability prediction based on the feature information samples to obtain the selection probability of each action sample in the action set sample. Based on the selection probability of each action sample, at least one action sample is selected from the action set sample as the action strategy sample to be executed by the virtual object; Based on the action strategy samples and the preset labeled action strategy, a second loss value is determined; Based on the second loss value, the parameters of the feature mapping model corresponding to the first layout information are updated to obtain the updated first neural network model.

12. The method according to claim 11, characterized in that, The step of updating the parameters of the feature mapping model corresponding to the first layout information based on the second loss value to obtain the updated first neural network model includes: Generate a first low-rank matrix initialized based on a Gaussian distribution, and generate a second low-rank matrix initialized based on all zeros; Fine-tuning weight parameters are constructed based on the first low-rank matrix and the second low-rank matrix, and the fine-tuning weight parameters are added to the parameters of the feature mapping model corresponding to the first layout information. The fine-tuned weight parameters are updated using the second loss value to obtain the updated first neural network model.

13. The method according to claim 8, characterized in that, After updating the first neural network model, the method further includes: The updated first neural network model is tested for decision performance to obtain the second decision performance parameters. In response to the second decision performance parameter being less than a preset decision performance parameter threshold, the updated first neural network model is updated again until the updated first neural network model passes the decision performance test.

14. The method according to any one of claims 1 to 13, characterized in that, The probability prediction is implemented through a first neural network model. Before performing probability prediction processing corresponding to the first layout information based on the first layout information and the feature information to obtain the selection probability of each action in the preset first action set, the method further includes: Obtain layout information samples and feature information samples of multiple virtual object samples in the virtual scene; Using the first neural network model to be trained, probability prediction is performed based on the layout information sample and feature information sample of each virtual object sample to obtain the selection probability of each action sample in the preset action set sample corresponding to each virtual object sample. Based on the selection probability of each action sample, at least one action sample is selected from the action set sample as the action strategy sample to be executed for each virtual object sample; Based on the action policy samples and labeled action policies, a third loss value is determined; The parameters of the first neural network model are updated based on the third loss value to obtain the trained first neural network model.

15. The method according to any one of claims 1 to 13, characterized in that, The first layout information includes any one of the following: the position of the virtual object in the virtual scene, the movement route of the virtual object in the virtual scene, the faction to which the virtual object belongs in the virtual scene, and the map grid in which the virtual object is located in the virtual scene; The first set of actions was generated in the following way: Obtain a pre-set first action set, wherein the first action set is specifically set for the first layout information, or the first action set is uniformly set for different layout information, and each action in the first action set includes at least one of the following attributes: action event, action direction, action end point, and target object.

16. The method according to any one of claims 1 to 13, characterized in that, The first set of actions was generated in the following way: Acquire interaction record data, wherein the interaction record data is used to record the interactions of the virtual object under the control of a real object; At least one action event is detected from the interaction record data, wherein the at least one action event is performed by the virtual object; Obtain the position sequence of the virtual object when it performs the action event, and determine the action direction corresponding to the action event based on the position sequence; Query the distance parameters that are pre-associated with the action event, and determine the action end point of the action event based on the distance parameters and the action direction; The interaction record data is used for target identification to obtain the target object; Based on the action event, the action direction, the action end point, and the target object, a first action set corresponding to the first layout information is obtained.

17. A virtual object action decision-making device, characterized in that, The device includes: The data acquisition module is used to acquire the first layout information and feature information of the virtual objects in the virtual scene; The action prediction module is used to perform probability prediction processing corresponding to the first layout information based on the first layout information and the feature information, so as to obtain the selection probability of each action in the preset first action set. The action prediction module is further configured to select at least one action from the action set based on the selection probability of each action, as the action strategy to be executed by the virtual object.

18. An electronic device, characterized in that, The electronic device includes: Memory is used to store executable instructions or computer programs. A processor, when executing computer-executable instructions or computer programs stored in the memory, implements the action decision method for the virtual object as described in any one of claims 1 to 16.

19. A computer-readable storage medium storing computer-executable instructions or a computer program, characterized in that, When the computer-executable instructions or computer program are executed by a processor, the action decision method for the virtual object as described in any one of claims 1 to 16 is implemented.

20. A computer program product comprising computer-executable instructions or a computer program, characterized in that, When the computer-executable instructions or computer program are executed by a processor, the action decision method for the virtual object as described in any one of claims 1 to 16 is implemented.

Images