Behavior control method and apparatus

Abstract

The application provides a behavior control method and device; the method comprises the following steps: determining a current behavior state of a virtual object and a state transition matrix of the virtual object in a current time period; determining a first state transition probability distribution corresponding to the current behavior state from the state transition matrix; in response to the first state transition probability distribution, transferring the behavior state of the virtual object from the current behavior state to a first target behavior state in a behavior state set, generating a first behavior event queue based on a Poisson rate corresponding to the first target behavior state; and controlling the virtual object to execute behavior events in the first behavior event queue in sequence. Through the application, accurate control of the behavior of the virtual object can be realized, thereby improving the fidelity of the behavior of the virtual object.

Description

Technical Field

[0001] This application relates to the field of Internet technology, and in particular to a behavior control method and device. Background Technology

[0002] In virtual environments such as gaming, rule-based artificial intelligence (AI) robot systems implemented using finite state machines or behavior trees can perform functions such as leaderboard population and beginner coaching by setting states, defining behavior priorities, and adjusting based on contextual information. However, they suffer from issues such as a lack of realism in behavior rhythm and a crude state transition mechanism. This essentially reflects a technical deficiency in the rationality and precision of simulating real player behavior. The root cause lies in the inadequacy of the behavior control mechanism. The behavior control models in related technologies struggle to accurately adapt to the behavioral patterns of real players, making it impossible to achieve a more realistic behavior rhythm for the AI ​​robot through effective behavior control, and also making it difficult to achieve natural state transitions through refined behavior control. Summary of the Invention

[0003] This application provides a behavior control method and apparatus that can achieve precise control over the behavior of virtual objects, thereby improving the realism of the behavior of virtual objects.

[0004] The technical solution of this application embodiment is implemented as follows:

[0005] This application provides a behavior control method, the method comprising: determining the current behavior state of a virtual object and a state transition matrix of the virtual object in the current time period; determining a first state transition probability distribution corresponding to the current behavior state from the state transition matrix; the first state transition probability distribution including a plurality of first transition probabilities; the first transition probability being the probability of transitioning the behavior state of the virtual object from the current behavior state to a preset behavior state, the preset behavior state being any behavior state in a preset set of behavior states; responding to transitioning the behavior state of the virtual object from the current behavior state to a first target behavior state in the set of behavior states based on the first state transition probability distribution, generating a first behavior event queue based on the Poisson rate corresponding to the first target behavior state; and controlling the virtual object to sequentially execute behavior events in the first behavior event queue.

[0006] This application provides a behavior control device, comprising: a first determining module, configured to determine the current behavior state of a virtual object and a state transition matrix of the virtual object in the current time period; a second determining module, configured to determine a first state transition probability distribution corresponding to the current behavior state from the state transition matrix; the first state transition probability distribution includes a plurality of first transition probabilities; the first transition probability refers to the probability of transitioning the behavior state of the virtual object from the current behavior state to a preset behavior state, the preset behavior state being any behavior state in a preset set of behavior states; a generating module, configured to, in response to the first state transition probability distribution, transition the behavior state of the virtual object from the current behavior state to a first target behavior state in the set of behavior states, and generate a first behavior event queue based on the Poisson rate corresponding to the first target behavior state; and a control module, configured to control the virtual object to sequentially execute behavior events in the first behavior event queue.

[0007] In the above scheme, the first determining module is further configured to: obtain a transition matrix corresponding to the current time period from a preset matrix set; the preset matrix set includes transition matrices corresponding to different preset time periods; and determine the transition matrix corresponding to the current time period as the state transition matrix of the virtual object in the current time period.

[0008] In the above scheme, each row in the state transition matrix corresponds to a behavior state in the behavior state set; the second determining module is further configured to: determine the row vector in the state transition matrix corresponding to the index of the current behavior state in the behavior state set; and determine the row vector as the first state transition probability distribution corresponding to the current behavior state.

[0009] In the above scheme, the first behavior event queue includes M behavior events; M is an integer greater than 1; the generation module is further configured to: in response to the Poisson rate corresponding to the first target behavior state being greater than a preset value, iterate i and perform the following processing: based on the Poisson rate, determine the time interval of the i-th behavior event to be generated corresponding to the first target behavior state; where 1≤i≤M; based on the time interval and the current time, determine the occurrence time of the i-th behavior event to be generated; based on the occurrence time, generate the first behavior event queue.

[0010] In the above scheme, the generation module is further configured to: in response to determining that the i-th behavior event to be generated has not reached the preset queue termination condition based on the occurrence time, generate the i-th behavior event corresponding to the first target behavior state; associate the i-th behavior event with the occurrence time, add it to the event queue, and update the current time to the occurrence time; in response to determining that the i-th behavior event to be generated has reached the queue termination condition based on the occurrence time, prohibit the generation of the i-th behavior event, and determine the queue of generated behavior events as the first behavior event queue.

[0011] In the above scheme, the control module is further configured to: determine the occurrence time of each behavior event in the first behavior event queue; and control the virtual object to execute each behavior event in the first behavior event queue in sequence according to the occurrence time.

[0012] In the above scheme, the device further includes an adding module, which is used to: determine a second state transition probability distribution corresponding to the first target behavior state from the state transition matrix; the second state transition probability distribution includes a plurality of second transition probabilities; the second transition probability refers to the probability of transitioning the behavior state of the virtual object from the first target behavior state to the preset behavior state; in response to transitioning the behavior state of the virtual object from the first target behavior state to the second target behavior state in the behavior state set based on the second state transition probability distribution, generate a second behavior event queue based on the Poisson rate corresponding to the second target behavior state; and add the second behavior event queue to the first behavior event queue according to the occurrence time of each second behavior event in the second behavior event queue.

[0013] In the above scheme, the control module is further configured to: respond to the behavior event executed by the virtual object being a behavior event corresponding to a third target behavior state, determine the behavior event result of the behavior event according to a preset behavior event probability; the third target behavior state is used to characterize the virtual object being in a challenge state; and execute the behavior event according to the behavior event result.

[0014] In the above scheme, the device further includes an update module, which is used to: update the index value of the virtual object in the virtual environment according to the result of the behavior event; and in response to the behavior event reaching a preset behavior termination condition, transfer the behavior state of the virtual object from the third target behavior state to the fourth target behavior state, the fourth target behavior state being used to characterize the virtual object being in the behavior termination state in the virtual environment.

[0015] This application provides an electronic device, including: a memory for storing computer-executable instructions; and a processor for implementing the behavior control method provided in this application when executing the computer-executable instructions stored in the memory.

[0016] This application provides a computer-readable storage medium storing computer-executable instructions for implementing the behavior control method provided in this application when executed by a processor.

[0017] This application provides a computer program product including computer-executable instructions stored in a computer-readable storage medium. When the processor of an electronic device reads the computer-executable instructions from the computer-readable storage medium and executes the computer-executable instructions, it implements the behavior control method provided in this application.

[0018] The embodiments of this application have the following beneficial effects:

[0019] In the behavior control process, firstly, by accurately determining the first state transition matrix between the current behavior state and the current time period, the state transitions of virtual objects are correlated and adapted with the time dimension, avoiding a disconnect between the behavior logic of virtual objects and the time regularity, and improving the rationality of state transitions. Secondly, based on the first state transition probability distribution extracted from the first state transition matrix, a probabilistic decision-making basis is provided for behavior state transitions, making the transition of virtual objects from the current behavior state to the first target behavior state exhibit a natural tendency consistent with probabilistic characteristics, breaking through the mechanical limitations of fixed patterns. Next, by combining the Poisson rate corresponding to the first target behavior state to generate a first behavior event queue, the frequency of occurrence of behavior events can be precisely controlled, ensuring that the activity rhythm of virtual objects in a specific state matches the characteristics of that state, avoiding the problem of behavior density imbalance. Finally, by controlling the virtual objects to execute the first behavior event queue sequentially, the coherence and orderliness of the behavior event sequence are ensured, making the overall behavior performance of virtual objects highly consistent with the expected behavior pattern in the virtual environment, significantly enhancing the realism and immersion of virtual object behavior. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of the structure of the behavior control system provided in the embodiments of this application;

[0021] Figure 2 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application;

[0022] Figure 3 This is an optional flowchart illustrating the behavior control method provided in the embodiments of this application;

[0023] Figure 4This is another optional flowchart illustrating the behavior control method provided in the embodiments of this application;

[0024] Figure 5 This is a schematic diagram illustrating the implementation process of determining the state transition matrix provided in an embodiment of this application;

[0025] Figure 6 This is a schematic diagram of an optional implementation process for generating a first-beginning event queue provided in an embodiment of this application;

[0026] Figure 7 This is a schematic diagram of another optional implementation process for generating the first behavior event queue provided in an embodiment of this application;

[0027] Figure 8 This is a schematic diagram of an optional implementation process of the execution behavior event provided in an embodiment of this application;

[0028] Figure 9 This is a schematic diagram of another optional implementation process of the execution behavior event provided in the embodiments of this application;

[0029] Figure 10 This is a schematic diagram illustrating the implementation of updating index values ​​and state transitions provided in the embodiments of this application. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0031] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments, but it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0032] In the following description, the terms "first, second, third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0033] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0034] Unless otherwise defined, all technical and scientific terms used in the embodiments of this application have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in the embodiments of this application is for the purpose of describing the embodiments of this application only and is not intended to limit this application.

[0035] In the implementation of this application, the collection and processing of relevant data should strictly comply with the requirements of relevant laws and regulations, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of laws and regulations and the authorization of the personal information subject.

[0036] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0037] 1) Markov Modulated Poisson Process (MMPP): This is a non-homogeneous Poisson process where the arrival rate (i.e., the event occurrence rate) is determined by a Markov chain of finite states. The core idea is twofold: firstly, a hidden Markov chain represents the system's behavioral states, such as whether an AI player is currently online or in a level-based progression state; secondly, each Markov state corresponds to a different arrival rate, representing the expected number of actions occurring per unit of time.

[0038] 2) Poisson Process: A mathematical model used to model the random occurrence of sparse and independent events. It has the following properties: the expected number of events occurring per unit time is λ (i.e., the Poisson rate, in times / hour); the interval between adjacent events follows an exponential distribution, which can be expressed as T ~ Exp(λ), and the probability density function of the exponential distribution is f(t) = λe^{-λt}, where t ≥ 0.

[0039] 3) Markov transition matrix: A state transition probability matrix used to describe the probability of a system transitioning to other states from its current state. Assume the AI ​​player has N behavioral states S0, S1, ..., S... n-1This matrix is ​​an N×N matrix, denoted by P, with matrix elements P. ij Represents the state S i Transition to state S j The probability of P is such that the sum of the elements in each row equals 1, and all P's are also such that P's are equal to 1. ij All are within the interval [0,1].

[0040] In virtual environments such as gaming, rule-based AI bot systems are often used for leaderboard population or beginner coaching. These systems are typically implemented using Finite State Machines (FSMs) or Behavior Trees (BTs). FSMs assign several states to each AI player, such as online, matchmaking, combat, and offline. BTs, on the other hand, use behavior trees to define the priority and execution conditions of behaviors, similar to a script. Furthermore, these states or behaviors are influenced by contextual information such as timestamps, win / loss records, and holidays. However, the relevant technologies have some obvious drawbacks: First, the behavior rhythm lacks realism. The behavior of AI players is mostly driven by rules or generates events evenly, without modeling the online frequency of real players at different times, making them easy to identify as bots. Second, the state transition mechanism is crude. AI players usually do not distinguish between rest and active states, and cannot reflect the switching process of real players' multiple states such as online, playing, and resting. Third, there is a lack of adjustable parameter control. AI bot systems have difficulty adjusting dimensions such as activity frequency and win rate according to the target. Fourth, there is a lack of randomness in real situations, while real leaderboards often contain a large number of random events.

[0041] To address at least one of the problems in the aforementioned related technologies, this application proposes a game agent design method based on Markov-modulated Poisson processes. This method is suitable for leaderboard environments in light to moderate-intensity games and has the following technical improvements: In behavior modeling, Markov-modulated Poisson processes are used to simulate the online and level-clearing behaviors of AI players, giving the AI ​​player behavior human-like rhythm and statistical characteristics, thereby enhancing the player's immersion and interactivity; simultaneously, a dynamic state control mechanism is constructed to set an adjustable set of states for the AI ​​player (e.g., resting and level-clearing), and automatically transfer states based on strategies such as time period, behavior rhythm, and success rate, achieving behavior modeling that combines periodicity and randomness; furthermore, it promotes integration with leaderboard systems, designing a scheduling mechanism that can be embedded in the leaderboard system, adjusting the number and performance of players through AI, incentivizing real players to participate in competition, and thus improving user stickiness.

[0042] The following describes exemplary applications of the behavior control device (i.e., electronic device) provided in the embodiments of this application. The behavior control device provided in the embodiments of this application can be implemented as various types of user terminals capable of behavior control, such as laptops, tablets, desktop computers, set-top boxes, mobile devices (e.g., mobile phones, portable music players, personal digital assistants, dedicated messaging devices, portable gaming devices), smartphones, smart speakers, smartwatches, smart TVs, and in-vehicle terminals. It can also be implemented as a server. The following will describe exemplary applications when the behavior control device is implemented as a server.

[0043] See Figure 1 , Figure 1 This is a schematic diagram of the architecture of the behavior control system 100 provided in this application embodiment. In order to support a behavior control application, the terminal 400 connects to the server 200 through the network 300. The network 300 can be a wide area network or a local area network, or a combination of the two.

[0044] Terminal 400 sends a behavior control request to server 200, which constitutes the behavior control device in this embodiment. In response to the behavior control request, server 200 determines the current behavior state of a virtual object and the state transition matrix of the virtual object in the current time period. Then, it determines a first state transition probability distribution corresponding to the current behavior state from the state transition matrix. The first state transition probability distribution includes multiple first transition probabilities. Each first transition probability is the probability of transitioning the virtual object's behavior state from the current behavior state to a preset behavior state, where the preset behavior state is any behavior state in a preset set of behavior states. Next, in response to transitioning the virtual object's behavior state from the current behavior state to a first target behavior state in the set of behavior states based on the first state transition probability distribution, a first behavior event queue is generated based on the Poisson rate corresponding to the first target behavior state. Finally, the virtual object is controlled to sequentially execute the behavior events in the first behavior event queue to obtain a behavior control result. The behavior control result is then returned to terminal 400 to enable terminal 400 to output the behavior control result or perform subsequent business processing based on the behavior control result.

[0045] In some embodiments, server 200 may be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. Terminal 400 may be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, in-vehicle terminal, etc., but is not limited to these. The terminal and server can be directly or indirectly connected via wired or wireless communication, which is not limited in this embodiment.

[0046] See Figure 2 , Figure 2 This is a schematic diagram of the structure of the electronic device 40 provided in the embodiment of this application. Figure 2 The illustrated electronic device 40 may be a behavior control device, which includes at least one processor 410, a memory 450, at least one network interface 420, and a user interface 430. The various components of the behavior control device are coupled together via a bus system 440. It is understood that the bus system 440 is used to implement communication between these components. In addition to a data bus, the bus system 440 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 2 The general labeled all buses as Bus System 440.

[0047] The processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0048] User interface 430 includes one or more output devices 431 that enable the presentation of media content, including one or more speakers and one or more visual displays. User interface 430 also includes one or more input devices 432, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0049] The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state storage, hard disk drives, optical disk drives, etc. The memory 450 may optionally include one or more storage devices physically located away from the processor 410.

[0050] The memory 450 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 450 described in this application embodiment is intended to include any suitable type of memory.

[0051] In some embodiments, memory 450 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, as illustrated below.

[0052] Operating system 451 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, driver layer, etc., for implementing various basic business functions and handling hardware-based tasks;

[0053] The network communication module 452 is used to reach other electronic devices via one or more (wired or wireless) network interfaces 420, exemplary network interfaces 420 including Bluetooth, WiFi, and Universal Serial Bus (USB); the presentation module 453 is used to enable the presentation of information (e.g., a user interface for operating peripheral devices and displaying content and information) via one or more output devices 431 associated with the user interface 430 (e.g., a display screen, a speaker, etc.); the input processing module 454 is used to detect and translate one or more user inputs or interactions from one or more input devices 432.

[0054] In some embodiments, the apparatus provided in this application can be implemented in software. Figure 2 A behavior control device 455 stored in memory 450 is shown. This device can be software in the form of programs and plug-ins, and includes the following software modules: a first determining module 4551, a second determining module 4552, a generating module 4553, and a control module 4554. These modules are logically connected and can therefore be arbitrarily combined or further separated according to the functions they implement. The functions of each module will be described below.

[0055] In some other embodiments, the apparatus provided in this application can be implemented in hardware. As an example, the apparatus provided in this application can be a processor in the form of a hardware decoding processor, which is programmed to execute the behavior control method provided in this application. For example, the processor in the form of a hardware decoding processor can be one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0056] In some embodiments, the terminal or server can implement the behavior control method provided in this application embodiment by running various computer-executable instructions or computer programs. For example, computer-executable instructions can be microprogram-level commands, machine instructions, or software instructions. Computer programs can be native programs or software modules in an operating system; they can be native applications (APPs), i.e., programs that need to be installed in the operating system to run, or they can be applets that can be embedded in any APP, i.e., programs that only need to be downloaded to a browser environment to run. In summary, the aforementioned computer-executable instructions can be any form of instruction, and the aforementioned computer programs can be any form of application, module, or plugin.

[0057] The behavior control methods provided in the embodiments of this application can be executed by an electronic device, which can be a server or a terminal. That is, the behavior control methods in the embodiments of this application can be executed by a server, by a terminal, or by interaction between a server and a terminal.

[0058] See Figure 3 , Figure 3 This is an optional flowchart illustrating the behavior control method provided in this application embodiment, which will be combined with... Figure 3 The steps shown are explained below, taking the server as the execution subject of the behavior control method as an example. The method includes the following steps S101 to S104:

[0059] Step S101: Determine the current behavior state of the virtual object and the state transition matrix of the virtual object in the current time period.

[0060] In this embodiment, a virtual object refers to a digital entity driven and controlled by a program in a virtual environment (e.g., a game, a simulation system, or a metaverse). Virtual objects can exhibit behavior similar to real-world behavior patterns based on preset logic. For example, a virtual object can be a non-player character (NPC) in a game, a virtual robot in a simulation system, or an AI player such as a virtual human in a metaverse. The current behavioral state is a comprehensive reflection of the virtual object's current behavior and must conform to behavioral patterns in real-world scenarios. For example, an NPC in a game might be currently in a level-clearing state, while a virtual robot might be currently in a working state. The current time period is a specific time interval defined for analyzing the behavioral state and state transition matrix of virtual objects. Different time periods can correspond to different state transition matrices to adapt to the behavioral patterns of different time periods. Different time periods can also be flexibly set according to actual needs. For example, in a virtual environment, morning (6:00–10:00), midday (10:00–14:00), afternoon (14:00–18:00), evening (18:00–22:00), and early morning (22:00–6:00) time periods can be set. The state transition matrix is ​​a matrix representation of the probability set of a virtual object transitioning from any given behavioral state to all other possible behavioral states. In this matrix, rows represent the current behavioral state, columns represent possible target behavioral states, and each element represents the corresponding transition probability. The probability distribution in the state transition matrix references the behavioral transition patterns in real-world scenarios. For example, if a virtual object has four behavioral states, and the value in the first row, third column of the matrix is ​​0.2, it means that the probability of the virtual object transitioning from the first behavioral state to the third behavioral state is 20%.

[0061] In some embodiments, when determining the current behavioral state of a virtual object, real-time behavioral data of the virtual object can first be collected through a behavior detection module; then, the real-time behavioral data is compared and analyzed with a preset behavioral state feature library to obtain the comparison result; finally, the preset behavioral state that the virtual object currently conforms to is determined based on the comparison result, and the preset behavioral state that the virtual object currently conforms to is determined as the current behavioral state of the virtual object.

[0062] In some embodiments, the state transition matrix of a virtual object in the current time period is determined: the input is the current time, a preset time period division rule, and a record of the number of times the virtual object transitions from each behavior state to all other behavior states in the same historical time period; first, the system determines the time period to which the current time belongs and clarifies the corresponding matrix category; then, the system retrieves and summarizes the historical transition count records; then, for each initial behavior state, the system calculates the proportion of the number of times it transitions to each target behavior state to the total number of transitions for that initial state, obtaining the transition probability; finally, the system arranges the transition probabilities according to the rule that rows correspond to initial behavior states and columns correspond to target behavior states, and outputs the state transition matrix corresponding to the current time period.

[0063] First, based on the preset time period division rules, the current time period to which the current moment belongs is determined, and the matrix identifier corresponding to the current time period is then identified. Next, the state transition matrix corresponding to the matrix identifier is obtained from the preset matrix set. The preset matrix set includes different state transition matrices corresponding to different time periods. The state transition matrix for each time period can be obtained by: retrieving the transition count records of the virtual object within each historical time period and summarizing these records; then, for each initial behavior state, the proportion of the number of times the initial behavior state transitions to each target behavior state to the total number of transitions for that initial behavior state (i.e., the sum of the number of times it transitions from the initial behavior state to all target behavior states) is calculated, thus obtaining the transition probability from the initial behavior state to each target behavior state; finally, according to the rule that rows correspond to initial behavior states and columns correspond to target behavior states, the calculated transition probabilities are arranged to form the state transition matrix corresponding to each historical time period.

[0064] In some embodiments, the state transition matrix in the preset matrix set can be updated in real time according to actual conditions. The purpose is to enable the state transition matrix to more accurately reflect the behavioral state transition patterns of virtual objects over different time periods. As time goes by and virtual object behavior data accumulates, their behavior patterns may change. Real-time updates allow the state transition matrix to adapt to these changes in a timely manner. Therefore, when determining the state transition matrix for the current time period, the latest and most realistic behavioral characteristics are used, further improving the accuracy and reliability of state transition simulation and better meeting the needs of modeling and predicting virtual object behavior.

[0065] Step S102: Determine the first state transition probability distribution corresponding to the current behavior state from the state transition matrix.

[0066] In this embodiment, the first state transition probability distribution is the combination of probability data in the state transition matrix corresponding to the current behavior state of the virtual object. For example, when the NPC is in an idle state, the corresponding row probability data is {0.1, 0.6, 0.2, 0.1}, and this set of data is the current first state transition probability distribution of the NPC. The first state transition probability distribution includes multiple first transition probabilities; a first transition probability is a single probability value in the first state transition probability distribution, and the first transition probability is used to represent the probability of transitioning the behavior state of the virtual object from the current behavior state to a preset behavior state. The probability value is set with reference to the behavior selection probability in a real scene. For example, in the first state transition probability distribution corresponding to the idle state mentioned above, 0.6 is the first transition probability of transitioning from the idle state to the challenge state.

[0067] A preset behavior state is any behavior state from a preset set of behavior states. The preset set of behavior states is the sum of all possible behavior states predefined in the virtual object control system. This set covers all possible behavior patterns that a virtual object may exhibit during operation, and the behavior states within the set can be set according to actual needs. For example, in a game scenario, the set of behavior states may include a hibernation state, an idle state, a challenge state, and an end state. A hibernation state means the virtual object is completely disconnected from its virtual environment and is offline. During this period, the virtual object will not generate any behavioral activities or interact with other elements within the virtual environment, similar to a closed / resting state in a real-world scenario. An idle state means the virtual object is online, maintaining a connection with the virtual environment, but has not started or participated in any challenge-related activities. At this time, the virtual object is in a waiting, static, or ready state, similar to an idle / standby state in a real-world scenario. A challenge state refers to the state where the virtual object is attempting a challenge. The system allows you to control virtual objects to initiate challenge behaviors at a certain frequency. Based on this frequency, it can be categorized into high-activity and low-activity states. High-activity refers to a high frequency of challenges, similar to a high-intensity task processing state in a real-world scenario, while low-activity refers to a low frequency of challenges, similar to a regular task processing state in a real-world scenario. The "end state" refers to the brief transition phase after a virtual object completes a challenge, before it officially enters a dormant state. You can also control virtual objects to perform some cleanup operations related to the challenge, such as viewing results or saving data, similar to the post-task cleanup phase in a real-world scenario.

[0068] In some embodiments, determining the first state transition probability distribution corresponding to the current behavior state from the state transition matrix can be achieved in the following way: First, locate the corresponding row in the state transition matrix according to the identification information of the current behavior state; then, extract all probability data in that row; finally, output all the extracted probability data as the first state transition probability distribution corresponding to the current behavior state.

[0069] Step S103: In response to the first state transition probability distribution, the behavior state of the virtual object is transferred from the current behavior state to the first target behavior state in the behavior state set, and a first behavior event queue is generated based on the Poisson rate corresponding to the first target behavior state.

[0070] In this embodiment, the first target behavior state is the behavior state that the virtual object will transition to next, determined by a specific mechanism from a preset set of behavior states based on the first state transition probability distribution. The transition result conforms to the behavior selection logic under the same situation. For example, when a task NPC is in an idle state, according to the first state transition probability distribution, if the probability of transitioning to the challenge state is the highest, then the challenge state is the first target behavior state of the task NPC. The Poisson rate is a parameter used to quantitatively describe the average frequency of related behavior events occurring when a virtual object is in a specific behavior state per unit time. The Poisson rate can be set with reference to the average activity frequency of this type of behavior in a real-world scenario. For example, a Poisson rate of 5 times / hour corresponds to high activity in the challenge state, meaning that the virtual object will initiate an average of 5 challenge-related behavior events per hour in this state. The first behavior event queue is an ordered list of specific behavior events generated based on the Poisson rate corresponding to the first target behavior state and arranged in chronological order. The arrangement of behavior events in the first behavior event queue reflects the rhythm and coherence of the behavior. For example, when the NPC is in a high-activity challenge state, the first action event queue generated is: 9:00 for task A, 9:01 for task B, 9:02 for task C, and 9:03 for task D.

[0071] In some embodiments, after obtaining the first state transition probability distribution of the virtual object, the specific preset behavior state corresponding to each probability value in the first state transition probability distribution can be identified first, and the correspondence between probability and state can be established. Then, a random sampling algorithm can be used to assign different selection weights to each preset behavior state according to the probability of each preset behavior state. The higher the probability, the higher the selection weight. Finally, a preset behavior state is selected from each preset behavior state through random sampling, and the preset behavior state selected by random sampling is determined as the first target behavior state.

[0072] In some embodiments, generating a first behavioral event queue based on the Poisson rate corresponding to the first target behavioral state can be achieved in the following way: First, the Poisson rate parameter corresponding to the first target behavioral state can be obtained from the parameter configuration library to determine the average occurrence frequency of behavioral events per unit time; then, based on the mathematical principles and probability distribution laws of the Poisson process, combined with the time period corresponding to the first target behavioral state, the specific time of occurrence of each behavioral event within the period can be calculated; finally, the behavioral events arranged in chronological order are organized into an ordered list and output as the first behavioral event queue.

[0073] Step S104: Control the virtual object to execute the behavior events in the first behavior event queue in sequence.

[0074] In this embodiment, controlling the virtual object to sequentially execute the behavior events in the first behavior event queue can be achieved through a control module and an execution module of the virtual object. The control module sends behavior execution instructions to the virtual object sequentially according to the time order of the behavior events in the first behavior event queue. Then, after the execution module of the virtual object receives the behavior execution instructions, it executes the corresponding behavior events as required at the time point specified by the behavior execution instructions.

[0075] In some embodiments, after controlling a virtual object to sequentially execute behavioral events in the first behavioral event queue, a behavioral control result is also obtained. The behavioral control result is the final behavioral performance, effect, or outcome achieved by the virtual object after it has sequentially executed all behavioral events according to the order of the first behavioral event queue. This behavioral control result closely approximates the typical performance of the same behavioral sequence in a real-world scenario. For example, in a game scenario, an NPC successfully completes a level by executing a level-clearing event according to the queue; this result is the behavioral control result. The determination of the behavioral control result can be achieved through a recording module. The recording module can comprehensively record and evaluate the final behavioral performance, effect, or outcome achieved by the virtual object after executing all behavioral events, and the final output is the behavioral control result.

[0076] The behavior control method provided in this application firstly determines the first state transition matrix between the current behavior state and the current time period, ensuring that the state transitions of the virtual object are correlated with the time dimension. This avoids a disconnect between the behavior logic of the virtual object and the time pattern, improving the rationality of state transitions. Then, based on the first state transition probability distribution extracted from the first state transition matrix, a probabilistic decision-making basis is provided for behavior state transitions. This ensures that the transition of the virtual object from the current behavior state to the first target behavior state exhibits a natural tendency consistent with probabilistic characteristics, overcoming the mechanical limitations of fixed patterns. Next, a first behavior event queue is generated by combining the Poisson rate corresponding to the first target behavior state. This allows for precise control of the frequency of behavior events, ensuring that the activity rhythm of the virtual object in a specific state matches the characteristics of that state, avoiding behavior density imbalance. Finally, by controlling the virtual object to execute the first behavior event queue sequentially, the continuity and orderliness of the behavior event sequence are ensured, making the overall behavior of the virtual object highly consistent with the expected behavior pattern in the virtual environment, significantly enhancing the realism and immersion of the virtual object's behavior.

[0077] The behavior control method in this application embodiment will be described below in conjunction with the interaction between the terminal and the server in the behavior control system. It should be noted that the behavior control method here is implemented through interaction between the terminal and the server, and is essentially the same as the behavior control method executed by the server in the above embodiments. The only difference is that this application embodiment also describes the actions performed by the terminal during the execution of the behavior control method. Furthermore, some steps can be executed by either the terminal or the server. Therefore, for steps in this embodiment that are the same as those in the above embodiments but have different execution subjects, this embodiment is merely illustrative. In the implementation process, any execution subject can perform the steps, and this application embodiment does not limit this.

[0078] Figure 4 This is another optional flowchart illustrating the behavior control method provided in the embodiments of this application, such as... Figure 4 As shown, the method includes the following steps S201 to S210:

[0079] Step S201: The terminal receives user input for behavior control operations.

[0080] In this embodiment of the application, a behavior control application may run on the terminal. Users can perform behavior control operations on the client side of the behavior control application. The behavior control application can provide behavior control functions. Users can perform behavior control operations on the behavior control function page to trigger behavior control requests.

[0081] In some embodiments, when a user inputs a behavior control operation, they can also simultaneously input the current behavior state of the virtual object and the state transition matrix for the current time period. When the terminal receives the current behavior state and state transition matrix, a confirmation window for behavior control will pop up on the behavior control function page. After the terminal detects that the user has clicked the confirmation button, it further processes the current behavior state and state transition matrix to achieve behavior control. Alternatively, in other embodiments, the user can directly input the current behavior state and state transition matrix on the behavior control function page. Upon receiving the current behavior state and state transition matrix, the terminal can directly trigger the behavior control function to further process the current behavior state and state transition matrix to achieve behavior control.

[0082] In step S202, the terminal generates a behavior control request in response to the behavior control operation.

[0083] In this embodiment, user-input data can be encapsulated into a behavior control request. For example, the current behavior state and state transition matrix are displayed on the behavior control application's display interface. The user can select or confirm the current behavior state and state transition matrix on this display interface according to actual needs, and then the user-input current behavior state and state transition matrix can be encapsulated into a behavior control request.

[0084] In step S203, the terminal sends a behavior control request to the server.

[0085] In step S204, the server responds to the behavior control request by determining the current behavior state of the virtual object and the state transition matrix of the virtual object in the current time period.

[0086] In this embodiment of the application, the specific meaning of the step of determining the current behavior state of the virtual object and the state transition matrix of the virtual object in the current time period can be explained by referring to the description of step S101 above, and will not be repeated here.

[0087] In some embodiments, when the server receives a behavior control request from the terminal, it can directly parse the behavior control request to obtain the current behavior state of the virtual object and the state transition matrix of the virtual object in the current time period.

[0088] In other embodiments, see Figure 5 , Figure 5 The state transition matrix of the virtual object in the current time period is determined by the server in step S204, as shown in step S2041 to S2042.

[0089] Step S2041: Obtain the transition matrix corresponding to the current time period from the preset matrix set.

[0090] Step S2042: Determine the transition matrix corresponding to the current time period as the state transition matrix of the virtual object in the current time period.

[0091] In this embodiment, the preset matrix set includes transition matrices corresponding to different preset time periods. The preset matrix set is a collection of multiple pre-constructed and stored state transition matrices. For example, when the preset time periods are divided into 0:00–9:00, 9:00–13:00, 13:00–18:00, and 18:00–24:00, the preset matrix set includes transition matrices A, B, C, and D corresponding to these four time periods, respectively.

[0092] In some embodiments, obtaining the transition matrix corresponding to the current time period from the preset matrix set can be achieved in the following way: First, parse the preset time period corresponding to each transition matrix in the preset matrix set and establish a mapping relationship between the preset time period and the transition matrix; then, match the current time period with the preset time period in the mapping relationship to find the transition matrix corresponding to the preset time period that matches the current time period; finally, extract the transition matrix from the preset matrix set and output it as the transition matrix corresponding to the current time period, that is, the state transition matrix of the virtual object in the current time period.

[0093] Here, by obtaining and determining the state transition matrix corresponding to the current time period from the preset matrix set, the process of real-time matrix calculation is eliminated, the operation process is simplified, and the efficiency of determining the state transition matrix is ​​improved. At the same time, the preset transition matrix can be optimized in advance according to the needs of the scenario, ensuring the applicability and accuracy of the transition matrix, making the state transition of the virtual object in different time periods more in line with the expected pattern, and enhancing the stability and reliability of behavior control.

[0094] In some embodiments, each row of the state transition matrix corresponds to a single behavior state in the set of behavior states. That is, in this state transition matrix, all elements in each row are associated with a specific behavior state in the set of behavior states. Specifically, if the set of behavior states includes behavior states S0, S1, S2, and S3, then the first row of the matrix corresponds to S0, and each element in this row represents the transition probability from S0 to other behavior states; for example, the transition probabilities from S0 to S0, S0 to S1, S0 to S2, and S0 to S3, etc. The second row of the matrix corresponds to S1, and the elements in this row represent the probability of transitioning from S1 to other behavior states, and so on. By using this method of one row corresponding to one initial behavior state, the probability distribution of each behavior state transitioning to other states can be clearly presented.

[0095] Step S205: The server determines the row vector in the state transition matrix corresponding to the index of the current behavior state in the behavior state set.

[0096] In step S206, the server determines the row vector as the first state transition probability distribution corresponding to the current behavior state.

[0097] In this embodiment, the index is a sequence number used to identify the position of a behavior state within the set of behavior states. This index uniquely corresponds to a specific behavior state within the set. For example, when the set of behavior states is [dormant, idle, challenge, end], the index of the dormant state is S0, the index of the idle state is S1, the index of the challenge state is S2, and the index of the end state is S3. A row vector is a vector composed of all elements of a row in the state transition matrix arranged sequentially. This row vector contains the probability of the current behavior state transitioning to any other behavior state. For example, if the elements in the second row of a state transition matrix are [0.1, 0.2, 0.5, 0.2], the row vector formed by these elements represents the transition probability of the corresponding behavior state.

[0098] In some embodiments, determining the row vector corresponding to the index in the state transition matrix based on the index of the current behavior state in the behavior state set can be achieved in the following way: First, find the current behavior state in the behavior state set and obtain its corresponding index; then, locate the corresponding row in the state transition matrix based on the index; finally, extract all elements of the row to form a row vector, and output the row vector corresponding to the index.

[0099] After obtaining the row vectors corresponding to the indices in the state transition matrix, we can verify whether the sum of the probability values ​​in the row vectors is 1, ensuring the validity of the probability distribution. Once valid, the row vectors are defined as the first state transition probability distribution corresponding to the current behavioral state.

[0100] Here, by using index-based row vector location to determine the first state transition probability distribution, the required probability information can be extracted quickly and accurately from the state transition matrix, simplifying the determination process and improving efficiency. At the same time, the validity of the probability distribution is ensured by verifying the probability sum and other methods, making the acquisition of the virtual object's state transition probability more reliable, thereby enhancing the accuracy and stability of behavior control.

[0101] In step S207, the server responds by transferring the behavior state of the virtual object from the current behavior state to the first target behavior state in the behavior state set based on the first state transition probability distribution, and generates a first behavior event queue based on the Poisson rate corresponding to the first target behavior state.

[0102] In this embodiment of the application, the specific meaning of the step of generating a first behavioral event queue based on the Poisson rate corresponding to the first target behavioral state in the set of behavioral states in response to the transition of the behavior state of the virtual object from the current behavior state to the first target behavior state based on the first state transition probability distribution can be found in the description of step S103 above, and will not be repeated here.

[0103] In some embodiments, the first behavior event queue includes M behavior events; M is an integer greater than 1. The server generates the first behavior event queue based on the Poisson rate corresponding to the first target behavior state, which can be achieved in the following way: In response to the Poisson rate corresponding to the first target behavior state being greater than a preset value, iteration i performs the following processing: First, based on the Poisson rate, determine the time interval of the i-th behavior event to be generated corresponding to the first target behavior state; where 1≤i≤M; then, based on the time interval and the current time, determine the occurrence time of the i-th behavior event to be generated; finally, based on the occurrence time, generate the first behavior event queue.

[0104] In this embodiment, the preset value is a baseline value used to determine whether to initiate the behavior event generation process, specifically 0. It serves as a reference to measure whether the Poisson rate corresponding to the first target behavior state meets the conditions for generating a behavior event. When the Poisson rate is greater than the preset value of 0, it means that a behavior event needs to be generated in this state; conversely, when the Poisson rate is equal to 0, it indicates that no behavior event will be generated in this state. The i-th behavior event to be generated is the specific behavior event that needs to be generated in the i-th iteration of generating the first behavior event queue. The i-th behavior event to be generated is an event arranged sequentially in the queue. For example, when generating a queue containing 5 behavior events, the 3rd behavior event to be generated might be a successful completion of a level. The time interval is the time difference between the i-th behavior event to be generated and the previous generated behavior event. This time interval is calculated from the Poisson rate and reflects the interval pattern of event generation. For example, when the Poisson rate is 10 times / hour, the time interval of the i-th behavior event to be generated could be 5 minutes. The current time is the system time when the i-th behavior event to be generated is generated, and the current time is the baseline for calculating the event occurrence time. The occurrence time is the specific time when the i-th planned behavioral event to be generated is scheduled to execute. It is calculated from the time interval and the current time. For example, assuming the time interval is 30 minutes and the current time is 8:00:00, then the occurrence time of the i-th planned behavioral event is 8:30:00.

[0105] The first action event queue is a set of M action events arranged in chronological order of their occurrence. The first action event queue contains all action events to be executed in the first target action state. For example, when M=4, the first action event queue may be [8:00:00 first attempt, 8:30:00 second attempt, 9:00:00 third attempt, 9:30:00 fourth attempt].

[0106] In some embodiments, for the i-th iteration, the time interval of the i-th behavioral event to be generated corresponding to the first target behavioral state is determined based on the Poisson rate. This can be achieved as follows: First, a random value conforming to the probability distribution model corresponding to the Poisson rate is generated; then, the random value is converted into a time interval, in seconds or minutes; finally, this time interval is determined as the time interval of the i-th behavioral event to be generated. The probability distribution model refers to the exponential distribution model, because the time interval between adjacent events in the Poisson process follows an exponential distribution, and its probability density function is f(t)=λe^(-λt))(t≥0), where λ is the Poisson rate and t is the time interval.

[0107] For example, if the Poisson rate λ corresponding to the first target behavior state is 20 times / hour, when determining the time interval of the i-th behavior event to be generated in the i-th iteration, a random value conforming to the exponential distribution model is first generated. This can be generated using the formula t = -ln(U) / λ, where U is a uniformly random number on [0,1). Assuming the generated U is 0.3, then t = -ln(0.3) / 20 ≈ (-(-1.204)) / 20 ≈ 1.204 / 20 ≈ 0.0602 hours. This is then converted to minutes: 0.0602 × 60 ≈ 3.61 minutes. Therefore, the time interval is approximately 3.61 minutes, meaning the interval between the i-th behavior event and the previous event is approximately 3.61 minutes.

[0108] In some embodiments, for the i-th iteration, the occurrence time of the i-th behavioral event to be generated is determined based on the time interval and the current time. This can be achieved by first converting the current time into a computable timestamp format; then, adding the timestamp corresponding to the time interval to the timestamp of the current time; and finally, converting the added timestamp into a specific time format and outputting it as the occurrence time of the i-th behavioral event to be generated.

[0109] In some embodiments, in response to the Poisson rate corresponding to the first target behavior state being greater than a preset value, the server starts iterating i (1≤i≤M) to execute the generation process: for each i, the time interval of the i-th behavior event to be generated is first determined based on the Poisson rate, and then the occurrence time of the event is calculated based on the time interval and the current time, and an event containing the occurrence time and the corresponding behavior is generated; after completing M iterations, the M generated behavior events are arranged in chronological order of occurrence time to form the first behavior event queue.

[0110] In some embodiments, see Figure 6 , Figure 6 The server generates a first-action event queue based on the occurrence time, which can also be achieved through the following steps S601 to S603:

[0111] Step S601: In response to the determination based on the occurrence time that the i-th behavior event to be generated has not reached the preset queue termination condition, generate the i-th behavior event corresponding to the first target behavior state.

[0112] In this embodiment, the preset queue termination condition is a pre-defined standard for determining whether to stop generating behavioral events, thus deciding when the behavioral event queue will no longer add new events. When the queue termination condition is met, the generation of new behavioral events stops, thereby preventing the queue from extending indefinitely and ensuring that the queue length or event coverage time is within a reasonable range. The i-th behavioral event is a specific behavioral instruction generated in the i-th iteration that corresponds to the first target behavioral state. The i-th behavioral event is a constituent unit of the first behavioral event queue.

[0113] In some embodiments, generating the i-th behavior event corresponding to the first target behavior state can be achieved in the following way: First, the occurrence time of the i-th behavior event to be generated is compared with a preset maximum time threshold, or the number of generated behavior events is compared with the maximum event threshold to determine whether the queue termination condition has not been met, that is, whether the difference between the occurrence time and the initial time is less than the maximum time threshold, or whether the number of generated behavior events is less than the maximum event threshold; then, if it is confirmed that the queue termination condition has not been met, the corresponding i-th behavior event can be generated according to the attributes of the first target behavior state (such as behavior type and behavior characteristics).

[0114] Step S602: After associating the i-th behavior event with the occurrence time, add it to the event queue and update the current time to the occurrence time.

[0115] In this embodiment, association is the operation of establishing a correspondence between the i-th behavioral event and its occurrence time. Adding is the operation of placing the i-th behavioral event associated with its occurrence time into the event queue. First, the i-th behavioral event can be bound to its occurrence time to form associated data; then, the associated data is appended to the end of the behavioral event queue; finally, the current time value is replaced with the occurrence time value, and the updated behavioral event queue and the current time are output.

[0116] Step S603: In response to the determination that the i-th behavior event to be generated has reached the queue termination condition based on the occurrence time, the generation of the i-th behavior event is prohibited, and the queue of generated behavior events is determined as the first behavior event queue.

[0117] In this embodiment, the generated behavior event queue is a collection of all generated and added behavior events before the queue termination condition is met. For example, if a virtual object generates 12 behavior events while in a challenge state and then reaches the termination condition, the generated behavior event queue is the collection of these 12 behavior events. When the i-th behavior event to be generated reaches the queue termination condition based on its occurrence time, the behavior event generation process is stopped, i.e., the generation of the i-th behavior event is prohibited, and the previously generated behavior event queue is determined as the first behavior event queue, which is then output.

[0118] Here, by combining the occurrence time and the queue termination condition to generate the first behavioral event queue, the number and time range of events can be precisely controlled for different behavioral states, avoiding event redundancy or timeout; associating behavioral events with the occurrence time and adding them in an orderly manner ensures the timeliness of the queue and the accuracy of execution; the overall process makes the queue generation adaptable to the characteristics of each state, making the behavior of virtual objects more in line with the scene logic, and improving the rationality and immersion of behavior control in the game environment.

[0119] Here, the prerequisites for generating behavioral events are clearly defined by preset values, which can avoid meaningless calculations in a state where there is no behavioral requirement and save system resources. At the same time, generating multiple behavioral events iteratively according to the Poisson rate and determining their occurrence time can make the behavioral event queue conform to the behavioral rhythm and pattern of this state, accurately simulate the continuous behavior of virtual objects in the first target behavioral state, improve the realism and accuracy of behavioral simulation, and provide a reliable basis for subsequent behavior scheduling, display and other links.

[0120] In some embodiments, see Figure 7 , Figure 7 It is shown that after step S207, the server can also perform the following steps S701 to S703:

[0121] Step S701: Determine the second state transition probability distribution corresponding to the first target behavior state from the state transition matrix.

[0122] In this embodiment, the second state transition probability distribution includes multiple second transition probabilities; the second transition probability refers to the probability of transitioning the behavior state of the virtual object from the first target behavior state to a preset behavior state. The second state transition probability distribution is the row vector in the state transition matrix corresponding to the first target behavior state. For the specific meaning and implementation of determining the second state transition probability distribution corresponding to the first target behavior state from the state transition matrix, please refer to step S102 above, and it will not be repeated here.

[0123] Step S702: In response to the second state transition probability distribution, the behavior state of the virtual object is transferred from the first target behavior state to the second target behavior state in the behavior state set, and a second behavior event queue is generated based on the Poisson rate corresponding to the second target behavior state.

[0124] In this embodiment, the second target behavior state is the behavior state that the virtual object will transition to from the first target behavior state, determined from a preset behavior state set based on a second state transition probability distribution. For example, when the first target behavior state is a dormant state, the second target behavior state may be an idle state. The second behavior event queue is an ordered list of a series of behavior events generated based on the Poisson rate corresponding to the second target behavior state and sorted by occurrence time. The specific meaning and implementation details of generating the second behavior event queue based on the Poisson rate corresponding to the second target behavior state in response to the transition of the virtual object's behavior state from the first target behavior state to the second target behavior state in the behavior state set based on the second state transition probability distribution can be found in step S103 above, and will not be repeated here.

[0125] Step S703: Add the second-line event queue to the first-line event queue according to the occurrence time of each second-line event in the second-line event queue.

[0126] In this embodiment of the application, after generating the first behavioral event queue and the second behavioral event queue, the occurrence time of all behavioral events in the second behavioral event queue can be determined first; then these behavioral events can be inserted into the corresponding positions of the first behavioral event queue according to their occurrence time to maintain overall order; finally, the merged first behavioral event queue is output.

[0127] Here, by generating a second state transition probability distribution and determining the second target behavior state, a second behavior event queue is generated and integrated into the first queue according to time, making the behavior state transition of virtual objects more coherent and the behavior event coverage more comprehensive. Adding queues according to time ensures the rationality of the timing of event execution, making the behavior sequence of virtual objects closer to the laws of nature, improving the smoothness of behavior and the realism of the scene, and enhancing the interactive experience of the virtual environment.

[0128] In step S208, the server controls the virtual object to execute the behavior events in the first behavior event queue in sequence, and obtains the behavior control result.

[0129] In this embodiment of the application, the specific meaning of the step of controlling the virtual object to execute the behavior events in the first behavior event queue in sequence to obtain the behavior control result can be explained in the description of step S104 above, and will not be repeated here.

[0130] In some embodiments, see Figure 8 , Figure 8 The step S208, in which the server controls the virtual object to sequentially execute the behavior events in the first behavior event queue, can also be achieved through the following steps S2081 to S2082:

[0131] Step S2081: Determine the occurrence time of each action event in the first action event queue.

[0132] Step S2082: According to the occurrence time, control the virtual object to execute each action event in the first action event queue in sequence.

[0133] In this embodiment of the application, the occurrence time of each behavior event in the first behavior event queue can be determined in the following way: First, traverse all behavior events in the first behavior event queue; then, extract the corresponding time information from each behavior event; finally, organize the extracted time information according to the order of behavior events in the queue, and output it as the occurrence time of each behavior event in the first behavior event queue.

[0134] After determining the occurrence time of each behavior event in the first behavior event queue, the current system time can be obtained. Then, the current system time is compared with the occurrence time of each behavior event. When the current system time reaches the occurrence time of a certain behavior event, an instruction to execute that behavior event is sent to the virtual object. The virtual object then receives the instruction and executes the corresponding behavior event. Finally, after the virtual object completes the current behavior event, it continues to detect the occurrence time of the next behavior event and repeats the above process until all behavior events are executed. The output is the result of the virtual object executing the first behavior event queue.

[0135] Here, by determining the occurrence time of each behavioral event and controlling the virtual objects to execute sequentially according to that time, the execution order of the behavioral events is ensured to be consistent with the time schedule, avoiding confusion or timing errors in event execution; this gives the behavior of the virtual objects a clear temporal rhythm, enhances the orderliness and coherence of the behavior, and further improves the rationality of the virtual object's behavior and the immersiveness of the virtual environment.

[0136] In some embodiments, see Figure 9 , Figure 9 The step S208, in which the server controls the virtual object to sequentially execute the behavior events in the first behavior event queue, can also be achieved through the following steps S2083 to S2084:

[0137] Step S2083: In response to the behavior event executed by the control virtual object being the behavior event corresponding to the third target behavior state, determine the behavior event result of the behavior event according to the preset behavior event probability.

[0138] In this embodiment, the third target behavior state is used to characterize the virtual object being in a challenge state. The preset behavior event probability is a pre-set probability value for the occurrence of a successful or failed behavior event corresponding to the third target behavior state, belonging to the Bernoulli event probability. This behavior event probability is used to determine the probability of a behavior event succeeding or failing. For example, when the virtual object performs an obstacle-breaking event in the challenge state, the preset behavior event probability might be a 60% success probability and a 40% failure probability. The behavior event result refers to the specific outcome produced by the virtual object after performing a specific behavior event, used to represent the effectiveness of the behavior event. When the virtual object performs a challenge behavior event, if the behavior event probability determines success, the behavior event result is successful completion of the challenge, possibly accompanied by level progress updates and rewards; if the behavior event probability determines failure, the behavior event result is challenge failure, possibly accompanied by retaining the current level progress and recording the number of failures.

[0139] In some embodiments, it can first be determined whether the currently executed behavior event is associated with the third target behavior status identifier; if associated, the preset behavior event probability corresponding to the behavior event is obtained, and the success probability and failure probability are determined; then, a random number generation algorithm is used to generate a random value between 0 and 1, and the random value is compared with the success probability. If the random value is less than or equal to the success probability, it corresponds to a success result; otherwise, it corresponds to a failure result; finally, the determined success or failure result is taken as the behavior event result of the behavior event and output as the behavior event result.

[0140] Step S2084: Execute the behavior event based on the result of the behavior event.

[0141] In this embodiment, executing a behavior event based on its result can be achieved in the following way: First, the result of the behavior event is parsed as success or failure; then, based on the result type, the success execution module or failure execution module corresponding to the currently executed behavior event is called; next, if the success execution module is called, the success execution module drives the virtual object to perform an action matching the success result and triggers the corresponding effect; if the failure execution module is called, the failure execution module drives the virtual object to perform an action matching the failure result and triggers the corresponding effect; finally, the execution process, result type, and resulting effect of the behavior event are recorded, and the output is the execution result of the behavior event.

[0142] Here, by determining the success or failure result based on the preset Bernoulli event probability for the behavioral events in the third target behavioral state, the behavioral events in the challenge state exhibit a clear binary random characteristic, avoiding the uniformity of the result; making the behavior of virtual objects in the challenge process more unpredictable, enhancing the challenge and fun of the behavior, and improving the interactive depth of the virtual environment and the user's participation experience.

[0143] In some embodiments, see Figure 10 , Figure 10 It is shown that after step S2084, the server can also perform the following steps S1001 to S1002:

[0144] Step S1001: Update the index values ​​of the virtual object in the virtual environment based on the behavioral event results.

[0145] In this embodiment, updating is an operation whereby the system modifies or adjusts the relevant data of the virtual object in the virtual environment based on the results of behavioral events, so that the data can reflect the latest state of the virtual object in real time. The indicator value is a specific numerical value used to measure the performance or state of the virtual object in the virtual environment. This indicator value can intuitively reflect the behavioral results or state characteristics of the virtual object. For example, when a virtual object successfully completes a level, its progress is updated; when a virtual object fails a level, its failure count is updated; when a virtual object is in a dormant state, if its energy value changes, the energy data will also be updated.

[0146] In some embodiments, the behavior event result can first be parsed to determine whether it is successful or not; then, if the behavior event result is successful, the type of indicator value to be updated and the corresponding update rule are determined; next, the indicator value is modified according to the update rule, such as increasing the progress of the level; if the result is failure, the corresponding indicator value type and update rule are determined, and the indicator value is modified according to the rule, such as increasing the number of failures; finally, the updated indicator value is saved and the output is the updated indicator value of the virtual object.

[0147] Step S1002: In response to the behavior event reaching the preset behavior termination condition, the behavior state of the virtual object is transferred from the third target behavior state to the fourth target behavior state.

[0148] In this embodiment, the preset behavior termination condition is a pre-defined standard used to determine whether the behavior event stops in the third target behavior state. This behavior termination condition determines when the virtual object ends its challenge state. For example, the preset behavior termination condition could be three consecutive failures, or a total challenge time of 10 minutes. The fourth target behavior state is used to characterize the virtual object in the virtual environment as being in a behavior-ended state.

[0149] In some embodiments, in response to a behavior event reaching a preset behavior termination condition, the behavior state of a virtual object is transferred from a third target behavior state to a fourth target behavior state. This can be achieved in the following way: First, relevant data, such as the number of failures or the total time taken to complete the level, is extracted from the execution record of the behavior event. Then, the extracted data is compared with the preset behavior termination condition to determine whether the condition is met. Next, if the condition is met, a state transition instruction is triggered, which contains information about the transition from the third target behavior state to the fourth target behavior state. After that, the instruction is received and executed to change the behavior state identifier of the virtual object. Finally, it is confirmed that the behavior state of the virtual object has been transferred to the fourth target behavior state, and the current fourth target behavior state of the virtual object is output.

[0150] Here, by updating the indicator values ​​based on the results of behavioral events, the performance data of virtual objects is reflected in real time and accurately, providing a reliable basis for subsequent behavioral analysis; state transitions are achieved through preset behavior termination conditions, ensuring that the behavior of virtual objects in the challenge state does not continue indefinitely, making the behavior process more standardized and complete; the overall process makes the behavioral state transitions of virtual objects more reasonable, the indicator data more accurate, and improves the logic of the virtual environment and the smoothness of the user experience.

[0151] Step S209: The server sends the behavior control result to the terminal.

[0152] Step S210: The terminal outputs the behavior control results.

[0153] In this embodiment, during behavior control, obtaining the state transition matrix corresponding to the current time period from a preset matrix set not only simplifies the process and improves efficiency and accuracy, but also makes the state transitions of virtual objects more consistent with time patterns. Based on this, using index positioning to extract the first state transition probability distribution further ensures reliability and accuracy, making the transitions of virtual object behavior states more natural. Simultaneously, generating a behavior event queue at a Poisson rate based on preset values ​​saves resources, improves simulation realism, and makes the behavior rhythm of virtual objects in specific states more reasonable. Furthermore, to enhance behavioral coherence and scene realism, a second state transition probability distribution is generated. By integrating queues, the behavior sequence of virtual objects becomes smoother. Then, by controlling the execution of behavioral events according to their occurrence time, the correct timing and rationality of behavior are ensured, making the behavior of virtual objects more organized. For behavioral events in the challenge state, the results are determined based on Bernoulli probability, increasing the fun and depth of interaction, making the challenge behavior of virtual objects more attractive. Finally, updating indicator values ​​based on the results and implementing state transitions according to conditions ensures behavioral norms and data accuracy, making the behavior process of virtual objects more complete. All of the above improves the logic, immersion, and user experience of the virtual environment, making the behavior of virtual objects more intelligent, realistic, and in line with expectations.

[0154] This application's embodiments can be applied to various scenarios, such as games, simulation systems, or metaverses. Taking a game scenario as an example, the server includes a multimodal data acquisition module, a behavior state classification module, a preset matrix storage module, a behavior decision module, a time calculation module, and a preset probability calculation module. First, the server can activate the multimodal data acquisition module to receive in-game data of virtual objects and input the cleaned in-game data into the behavior state classification module, outputting the current behavior state of the virtual objects. Simultaneously, the server retrieves the transition matrix corresponding to the current time period from the preset matrix set in the preset matrix storage module as the state transition matrix for the current time period. Then, the server uses the index mapping table of the behavior state set to locate the corresponding row vector in the state transition matrix according to the index of the current behavior state, and determines this row vector as the first state transition probability distribution. Based on this first state transition probability distribution, after the server's behavior decision module selects the first target behavior state, if the Poisson rate corresponding to the first target behavior state is greater than a preset value, the server immediately starts an iteration timer and simultaneously transmits the Poisson rate to the time calculation module. The time calculation module calculates the time interval of each behavior event to be generated based on the Poisson rate, and then combines it with the current time to determine the occurrence time of each behavior event to be generated. The server then matches the occurrence time of each behavioral event to be generated with the preset queue termination condition. If the queue termination condition is not met, a behavioral event corresponding to the first target behavioral state is generated, the behavioral event is associated with the corresponding occurrence time, and the associated behavioral event is added to the temporary event queue. The current time is then updated to the occurrence time of the behavioral event. If the queue termination condition is met, the generation of new behavioral events stops, and the temporary event queue is determined as the final first behavioral event queue.

[0155] Next, after determining the first action event queue, the server accesses the state transition matrix for the current time period. Based on the index of the first target action state in the action state set, it locates the corresponding row vector in the matrix to extract the second state transition probability distribution corresponding to the first target action state. The server inputs the extracted second state transition probability distribution into the action decision module, which evaluates and filters the transition probabilities in the distribution based on the current game scene requirements and the virtual object task progress, ultimately determining the second target action state. Subsequently, the server obtains the Poisson rate corresponding to the second target action state, calls the time calculation module and the event generation module, and, referring to the logic for generating the first action event queue, calculates the time interval and occurrence time of each second action event to be generated, generating a second action event queue containing multiple second action events. Finally, the server reads the occurrence time of each second action event in the second action event queue and inserts these second action events into the first action event queue in chronological order of occurrence, completing the merging of the two queues.

[0156] During the behavior execution phase, the server sequentially reads the occurrence time of each behavior event from the merged first behavior event queue and sends behavior control instructions to the virtual object in chronological order, driving the virtual object to execute the corresponding behavior event. Furthermore, if the server detects that the currently executing behavior event is associated with a third target behavior state representing the level completion status, it immediately retrieves the game's preset behavior event probability parameters and passes them to the preset probability calculation module. The preset probability calculation module calculates and outputs the specific result of the behavior event. Based on the calculated behavior event result, the server adjusts the virtual object's execution action to ensure the event is executed accurately according to the result. After execution, the server synchronizes the behavior event result to the virtual environment indicator database, updating key indicator values ​​such as the virtual object's score, level, and item quantity. Simultaneously, the server checks whether the current behavior event meets the preset behavior termination condition. If it does, it triggers a state transition instruction, switching the virtual object's behavior state from the third target behavior state representing the level completion status to the fourth target behavior state representing the end of the behavior.

[0157] Here, this embodiment of the application achieves full-process control of virtual object behavior through multi-module collaboration, balancing the rationality of challenge events and dynamic updates of indicators during behavior execution. This allows virtual objects to flexibly adjust their strategies according to the scenario, just like real players, with behavioral responses synchronized and natural with real player operations. This not only enhances the stability of game operation but also ensures that the behavior of virtual objects is consistent with the interaction logic of real players, and indicator updates are synchronized to the game interface in real time, thereby effectively improving the activity and competitive spirit of real players in the game.

[0158] The following will describe an exemplary application of the embodiments of this application in a real-world application scenario.

[0159] This application provides a game agent design method based on Markov-modulated Poisson processes. This method models AI player behavior using Markov-modulated Poisson processes, making AI players more human-like and statistically closer to real human behavior. This results in more natural and realistic behavioral trajectories in the leaderboard, incentivizing the activity and competitive spirit of real players. The specific AI behavior modeling process based on Markov-modulated Poisson processes is as follows:

[0160] First, define a set of AI player behavior states, setting the finite set S = {S0, S1, S2, S3}, where each state represents a different stage of the AI ​​player's behavior. S0 is the dormant state, indicating that the AI ​​player is completely offline and does not generate any behavior; S1 is the idle state, indicating that the AI ​​player is online but has not started playing; S2 is the playing state, indicating that the AI ​​player is attempting to play at a certain frequency; and S3 is the end state, indicating that the AI ​​player is preparing to go offline after completing a playthrough and is briefly offline. These states are used to simulate the human-like behavior pattern of the AI ​​player from logging in to playing, then logging off to rest, and then logging back in. S0 is mainly in an absorptive state at night (e.g., 0:00-9:00), while S1, S2, and S3 cycle through during the day. The state transitions can use a time-varying matrix to match the different behavioral rhythms of the morning and evening. For example, the probability of logging in is higher in the morning, and the playthrough time is more concentrated in the afternoon.

[0161] Then, construct the state transition Markov chain. A time-varying state transition matrix P(t) can be defined for the above state set S, with matrix elements P... ij (t) represents the state S at time t. i Transfer to S j The probability matrix satisfies the Markov property, which means that the transition of the current state depends only on the current state and not on the historical state sequence.

[0162] The application process of the Markov transition matrix is ​​as follows: First, set the current state S. i The system records the current state of the AI ​​player, such as during a level progression; then it obtains the state transition matrix P corresponding to the current time period. t The matrix can be selected from a preset set based on the system simulation time. For example, P can be used for the time period from 0:00 to 9:00. night Use P during the period from 9:00 to 13:00 morning Use P during the period from 13:00 to 18:00 afternoon Use P during the period from 18:00 to 24:00 evening The state transition matrix P corresponding to the morning period. morning For example, its matrix is This means that during the morning hours, if the AI ​​player is currently in a dormant state, it is highly likely to remain in a dormant state (S0→S0=0.80), but there is a small probability that it will transition to an idle state (S0→S1=0.20). If the current state is idle, it is relatively likely to remain idle, but may also start a challenge or prepare to log off. If the current state is in a challenge state, it is relatively likely to continue in the challenge state, or it may transition from the challenge state to the end state. If the current state is the end state, after the challenge ends, the AI ​​player has a high probability of logging off directly or returning to idle. Next, the next state is sampled based on the probability distribution of the i-th row of the state transition matrix, starting from the current state S... i The corresponding row [p i,0 ,p i,1 ,...,p i,N-1 In the diagram, the next state S is obtained by random sampling using a discrete distribution. j If the current state is S1, read [p] 1,0 ,p 1,1 ,p 1,2 ,p 1,3 The system samples the data according to weights. Finally, it updates the state and continues generating events. If the state remains unchanged, it continues to generate Poisson events at the Poisson rate corresponding to the current state. If the state changes, it switches the behavior logic module. If the state transitions to S0, the AI ​​player enters an inactive state, and events stop until the next wake-up.

[0163] Then, the Poisson rate λ for generating Poisson events in each state is assigned, and the rate function λ is defined. i This indicates that the AI ​​player is in state S. i The Poisson rate λ is the average frequency of behavioral events (such as level attempts). For example, S0 is 0 times / hour, with no events occurring; S1 is 0.1 times / hour, with occasional activity; S2 is 20 times / hour, with active level completion (once every 3 minutes on average), and 30 times / hour during high activity; S3 is 0.2 times / hour, with brief pauses after level completion. The Poisson rate λ is a key parameter for controlling the frequency of behavioral attempts per unit time and can be flexibly adjusted. The sources and methods for setting the Poisson rate λ include: empirical modeling, analyzing real player behavior logs, and statistically estimating the average number of events per unit time under different states; manual setting, setting different values ​​to simulate different levels of activity, such as 30 times / hour for high activity, 20 times / hour for normal activity, and 10 times / hour for low activity; dynamic control, which can be set as a function of variables such as state, time period, and win rate, λ = f(state, time period, historical behavior); and individual difference simulation, setting different λ values ​​for different AI players to reflect differences in behavioral styles and improve personalized realism.

[0164] Next, the agent state and time are initialized. The current state (e.g., S0) is initialized for each AI player, and an initial timestamp t0 is set. Then, the state sequence and event sequence are simulated, and the following operations are performed repeatedly until the simulation ends: Current state determination, let the current state be S... i Find the corresponding λ i If λ i >0, the time interval Δt ~ Exp(λ) for generating the next event according to the exponential distribution. i ), and calculate the time t of the next event. next =t current +Δt,t current Indicates the current event time; when the state allows, generates a behavioral event.

[0165] When generating a level attempt event, it can be based on the current level's win rate p. n (where p is the probability of success and n is the level number), a Bernoulli event is generated to determine success or failure. If the exit condition is met (too many failures or exceeding the time limit), the state transitions; otherwise, a new attempt is generated. The processing logic for each attempt is as follows: generate a binary result of success or failure; if successful, update the leaderboard status and AI behavior log; if unsuccessful, record the number of failures. When there are several consecutive failures or the total time reaches the limit (e.g., 40 minutes), the system transitions to S3 to prepare for offline.

[0166] Determine whether to transition states based on the current time t and the state transition matrix P(t). If a transition is needed, update the state to S. j If it enters an absorption state (such as a dormant state), it waits for the next wake-up. Then, it records the behavior trajectory and leaderboard data, logging the simulated event sequence (behavior timestamps, types, and success / failure status, etc.). The leaderboard display data is updated each time a level is successfully completed; real-time display of behavior or periodic batch updates of the front-end leaderboard can be configured as needed.

[0167] Furthermore, the method allows for parameter adjustment and customized behavior, enabling parameter tuning to adapt to different scenarios and strategy requirements, including the Poisson rate λ for each state. i State transition matrix P(t), current level win rate p nThe system also includes exit conditions after failure (such as maximum number of failures or longest playtime). It supports generating AI individuals with different behavioral patterns through configuration parameter templates, enhancing the diversity and naturalness of the leaderboard. Customizable parameters include: sleep period (daily inactivity period, default 0:00-9:00); average playtime (total playtime per day, such as 40 minutes); probability function for each level (a win rate model that dynamically changes with level depth); state transition matrix (dynamically adjusted by time period to simulate real user habits); and multi-AI individual differences (adding individualized behaviors to different AI players, such as reversed day / night cycles, irritability, or immediate offline upon failure).

[0168] Finally, the system deployment and operation process: This AI behavior system can be deployed as a backend service on the game server. The operation process is as follows: Initialization phase: Every day at midnight, a state transition trajectory and event flow (which can be cached as an event queue) are generated for each AI individual; Scheduling phase: Triggers are scheduled according to event timestamps, and an action is executed once for each trigger; Recording and display: Behavior results are written to the leaderboard and user information in real time for client display; Dynamic feedback mechanism: The AI ​​behavior strategy is adjusted based on the overall performance of real players that day, for example, the win rate is slightly higher than the average by 2%-5% to enhance incentive.

[0169] This application's embodiments model the online and level-clearing behavior of AI players through Markov-modulated Poisson processes, making them closer to real players in terms of behavioral rhythm, time distribution, and success rate, thus reducing the risk of being identified as such. Simultaneously, it enhances the activity and competitiveness of the leaderboard system, effectively filling gaps in the leaderboard, stimulating the competitive spirit of real players, and increasing engagement and game duration. It also possesses high parameterization and controllability, supporting the flexible generation of AI players with different activity styles by adjusting Markov transition probabilities, Poisson λ values, and win rate parameters, facilitating precise deployment and debugging by the operations team. Furthermore, it benefits long-term retention optimization in the later stages of the game; during periods of player churn, AI players can fill in the gaps to create a sense of excitement, helping to extend the game's lifespan. The simulated behavior can be further enriched by adding additional behavioral states (e.g., waiting for stamina to recover, temporary network disconnection, and receiving pop-ups), demonstrating strong scalability and significantly improving the credibility of AI player behavior.

[0170] It is understood that in the embodiments of this application, if the content involving user information, such as state transition matrix and first behavior event queue, involves data related to user information or enterprise information, when the embodiments of this application are applied to specific products or technologies, it is necessary to obtain user permission or consent, or to obfuscate this information to eliminate the correspondence between this information and the user; and the collection and processing of related data should strictly comply with the requirements of relevant national laws and regulations when applied in practice, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of laws and regulations and the authorization of the personal information subject.

[0171] The following description continues to illustrate the exemplary structure of the behavior control device 455 provided in the embodiments of this application as a software module. In some embodiments, such as Figure 2 As shown, the software modules stored in the behavior control device 455 in the memory 450 may include: a first determining module 4551, used to determine the current behavior state of the virtual object and the state transition matrix of the virtual object in the current time period; a second determining module 4552, used to determine a first state transition probability distribution corresponding to the current behavior state from the state transition matrix; the first state transition probability distribution includes multiple first transition probabilities; the first transition probability refers to the probability of transitioning the behavior state of the virtual object from the current behavior state to a preset behavior state, the preset behavior state being any behavior state in a preset set of behavior states; a generating module 4553, used to, in response to the first state transition probability distribution, transition the behavior state of the virtual object from the current behavior state to a first target behavior state in the set of behavior states, and generate a first behavior event queue based on the Poisson rate corresponding to the first target behavior state; and a control module 4554, used to control the virtual object to sequentially execute the behavior events in the first behavior event queue.

[0172] In some embodiments, the first determining module 4551 is further configured to: obtain a transition matrix corresponding to the current time period from a preset matrix set; the preset matrix set includes transition matrices corresponding to different preset time periods; and determine the transition matrix corresponding to the current time period as the state transition matrix of the virtual object in the current time period.

[0173] In some embodiments, each row of the state transition matrix corresponds to a behavior state in the set of behavior states; the second determining module 4552 is further configured to: determine the row vector in the state transition matrix corresponding to the index of the current behavior state in the set of behavior states; and determine the row vector as the first state transition probability distribution corresponding to the current behavior state.

[0174] In some embodiments, the first behavior event queue includes M behavior events; M is an integer greater than 1; the generation module 4553 is further configured to: in response to the Poisson rate corresponding to the first target behavior state being greater than a preset value, iterate i and perform the following processing: based on the Poisson rate, determine the time interval of the i-th behavior event to be generated corresponding to the first target behavior state; where 1≤i≤M; based on the time interval and the current time, determine the occurrence time of the i-th behavior event to be generated; based on the occurrence time, generate the first behavior event queue.

[0175] In some embodiments, the generation module 4553 is further configured to: in response to determining that the i-th behavior event to be generated has not reached a preset queue termination condition based on the occurrence time, generate the i-th behavior event corresponding to the first target behavior state; associate the i-th behavior event with the occurrence time, add it to the event queue, and update the current time to the occurrence time; in response to determining that the i-th behavior event to be generated has reached the queue termination condition based on the occurrence time, prohibit the generation of the i-th behavior event, and determine the queue of generated behavior events as the first behavior event queue.

[0176] In some embodiments, the control module 4554 is further configured to: determine the occurrence time of each behavior event in the first behavior event queue; and control the virtual object to execute each behavior event in the first behavior event queue sequentially according to the occurrence time.

[0177] In some embodiments, the device 455 further includes an adding module, the adding module being configured to: determine a second state transition probability distribution corresponding to the first target behavior state from the state transition matrix; the second state transition probability distribution includes a plurality of second transition probabilities; the second transition probability refers to the probability of transitioning the behavior state of the virtual object from the first target behavior state to the preset behavior state; in response to transitioning the behavior state of the virtual object from the first target behavior state to a second target behavior state in the behavior state set based on the second state transition probability distribution, generate a second behavior event queue based on the Poisson rate corresponding to the second target behavior state; and add the second behavior event queue to the first behavior event queue according to the occurrence time of each second behavior event in the second behavior event queue.

[0178] In some embodiments, the control module 4554 is further configured to: in response to the behavior event that the virtual object is controlled to perform being a behavior event corresponding to a third target behavior state, determine the behavior event result of the behavior event according to a preset behavior event probability; the third target behavior state is used to characterize the virtual object being in a challenge state; and execute the behavior event according to the behavior event result.

[0179] In some embodiments, the device 455 further includes an update module, which is configured to: update the index value of the virtual object in the virtual environment according to the result of the behavior event; and, in response to the behavior event reaching a preset behavior termination condition, transfer the behavior state of the virtual object from the third target behavior state to the fourth target behavior state, the fourth target behavior state being used to characterize the virtual object being in a behavior termination state in the virtual environment.

[0180] It should be noted that the description of the apparatus in this application embodiment is similar to the description of the method embodiment described above, and has similar beneficial effects as the method embodiment; therefore, it will not be repeated. For technical details not disclosed in this apparatus embodiment, please refer to the description of the method embodiment of this application for understanding.

[0181] This application provides a computer-readable storage medium storing computer-executable instructions. When these instructions are executed by a processor, they cause the processor to perform the behavior control method provided in this application. For example, ... Figure 3 The behavior control method shown.

[0182] This application provides a computer program product including computer-executable instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer-executable instructions from the computer-readable storage medium and executes the computer-executable instructions, causing the electronic device to perform the behavior control method described in this application.

[0183] In some embodiments, the computer-readable storage medium may be a memory such as RAM, ROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0184] In some embodiments, computer-executable instructions may take the form of programs, software, software modules, scripts, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as stand-alone programs or as modules, components, subroutines, or other units suitable for use in a computing environment.

[0185] As an example, computer-executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple co-located files (e.g., files that store one or more modules, subroutines, or code sections).

[0186] As an example, computer-executable instructions can be deployed to execute on a single electronic device, or on multiple electronic devices located at one location, or on multiple electronic devices distributed across multiple locations and interconnected via a communication network.

[0187] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. A behavior control method, characterized in that, The behavior control method is based on a Markov-modulated Poisson process and includes: Determine the current behavioral state of the virtual object and the state transition matrix of the virtual object in the current time period; the virtual object is an AI player. A first state transition probability distribution corresponding to the current behavior state is determined from the state transition matrix; the first state transition probability distribution includes multiple first transition probabilities; the first transition probability refers to the probability of transitioning the behavior state of the virtual object from the current behavior state to a preset behavior state, the preset behavior state is any behavior state in a preset set of behavior states; the set of behavior states includes sleep state, idle state, challenge state and end state; In response to the virtual object's behavior state being transferred from the current behavior state to a first target behavior state in the set of behavior states based on the first state transition probability distribution, a first behavior event queue is generated based on the Poisson rate corresponding to the first target behavior state; the first behavior event queue is a set of M behavior events arranged in chronological order of occurrence; M is an integer greater than 1; The virtual object is controlled to execute the behavior events in the first behavior event queue in sequence.

2. The method according to claim 1, characterized in that, Determining the state transition matrix of the virtual object in the current time period includes: Obtain the transition matrix corresponding to the current time period from a preset matrix set; the preset matrix set includes transition matrices corresponding to different preset time periods. The transition matrix corresponding to the current time period is determined as the state transition matrix of the virtual object in the current time period.

3. The method according to claim 1, characterized in that, Each row in the state transition matrix corresponds to a behavior state in the set of behavior states; Determining the first state transition probability distribution corresponding to the current behavioral state from the state transition matrix includes: Based on the index of the current behavior state in the set of behavior states, determine the row vector in the state transition matrix corresponding to the index; The row vector is determined as the first state transition probability distribution corresponding to the current behavior state.

4. The method according to claim 1, characterized in that, The first behavior event queue includes M behavior events; M is an integer greater than 1; the step of generating the first behavior event queue based on the Poisson rate corresponding to the first target behavior state includes: In response to the Poisson rate corresponding to the first target behavior state being greater than a preset value, iteration i performs the following processing: Based on the Poisson rate, the time interval for the i-th behavior event to be generated corresponding to the first target behavior state is determined; wherein, ; Based on the time interval and the current time, determine the occurrence time of the i-th behavioral event to be generated; The first behavioral event queue is generated based on the occurrence time.

5. The method according to claim 4, characterized in that, The step of generating the first behavioral event queue based on the occurrence time includes: In response to determining, based on the occurrence time, that the i-th behavior event to be generated has not reached the preset queue termination condition, the i-th behavior event corresponding to the first target behavior state is generated; After associating the i-th behavior event with the occurrence time, add it to the first behavior event queue, and update the current time to the occurrence time; In response to determining that the i-th behavior event to be generated has reached the queue termination condition based on the occurrence time, the generation of the i-th behavior event is prohibited, and the queue of generated behavior events is determined as the first behavior event queue.

6. The method according to claim 5, characterized in that, The step of controlling the virtual object to sequentially execute the behavior events in the first behavior event queue includes: Determine the occurrence time of each behavioral event in the first behavioral event queue; According to the occurrence time, the virtual object is controlled to execute each behavior event in the first behavior event queue in sequence.

7. The method according to any one of claims 1 to 6, characterized in that, After generating the first behavior event queue based on the Poisson rate corresponding to the first target behavior state, the method further includes: A second state transition probability distribution corresponding to the first target behavior state is determined from the state transition matrix; the second state transition probability distribution includes multiple second transition probabilities; the second transition probability refers to the probability of transitioning the behavior state of the virtual object from the first target behavior state to the preset behavior state; In response to the second state transition probability distribution, the behavior state of the virtual object is transferred from the first target behavior state to the second target behavior state in the set of behavior states, and a second behavior event queue is generated based on the Poisson rate corresponding to the second target behavior state. According to the occurrence time of each second behavior event in the second behavior event queue, the second behavior event queue is added to the first behavior event queue.

8. The method according to claim 7, characterized in that, The step of controlling the virtual object to sequentially execute the behavior events in the first behavior event queue includes: In response to a behavior event that controls the virtual object to perform, which is a behavior event corresponding to a third target behavior state, the behavior event result of the behavior event is determined according to a preset behavior event probability; the third target behavior state is used to characterize the virtual object as being in a challenge state. The behavior event is executed based on the result of the behavior event.

9. The method according to claim 8, characterized in that, After executing the behavior event based on the result of the behavior event, the method further includes: Based on the results of the behavioral events, update the indicator values ​​of the virtual object in the virtual environment; and In response to the behavior event reaching a preset behavior termination condition, the behavior state of the virtual object is transferred from the third target behavior state to the fourth target behavior state, which is used to characterize the virtual object as being in a behavior termination state in the virtual environment.

10. A behavior control device, characterized in that, The behavior control device is based on a Markov-modulated Poisson process and includes: The first determining module is used to determine the current behavioral state of the virtual object and the state transition matrix of the virtual object in the current time period; the virtual object is an AI player. The second determining module is used to determine a first state transition probability distribution corresponding to the current behavior state from the state transition matrix; the first state transition probability distribution includes multiple first transition probabilities; the first transition probability refers to the probability of transitioning the behavior state of the virtual object from the current behavior state to a preset behavior state, the preset behavior state is any behavior state in a preset set of behavior states; the set of behavior states includes a dormant state, an idle state, a challenge state, and an end state; The generation module is configured to, in response to the first state transition probability distribution, transfer the behavior state of the virtual object from the current behavior state to a first target behavior state in the set of behavior states, and generate a first behavior event queue based on the Poisson rate corresponding to the first target behavior state; the first behavior event queue is a set of M behavior events arranged in chronological order of occurrence; M is an integer greater than 1; The control module is used to control the virtual object to execute the behavior events in the first behavior event queue in sequence.

Images