A method and system for adaptively adjusting game difficulty

Abstract

The present invention relates to the field of game control technology, and specifically to a method and system for adaptively adjusting game difficulty, comprising: obtaining a player's operation sequence and game status data in a target game, performing feature fusion on the player's operation sequence and game status data to obtain the player's comprehensive ability characteristics; performing latent space modeling on the player's comprehensive ability characteristics based on a variational autoencoder to obtain player latent space variables, and performing t-SNE dimensionality reduction on the player's latent space variables. By obtaining the player's operation sequence and game status data, combining a variational autoencoder and t-SNE dimensionality reduction, the present invention can accurately model the comprehensive ability characteristics of each player. Based on the player's ability characteristics, the method can dynamically adjust the difficulty of the game to adapt it to the level of different players. In this way, players will neither feel it is too difficult nor too easy during the game, thereby maintaining the challenge and fun of the game.

Description

Technical Field

[0001] The present invention relates to the field of game control technology, and in particular to a method and system for adaptively adjusting game difficulty. Background Art

[0002] Traditional methods typically rely on game developers to manually set fixed difficulty levels or use predetermined rules to adjust difficulty. This approach lacks personalization and cannot dynamically adjust difficulty based on each player's actual performance and ability characteristics. This can easily lead to some players experiencing a game that is too easy or too difficult. Players may lose interest due to the mismatch in difficulty, affecting the challenge and fun of the game. Traditional methods also often cannot flexibly adjust the game's difficulty based on players' real-time performance and feedback. The difficulty level is usually pre-set and cannot quickly respond to changes in players' in-game operations. This means that players may not receive timely difficulty adjustments when facing challenges, resulting in an unsmooth gaming experience or even a feeling of being overly boring or frustrating. Traditional methods also lack the capabilities of deep learning and intelligent optimization. In many traditional games, difficulty adjustment strategies are static, and the difficulty levels and task designs in the game are mostly manually set based on the developer's experience. Because traditional methods often rely on manually designed difficulty rules, game developers need to invest a lot of time and effort in testing and adjustment to ensure that all difficulty settings meet the needs of different players. Each adjustment requires manual modification, resulting in a long testing cycle for updates. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to overcome the shortcomings of the above-mentioned prior art and provide a method and system for adaptively adjusting the difficulty of a game.

[0004] The technical solution adopted to solve the above technical problems is: a method for adaptively adjusting game difficulty, comprising:

[0005] Acquire the operation sequence and game status data of the player in the target game, and perform feature fusion on the operation sequence and game status data of the player to obtain the comprehensive ability characteristics of the player;

[0006] Performing latent space modeling on the comprehensive ability characteristics of the player according to a variational autoencoder to obtain a player latent space variable, and performing t-SNE dimensionality reduction on the player latent space variable to obtain a reduced-dimensional player latent space variable;

[0007] Performing category clustering on the reduced-dimensional player latent space variables according to the K-means algorithm to obtain category labels for the players;

[0008] Constructing a difficulty adjustment agent based on the reduced-dimensionality player latent space variables, the player's category label, and the historical difficulty sequence;

[0009] The difficulty adjustment agent is adjusted according to the deep deterministic policy gradient to obtain a difficulty adjustment strategy for the target game.

[0010] Preferably, the operation sequence includes key frequency, reaction time, movement path complexity and combat decision entropy, and the game status data includes level passing time, resource consumption rate and number of failures.

[0011] Preferably, obtaining the player's operation sequence and game status data in the target game includes:

[0012] The key input frequency of the player in a preset time window is recorded by counting the number of keys in the time window, thereby obtaining the key input frequency;

[0013] Recording the time difference between the player observing the target game state change and performing the operation, that is, obtaining the reaction time;

[0014] Record all positions of the player in the target game to obtain a coordinate point sequence. Calculate the change in position at each moment based on the coordinate point sequence to obtain a differential time series. The differential time series is expressed as follows:

[0015] X(t)=U(t)-U(t-1);

[0016] Among them, X(t) represents the differential time series at time t, and U(t) represents the coordinate point sequence at time t;

[0017] The cumulative deviation is calculated based on the differential time series, and the fluctuation range and standard deviation are calculated based on the cumulative deviation. The calculation formula of the cumulative deviation is as follows:

[0018]

[0019] Where Y(t) represents the cumulative deviation at time t;

[0020] The calculation formula of the fluctuation range is as follows:

[0021] R(t)=max(Y(t))-min(Y(t));

[0022] Among them, R(t) represents the fluctuation range at time t;

[0023] The movement path complexity is calculated according to the fluctuation range and the standard deviation, wherein the calculation formula of the movement path complexity is as follows:

[0024]

[0025] Among them, Cp represents the complexity of the moving path, and S(t) represents the standard deviation at time t.

[0026] Preferably, obtaining the player's operation sequence and game status data in the target game also includes:

[0027] Record each decision action of the player and calculate the selection probability of the decision action, wherein the calculation formula of the selection probability is as follows:

[0028]

[0029] Among them, P(A i ) represents decision action A i The probability of selection, n i Represents decision action A i The number of times selected, T represents the total number of action decisions;

[0030] The player's combat decision entropy is calculated based on the selection probability of the decision action, wherein the calculation formula of the combat decision entropy is as follows:

[0031]

[0032] Among them, H d represents the player's combat decision entropy, and N represents the total number of decision actions;

[0033] Recording the time it takes the player to pass each level in the target game, thereby obtaining the level passing time;

[0034] Recording the resource consumption rate of the player during the target game, that is, obtaining the resource consumption rate;

[0035] The number of times the player fails in the target game is recorded, that is, the number of failures is obtained.

[0036] Preferably, the operation sequence and game status data of the player are subjected to feature fusion to obtain the comprehensive ability features of the player, including:

[0037] Merging the player's operation sequence and game state data to obtain a merged feature vector, and normalizing the merged feature vector to obtain a normalized merged feature vector;

[0038] Calculate the information entropy of each feature in the normalized merged feature vector, and calculate the weight of each feature in the normalized merged feature vector based on the information entropy of the feature, wherein the calculation formula of the feature weight is as follows:

[0039]

[0040] Among them, ω i represents the weight of the i-th feature in the normalized merged feature vector, α represents the hyperparameter that controls the steepness of the weight distribution, I(x i ) represents the information entropy of the i-th feature in the normalized merged feature vector;

[0041] The comprehensive ability feature of the player is determined according to the weight of each feature in the normalized combined feature vector and the normalized combined feature vector.

[0042] Preferably, the comprehensive ability characteristics of the player are modeled in a latent space according to a variational autoencoder to obtain the player's latent space variables, including:

[0043] The mean and standard deviation of the potential distribution of the player's comprehensive ability characteristics are learned according to the variational autoencoder, where the calculation formula of the mean is as follows:

[0044] μ z =W μ ReLU(W e F p +b e );

[0045] Among them, μ z represents the mean of the potential distribution of the player’s comprehensive ability characteristics, W μ represents the trainable weight matrix used to generate the mean, W e Represents a trainable weight matrix, ReLU represents the activation function, F p Indicates the comprehensive ability characteristics of the player, b e represents the bias term;

[0046] The calculation formula of the mean is as follows:

[0047]

[0048] Among them, σ z represents the standard deviation of the potential distribution of the player’s comprehensive ability characteristics, W σ represents the trainable weight matrix used to generate the standard deviation;

[0049] The mean and standard deviation of the potential distribution of the player's comprehensive ability characteristics are sampled according to the reparameterization to obtain the player's latent space variable, wherein the calculation formula of the player's latent space variable is as follows:

[0050] z=μ z +σ z ☉∈,∈~N(0,I);

[0051] Where z represents the player latent space variable, ∈ represents the noise sampled from the standard normal distribution, and ⊙ represents element-wise multiplication.

[0052] Preferably, performing category clustering on the dimension-reduced player latent space variables according to the K-means algorithm to obtain category labels for the players includes:

[0053] Construct an objective function, which is used to minimize the sum of squares of distances from points within the cluster to the cluster center, wherein the objective function is as follows:

[0054]

[0055] Among them, J represents the sum of squares of distances from points within the cluster to the cluster center, and p i represents the dimensionality reduction player latent space variable, r ik Represents the dimensionality reduction player latent space variable p i Whether it belongs to cluster k, o k represents the center of cluster k, ||p i -o k || 2 Represents the dimensionality reduction player latent space variable x i The square of the distance to the center of cluster k;

[0056] Allocating the reduced-dimensional player latent space variables corresponding to each player to different clusters based on the objective function;

[0057] Calculate the silhouette coefficient of the cluster and compare the silhouette coefficient of the cluster with a preset silhouette coefficient threshold. If the silhouette coefficient of the cluster is greater than the preset silhouette coefficient threshold, it indicates that the dimensionality reduction player latent space variables corresponding to each player are clustered. Otherwise, it indicates that the dimensionality reduction player latent space variables corresponding to each player are not clustered. The calculation formula of the silhouette coefficient is as follows:

[0058]

[0059] Among them, S(i) represents the silhouette coefficient of the cluster, a(i) represents the average distance from the dimensionality-reduced player's latent space variable to other points in its cluster, and b(i) represents the average distance from the dimensionality-reduced player's latent space variable to the nearest other cluster.

[0060] Preferably, constructing a difficulty adjustment agent based on the dimensionally reduced player latent space variables, the player's category label, and the historical difficulty sequence includes:

[0061] The state of the difficulty adjustment agent is constructed according to the dimension-reduced player latent space variables, the player's category label, and the historical difficulty sequence, wherein the expression of the state is as follows:

[0062] s t =(zt ,c p ,{d t-τ ,...,d t-1});

[0063] Among them, s t represents the state of the difficulty adjustment agent, c p Indicates the player's category label, {d t-τ ,...,d t-1} represents the historical difficulty sequence;

[0064] The action of the difficulty adjustment agent is constructed according to the preset difficulty parameter, wherein the expression of the action is as follows:

[0065] a t =[δ enemy ,δ resource , δ puzzle ];

[0066] Among them, a t represents the action of the difficulty adjustment agent, δ enemy Indicates the strength of controlling enemy AI, δ resource Indicates the richness of control resources, δ puzzle Indicates the difficulty of solving puzzles in the game;

[0067] The reward of the difficulty adjustment agent is constructed according to the number of successes and failures of the player's target game, wherein the expression of the reward is as follows:

[0068] r t =0.7r immersion +0.3r frustration ;

[0069] Among them, r t represents the reward of the difficulty-adjusted agent, r immersion represents the immersion level, and r immersion =tanh(2C f -0.5), C f represents the challenge balance coefficient, and Tanh represents the hyperbolic tangent function, N success Represents the number of successes of the player in the time window, N attempts Indicates the total number of attempts, N fail represents the number of failures of the player in the time window, r frustration represents the degree of frustration, and r frustration = -log(N fail ).

[0070] Preferably, adjusting the difficulty adjustment agent according to the deep deterministic policy gradient to obtain the difficulty adjustment strategy of the target game includes:

[0071] Uniformly random selection of actions with a preset probability;

[0072] Execute the action and obtain the next state and reward at the same time;

[0073] Put the current strategy into the experience replay buffer, randomly and evenly collect multiple strategies, and update the main network parameters according to the loss function for each strategy, where the trajectory includes the current state, current action, next moment state and reward;

[0074] The target network parameters are updated using the main network parameters every preset number of training steps.

[0075] The technical solution adopted to solve the above technical problems is: a game adaptive difficulty adjustment system, which is applicable to the above-mentioned game adaptive difficulty adjustment method, comprising:

[0076] an ability characterization unit, the ability characterization unit being configured to obtain an operation sequence and game state data of a player in a target game, and to perform feature fusion on the operation sequence and game state data of the player to obtain a comprehensive ability feature of the player;

[0077] a spatial modeling unit, the spatial modeling unit being configured to perform latent space modeling on the comprehensive ability characteristics of the player according to a variational autoencoder to obtain a player latent space variable, and perform t-SNE dimensionality reduction on the player latent space variable to obtain a reduced-dimensional player latent space variable;

[0078] A player classification unit, configured to perform category clustering on the reduced-dimensional player latent space variables according to a K-means algorithm to obtain category labels for the players;

[0079] An agent construction unit, configured to construct a difficulty adjustment agent based on the dimensionality-reduced player latent space variables, the player's category label, and a historical difficulty sequence;

[0080] A difficulty adjustment unit is configured to adjust the difficulty adjustment agent according to a deep deterministic policy gradient to obtain a difficulty adjustment strategy for the target game.

[0081] The beneficial effects of the present invention are as follows: (1) The present invention can accurately model the comprehensive ability characteristics of each player by obtaining the player's operation sequence and game status data, combining variational autoencoders and t-SNE dimensionality reduction. According to the player's ability characteristics, the method can dynamically adjust the difficulty of the game to adapt it to the level of different players. In this way, the player will not feel too difficult or too simple during the game, thereby maintaining the challenge and fun of the game. The difficulty adjustment agent is adjusted by the deep deterministic policy gradient (DDPG) algorithm. The method can automatically optimize the difficulty adjustment strategy, which means that the game can automatically adapt according to the player's real-time feedback and continuously adjust to provide the most appropriate challenge, reducing the need for manual adjustment and improving the game's intelligence. (2) The present invention uses latent space modeling to abstract and extract the comprehensive ability characteristics of players, and t-SNE dimensionality reduction further simplifies these high-dimensional features into low-dimensional space, making the understanding of player abilities more intuitive and accurate. This method effectively reduces the difficulty of analyzing complex player behaviors and can accurately identify the ability differences of players, thereby achieving more refined difficulty adjustment; (3) The present invention clusters the player latent space variables after dimensionality reduction through the K-means algorithm and adjusts the difficulty in combination with the historical difficulty sequence. This can help game developers understand the game needs of different player groups. The clustering results can help the game system identify which players prefer high-difficulty or low-difficulty game environments and better predict the future difficulty needs of players based on historical data. BRIEF DESCRIPTION OF THE DRAWINGS

[0082] Figure 1 A schematic flow chart of the steps of the overall method in one embodiment of the present invention;

[0083] Figure 2 This is a schematic diagram of the system architecture of the overall system in an embodiment of the present invention.

[0084] Figure numerals: 1. Ability representation unit; 2. Space modeling unit; 3. Player classification unit; 4. Agent construction unit; 5. Difficulty adjustment unit. DETAILED DESCRIPTION

[0085] Example 1, as Figure 1 As shown, the present invention proposes a method for adaptively adjusting game difficulty, comprising:

[0086] S1. Obtain the player's operation sequence and game status data in the target game, and perform feature fusion on the player's operation sequence and game status data to obtain the player's comprehensive ability characteristics;

[0087] S2. Perform latent space modeling on the comprehensive ability characteristics of the player based on the variational autoencoder to obtain the player's latent space variables, and perform t-SNE dimensionality reduction on the player's latent space variables to obtain the reduced-dimensional player latent space variables;

[0088] S3. Perform category clustering on the reduced-dimensional player latent space variables using the K-means algorithm to obtain the player's category label;

[0089] S4. Construct a difficulty adjustment agent based on the reduced-dimensionality player latent space variables, the player's category label, and the historical difficulty sequence;

[0090] S5. Adjust the difficulty adjustment agent according to the deep deterministic policy gradient to obtain the difficulty adjustment strategy of the target game.

[0091] In the present invention, the autoencoder is an unsupervised learning model used to learn the low-dimensional representation of data. The variational autoencoder is a variant of the autoencoder. By introducing the variational inference technology, the data is mapped to the latent space and new data can be generated. Latent space modeling refers to finding a low-dimensional implicit representation (i.e., latent space variables) by modeling the player's operation and game state characteristics, so as to more efficiently represent the player's ability characteristics. t-SNE is a nonlinear dimensionality reduction technology used to map high-dimensional data to a low-dimensional space, usually two-dimensional or three-dimensional, for data visualization. It can maintain the distance between similar data points in the data and help understand the structure of the data. K-means is a common clustering algorithm that divides the data into pre-set K clusters and finds the center (mean) of each cluster so that the data points in each cluster are as similar as possible. It is used here to cluster the player's latent space features to distinguish players of different categories. In the clustering process, each player will be Assign a category label, which indicates which category the player belongs to. The player's category label can be used to help the game system adjust the game difficulty according to different types of players; the difficulty adjustment agent is an artificial intelligence algorithm that is responsible for dynamically adjusting the difficulty of the game based on the player's ability, category label and historical difficulty information, with the aim of ensuring that the player receives appropriate challenges during the game; Deep Deterministic Policy Gradient (DDPG) is an algorithm based on deep reinforcement learning, which is particularly suitable for tasks in continuous action space. In the context of game difficulty adjustment, DDPG dynamically adjusts the game difficulty by learning the optimal strategy, so that the player is always in a balance between challenge and entertainment; the difficulty adjustment strategy is the final strategy learned by the agent, which is used to decide how to change the difficulty of the game in different situations in order to provide players with a suitable gaming experience. For example, when the player behaves very easily, the agent may increase the difficulty of the game; when the player encounters difficulties, the agent may reduce the difficulty.

[0092] Example 2, a game adaptive difficulty adjustment method proposed by the present invention, compared with Example 1, this embodiment also includes: the operation sequence includes key frequency, reaction time, movement path complexity and combat decision entropy, and the game status data includes level passing time, resource consumption rate and number of failures.

[0093] In an optional embodiment, obtaining the operation sequence and game status data of the player in the target game includes:

[0094] A1. Count the number of keystrokes within a preset time window to record the player's keystroke frequency within the time window, thereby obtaining the keystroke frequency.

[0095] A2. Record the time difference between the player observing the target game state change and executing the action, which is the reaction time.

[0096] A3. Record all the player's positions in the target game to obtain a coordinate point sequence. Calculate the change in position at each moment based on the coordinate point sequence to obtain a differential time series. The differential time series is expressed as follows:

[0097] X(t)=U(t)-U(t-1);

[0098] Among them, X(t) represents the differential time series at time t, and U(t) represents the coordinate point sequence at time t;

[0099] A4. Calculate the cumulative deviation based on the differenced time series, and calculate the fluctuation range and standard deviation based on the cumulative deviation. The calculation formula for the cumulative deviation is as follows:

[0100]

[0101] Where Y(t) represents the cumulative deviation at time t;

[0102] The calculation formula for the fluctuation range is as follows:

[0103] R(t)=max(Y(t))-min(Y(t));

[0104] Among them, R(t) represents the fluctuation range at time t;

[0105] A5. Calculate the movement path complexity based on the fluctuation range and standard deviation. The calculation formula for the movement path complexity is as follows:

[0106]

[0107] Among them, C p represents the complexity of the moving path, and S(t) represents the standard deviation at time t.

[0108] It should be noted that the complexity of the player's movement path within a certain period of time generally refers to the higher the complexity, the more delicate or difficult the player's operation is; decision-making actions refer to every operation choice made by the player during the game. For example, the player's decision to attack, defend, or use skills in battle is a decision-making action.

[0109] In an optional embodiment, obtaining the operation sequence and game status data of the player in the target game further includes:

[0110] A6. Record each decision action of the player and calculate the selection probability of the decision action. The calculation formula for the selection probability is as follows:

[0111]

[0112] Among them, P(A i ) represents decision action A i The probability of selection, n i Represents decision action A i The number of times selected, T represents the total number of action decisions;

[0113] A7. Calculate the player's combat decision entropy based on the probability of selecting a decision action. The calculation formula for combat decision entropy is as follows:

[0114]

[0115] Among them, H d represents the player's combat decision entropy, and N represents the total number of decision actions;

[0116] A8. Record the time it takes the player to pass each level in the target game, thus obtaining the level passing time.

[0117] A9. Record the resource consumption rate of the player during the target game, that is, obtain the resource consumption rate;

[0118] A10. Record the number of times the player fails in the target game, that is, obtain the number of failures.

[0119] In an optional embodiment, the player's operation sequence and game state data are subjected to feature fusion to obtain the player's comprehensive ability features, including:

[0120] B1. Merge the player's operation sequence and game state data to obtain a merged feature vector, and normalize the merged feature vector to obtain a normalized merged feature vector;

[0121] B2. Calculate the information entropy of each feature in the normalized merged feature vector, and calculate the weight of each feature in the normalized merged feature vector based on the information entropy of the feature. The calculation formula of the feature weight is as follows:

[0122]

[0123] Among them, ω i represents the weight of the i-th feature in the normalized merged feature vector, α represents the hyperparameter that controls the steepness of the weight distribution, I(x i ) represents the information entropy of the i-th feature in the normalized merged feature vector;

[0124] B3. Determine the comprehensive ability characteristics of the player based on the weight of each feature in the normalized combined feature vector and the normalized combined feature vector.

[0125] In an optional embodiment, a latent space model is performed on the comprehensive ability characteristics of the player according to a variational autoencoder to obtain the player's latent space variables, including:

[0126] C1. Learn the mean and standard deviation of the potential distribution of the player's comprehensive ability characteristics based on the variational autoencoder. The calculation formula of the mean is as follows:

[0127] μ z =W μ ReLU(W e F p +b e );

[0128] Among them, μ z represents the mean of the potential distribution of the player’s comprehensive ability characteristics, W μ represents the trainable weight matrix used to generate the mean, W e Represents a trainable weight matrix, ReLU represents the activation function, F p Indicates the comprehensive ability characteristics of the player, b e represents the bias term;

[0129] The calculation formula for the mean is as follows:

[0130]

[0131] Among them, σ z represents the standard deviation of the potential distribution of the player’s comprehensive ability characteristics, W σ represents the trainable weight matrix used to generate the standard deviation;

[0132] C2. Sample the mean and standard deviation of the potential distribution of the player's comprehensive ability characteristics based on the reparameterization to obtain the player's latent space variables. The calculation formula of the player's latent space variables is as follows:

[0133] z=μ z +σ z ☉∈,∈~N(0,I);

[0134] Where z represents the player latent space variable, ∈ represents the noise sampled from the standard normal distribution, and ⊙ represents element-wise multiplication.

[0135] In an optional embodiment, the reduced-dimensional player latent space variables are clustered according to the K-means algorithm to obtain the player's category label, including:

[0136] D1. Construct an objective function. The objective function is used to minimize the sum of squares of distances from points within a cluster to the cluster center. The objective function is as follows:

[0137]

[0138] Among them, J represents the sum of squares of distances from points within the cluster to the cluster center, and p i represents the dimensionality reduction player latent space variable, r ik Represents the dimensionality reduction player latent space variable p i Whether it belongs to cluster k, o k represents the center of cluster k, ||p i -o k || 2 Represents the dimensionality reduction player latent space variable x i The square of the distance to the center of cluster k;

[0139] D2. Assign the reduced-dimensional player latent space variables corresponding to each player to different clusters based on the objective function;

[0140] D3. Calculate the cluster silhouette coefficient and compare it with the preset silhouette coefficient threshold. If the cluster silhouette coefficient is greater than the preset silhouette coefficient threshold, it indicates that the dimensionality reduction player latent space variables corresponding to each player are clustered. Otherwise, it indicates that the dimensionality reduction player latent space variables corresponding to each player are not clustered. The calculation formula of the silhouette coefficient is as follows:

[0141]

[0142] Among them, S(i) represents the silhouette coefficient of the cluster, a(i) represents the average distance from the dimensionality-reduced player's latent space variable to other points in its cluster, and b(i) represents the average distance from the dimensionality-reduced player's latent space variable to the nearest other cluster.

[0143] It should be noted that the objective function is a function used in the optimization algorithm to guide model learning and classification, and is usually used to measure the quality of a solution. The objective function here aims to minimize the sum of the squares of the distances from points within the cluster to the cluster center, that is, to ensure that the points within the same cluster are clustered as close to the center of the cluster as possible through the optimization algorithm; the silhouette coefficient is an important indicator for measuring the clustering effect. The value of the silhouette coefficient ranges from -1 to 1. The closer the value is to 1, the better the clustering result is, and the value close to -1 indicates a poor clustering effect.

[0144] In an optional embodiment, a difficulty adjustment agent is constructed based on the dimensionally reduced player latent space variables, the player's category label, and the historical difficulty sequence, including:

[0145] E1. Construct the state of the difficulty adjustment agent based on the reduced-dimensionality player latent space variables, the player's category label, and the historical difficulty sequence. The state expression is as follows:

[0146] s t =(z t ,c p ,{d t-τ ,...,d t-1});

[0147] Among them, s t represents the state of the difficulty adjustment agent, c p Indicates the player's category label, {d t-τ ,...,d t-1} represents the historical difficulty sequence;

[0148] E2. Construct the action of the difficulty adjustment agent based on the preset difficulty parameters, where the action expression is as follows:

[0149] a t =[δ enemy ,δ resource , δ puzzle ];

[0150] Among them, a t represents the action of the difficulty adjustment agent, δ enemy Indicates the strength of controlling enemy AI, δ resource Indicates the richness of control resources, δ puzzle Indicates the difficulty of solving puzzles in the game;

[0151] E3. Construct a reward for the difficulty-adjusted agent based on the number of successes and failures in the player's target game. The reward is expressed as follows:

[0152] r t =0.7r immersion +0.3r frustration ;

[0153] Among them, r t represents the reward of the difficulty-adjusted agent, r immersion represents the immersion level, and r immersion =tanh(2C f -0.5), C f represents the challenge balance coefficient, and Tanh represents the hyperbolic tangent function, N success Represents the number of successes of the player in the time window, N attempts Indicates the total number of attempts, N fail represents the number of failures of the player in the time window, r frustration represents the degree of frustration, and r frustration = -log(N fail ).

[0154] In an optional embodiment, adjusting the difficulty adjustment agent according to the deep deterministic policy gradient to obtain the difficulty adjustment strategy of the target game includes:

[0155] F1, uniformly randomly select actions under a preset probability;

[0156] F2. Execute the action and obtain the next state and reward;

[0157] F3. Put the current strategy into the experience replay buffer, randomly and evenly collect multiple strategies, and update the main network parameters according to the loss function for each strategy. The trajectory includes the current state, current action, next state and reward;

[0158] F4. Update the target network parameters using the main network parameters every preset number of training steps.

[0159] It should be noted that the agent will select an action based on the current state at each moment (each step). The purpose of uniformly random selection of actions is to explore the environment and avoid falling into local optimal solutions. The "uniform random" here means that the probability of each action being selected is equal; the agent selects an action according to the current strategy (which may be random or the optimal action based on the current strategy), and then passes the action to the environment; after executing the action, the environment feeds it back a new state and a reward based on the agent's behavior. The reward reflects the feedback given by the environment after the agent takes a specific action in the current state, and is usually used to measure the quality of the agent's behavior; after each interaction with the environment, the agent will store the current state, action, reward and next state (i.e. trajectory) in the experience replay buffer, so that the agent can use historical experience to Learning is done in a replay buffer without having to rely on the most recent steps every time; as training progresses, a large amount of experience is accumulated in the buffer. By randomly extracting samples from it (batch sampling), the agent can use these historical experiences for training, reducing the impact of time correlation during training; multiple strategies are randomly and uniformly sampled from the experience replay buffer, and these strategies are used to calculate the loss function to update the network parameters. This process is carried out through a deep neural network, and the goal is to optimize the network's behavior strategy so that the agent can maximize future cumulative rewards; the target network is used to calculate the target Q value to avoid an unstable learning process due to fluctuations in the estimated Q value during training. In order to avoid this instability, a target network (target Q network) is usually used to calculate the Q value. The parameters of this network are the same as those of the main network, but the update frequency is lower.

[0160] Example 3, as Figure 2 As shown, the present invention proposes a game adaptive difficulty adjustment system, which is applicable to the game adaptive difficulty adjustment method, including:

[0161] Ability characterization unit 1 is used to obtain the player's operation sequence and game status data in the target game, and perform feature fusion on the player's operation sequence and game status data to obtain the player's comprehensive ability characteristics;

[0162] Spatial modeling unit 2, which is used to perform latent space modeling on the comprehensive ability characteristics of the player based on the variational autoencoder to obtain the player's latent space variables, and perform t-SNE dimensionality reduction on the player's latent space variables to obtain the reduced-dimensional player latent space variables;

[0163] The player classification unit 3 is used to perform category clustering on the reduced-dimensional player latent space variables according to the K-means algorithm to obtain the player's category label;

[0164] The intelligent agent construction unit 4 is used to construct a difficulty adjustment intelligent agent based on the dimensionality reduction player latent space variables, the player's category label and the historical difficulty sequence;

[0165] The difficulty adjustment unit 5 is used to adjust the difficulty adjustment agent according to the deep deterministic policy gradient to obtain the difficulty adjustment strategy of the target game.

[0166] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings, but the present invention is not limited thereto. Various changes can be made within the scope of knowledge possessed by those skilled in the art without departing from the spirit of the present invention.

Claims

1. A method for adaptively adjusting game difficulty, characterized in that: include: Acquire the operation sequence and game status data of the player in the target game, and perform feature fusion on the operation sequence and game status data of the player to obtain the comprehensive ability characteristics of the player; Performing latent space modeling on the comprehensive ability characteristics of the player according to a variational autoencoder to obtain a player latent space variable, and performing t-SNE dimensionality reduction on the player latent space variable to obtain a reduced-dimensional player latent space variable; Performing category clustering on the reduced-dimensional player latent space variables according to the K-means algorithm to obtain category labels for the players; Constructing a difficulty adjustment agent based on the reduced-dimensionality player latent space variables, the player's category label, and the historical difficulty sequence; Adjusting the difficulty adjustment agent according to a deep deterministic policy gradient to obtain a difficulty adjustment strategy for the target game; The comprehensive ability characteristics of the player are modeled in a latent space according to the variational autoencoder to obtain the player's latent space variables, including: The mean and standard deviation of the potential distribution of the player's comprehensive ability characteristics are learned according to the variational autoencoder, where the calculation formula of the mean is as follows: ； in, Represents the mean of the potential distribution of the player's comprehensive ability characteristics, represents the trainable weight matrix used to generate the mean, represents the trainable weight matrix, represents the activation function, Indicates the comprehensive ability characteristics of the player, represents the bias term; The calculation formula of the standard deviation is as follows: ； in, represents the standard deviation of the potential distribution of the player's comprehensive ability characteristics, represents the trainable weight matrix used to generate the standard deviation; The mean and standard deviation of the potential distribution of the player's comprehensive ability characteristics are sampled according to the reparameterization to obtain the player's latent space variable, wherein the calculation formula of the player's latent space variable is as follows: ； in, represents the player’s latent space variable, represents noise sampled from a standard normal distribution, Represents element-wise multiplication; Constructing a difficulty adjustment agent according to the dimensionally reduced player latent space variables, the player's category label, and the historical difficulty sequence, including: The state of the difficulty adjustment agent is constructed according to the dimension-reduced player latent space variables, the player's category label, and the historical difficulty sequence, wherein the expression of the state is as follows: ； in, represents the state of the difficulty adjustment agent, Indicates the player's category label, represents the historical difficulty sequence, Represents the player's latent space variables; The action of the difficulty adjustment agent is constructed according to the preset difficulty parameter, wherein the expression of the action is as follows: ； in, represents the action of the difficulty adjustment agent, Indicates the strength of controlling the enemy AI, Indicates the richness of control resources, Display controls the difficulty of puzzles in the game; The reward of the difficulty adjustment agent is constructed according to the number of successes and failures of the player's target game, wherein the expression of the reward is as follows: ； in, represents the reward of the difficulty-adjusted agent, Indicates immersion, and , represents the challenge balance coefficient, and , represents the hyperbolic tangent function, represents the number of successes achieved by the player within the time window, Indicates the total number of attempts, Indicates the number of failures of the player within the time window, Indicates frustration, and ; Adjusting the difficulty adjustment agent according to a deep deterministic policy gradient to obtain a difficulty adjustment strategy for the target game includes: Uniformly random selection of actions with a preset probability; Execute the action and obtain the next state and reward at the same time; Put the current strategy into the experience replay buffer, randomly and evenly collect multiple strategies, and update the main network parameters according to the loss function for each strategy, where the current strategy includes the current state, current action, next moment state and reward; The target network parameters are updated using the main network parameters every preset number of training steps.

2. A method for adjusting game difficulty according to claim 1, characterized in that: in, The operation sequence includes keystroke frequency, reaction time, movement path complexity and combat decision entropy, and the game status data includes level passing time, resource consumption rate and number of failures.

3. A method for adjusting game difficulty according to claim 2, characterized in that: Obtain the player's operation sequence and game status data in the target game, including: The key input frequency of the player in a preset time window is recorded by counting the number of keys in the time window, thereby obtaining the key input frequency; Recording the time difference between the player observing the target game state change and performing the operation, that is, obtaining the reaction time; Record all positions of the player in the target game to obtain a coordinate point sequence. Calculate the change in position at each moment based on the coordinate point sequence to obtain a differential time series. The differential time series is expressed as follows: ； in, Indicates The time series of difference of moments, Indicates A sequence of coordinate points at a given moment; The cumulative deviation is calculated based on the differential time series, and the fluctuation range and standard deviation are calculated based on the cumulative deviation. The calculation formula of the cumulative deviation is as follows: ； in, Indicates The cumulative deviation of the time, Indicates moment-difference time series; The calculation formula of the fluctuation range is as follows: ； in, Indicates The fluctuation range of the moment; The movement path complexity is calculated according to the fluctuation range and the standard deviation, wherein the calculation formula of the movement path complexity is as follows: ； in, represents the complexity of the moving path, Indicates The standard deviation of the time.

4. A method for adaptively adjusting game difficulty according to claim 3, characterized in that: Get the player's action sequence and game status data in the target game, including: Record each decision action of the player and calculate the selection probability of the decision action, wherein the calculation formula of the selection probability is as follows: ； in, Indicates decision action The probability of selection, Indicates decision action The number of times selected, Indicates the total number of action decisions; The player's combat decision entropy is calculated based on the selection probability of the decision action, wherein the calculation formula of the combat decision entropy is as follows: ； in, represents the player's combat decision entropy, represents the total number of decision actions; Recording the time it takes the player to pass each level in the target game, thereby obtaining the level passing time; Recording the resource consumption rate of the player during the target game, that is, obtaining the resource consumption rate; The number of times the player fails in the target game is recorded, that is, the number of failures is obtained.

5. A method for adaptively adjusting game difficulty according to claim 4, characterized in that: Performing feature fusion on the player's operation sequence and game status data to obtain the player's comprehensive ability features, including: Merging the player's operation sequence and game state data to obtain a merged feature vector, and normalizing the merged feature vector to obtain a normalized merged feature vector; Calculate the information entropy of each feature in the normalized merged feature vector, and calculate the weight of each feature in the normalized merged feature vector based on the information entropy of the feature, wherein the calculation formula of the feature weight is as follows: ； in, Represents the first in the normalized merged feature vector The weight of the feature, represents the hyperparameter that controls the steepness of the weight distribution, Represents the first in the normalized merged feature vector The information entropy of each feature; The comprehensive ability feature of the player is determined according to the weight of each feature in the normalized combined feature vector and the normalized combined feature vector.

6. A method for adaptively adjusting game difficulty according to claim 5, characterized in that: The reduced-dimensional player latent space variables are clustered according to the K-means algorithm to obtain the player's category label, including: Construct an objective function, which is used to minimize the sum of squares of distances from points within the cluster to the cluster center, wherein the objective function is as follows: ； in, represents the sum of squares of distances from points within a cluster to the cluster center, represents the dimensionality reduction player latent space variable, Represents the dimensionality reduction player latent space variable Belongs to a cluster , Represents a cluster the center, Represents the dimensionality reduction player latent space variable To cluster The square of the distance to the center; Allocating the reduced-dimensional player latent space variables corresponding to each player to different clusters based on the objective function; Calculate the silhouette coefficient of the cluster and compare the silhouette coefficient of the cluster with a preset silhouette coefficient threshold. If the silhouette coefficient of the cluster is greater than the preset silhouette coefficient threshold, it indicates that the dimensionality reduction player latent space variables corresponding to each player are clustered. Otherwise, it indicates that the dimensionality reduction player latent space variables corresponding to each player are not clustered. The calculation formula of the silhouette coefficient is as follows: ； in, represents the silhouette coefficient of the cluster, Represents the average distance from the latent space variable of the dimensionality reduction player to other points in its cluster, Represents the average distance from the dimensionality reduction player's latent space variable to the nearest other clusters.

7. A game adaptive difficulty adjustment system, applicable to a game adaptive difficulty adjustment method according to any one of claims 1 to 6, characterized in that: include: A capability characterization unit (1), the capability characterization unit (1) is used to obtain an operation sequence and game state data of a player in a target game, and perform feature fusion on the operation sequence and game state data of the player to obtain a comprehensive capability feature of the player; A spatial modeling unit (2), the spatial modeling unit (2) is used to perform latent space modeling on the comprehensive ability characteristics of the player according to the variational autoencoder to obtain the player latent space variables, and perform t-SNE dimension reduction on the player latent space variables to obtain the reduced dimension player latent space variables; A player classification unit (3), the player classification unit (3) is used to perform category clustering on the dimension-reduced player latent space variables according to a K-means algorithm to obtain category labels of the players; An intelligent agent construction unit (4), the intelligent agent construction unit (4) is used to construct a difficulty adjustment intelligent agent according to the dimension-reduced player latent space variables, the player's category label and the historical difficulty sequence; A difficulty adjustment unit (5) is used to adjust the difficulty adjustment agent according to the deep deterministic policy gradient to obtain a difficulty adjustment strategy for the target game.

Images