Intelligent management platform system and management method

Abstract

The invention belongs to the technical field of management platforms, and discloses an intelligent management platform system and a management method. The method comprises the following steps: acquiring a user ID of a user side in adjacent unit time and corresponding user behavior data; preprocessing the user behavior data to obtain preprocessed data; based on the preprocessed data, inputting a pre-trained behavior prediction model, and predicting the loss probability and the payment potential of the user in the next unit time; acquiring level numbers and corresponding level information in the game; based on the level number and the corresponding level information, inputting the predicted loss probability and payment potential of the user in the next unit time into a pre-constructed level adjustment model, and outputting an optimal adjustment strategy; generating an adjustment instruction according to the optimal adjustment strategy, and adjusting the level in the corresponding user ID; and the user churn rate is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of management platforms, and more specifically, to an intelligent management platform system and a management method. Background Art

[0002] Game operation refers to a series of management, promotion, optimization, and support activities carried out after a game is launched. The purpose is to enhance the game experience, increase user retention rate and payment rate, and extend the game life cycle, ultimately achieving the commercial success of the game. With the rapid development of the game industry, game operation has become the core link affecting the game life cycle and commercial revenue. However, players' behaviors and preferences vary greatly, and traditional fixed operation strategies are difficult to meet the personalized needs of players. Therefore, as the number of games in the market increases and players' choices expand, how to improve the player retention rate and payment rate has become an important task in game operation.

[0003] The existing technology still has the following problems:

[0004] The game industry generally faces the problem of high user churn rate. At the same time, a large amount of user behavior data accumulated during game operation has not been fully explored and utilized, and the user behavior patterns contained in the data have not been deeply explored, limiting the optimization space of the game experience.

[0005] In view of this, the present invention proposes an intelligent management platform system and a management method to solve the above problems. Summary of the Invention

[0006] To overcome the above-mentioned defects of the existing technology, the present invention provides the following technical solutions: An intelligent management method, including:

[0007] Obtain the user ID and corresponding user behavior data at the client within adjacent unit time;

[0008] Preprocess the user behavior data to obtain preprocessed data;

[0009] Based on the preprocessed data, input it into a pre-trained behavior prediction model to predict the user churn probability and payment potential in the next unit time;

[0010] Obtain the level number and corresponding level information in the game;

[0011] Based on the level number and corresponding level information, input the predicted user churn probability and payment potential in the next unit time into a pre-constructed level adjustment model to output the optimal adjustment strategy;

[0012] Generate an adjustment instruction according to the optimal adjustment strategy to adjust the level in the corresponding user ID.

[0013] Further, the user behavior data includes total login duration, total activity, total consumption records, and level progress; the level progress is the level number where the user is at the end of a unit time; the preprocessed data includes total login duration difference, total activity difference, total consumption record difference, and level progress difference;

[0014] The training method of the behavior prediction model includes:

[0015] Obtain v groups of behavior data sets, where v is a positive integer greater than 1. The behavior data set includes total login duration difference, total activity difference, total consumption record difference, level progress difference, corresponding churn probability, and payment potential; use the behavior data set as a sample set, divide the sample set into a training set and a test set, construct an RNN classifier, use the total login duration difference, total activity difference, total consumption record difference, and level progress difference in the training set as input data, use the churn probability and payment potential in the training set as output data, perform iterative training on the classifier to obtain an initial prediction model, use the test set to test the initial prediction model, and if the accuracy of the initial prediction model meets the preset accuracy, use the initial prediction model as the behavior prediction model, and the classifier is an RNN neural network model.

[0016] Further, the level adjustment model is a reinforcement learning model using a Q-network structure, and the construction steps are as follows:

[0017] Step 1: Design the state representation dimension;

[0018] Step 2: Design the action representation dimension;

[0019] Step 3: Construct the Q-network;

[0020] The Q-network includes 1 input layer, 2 hidden layers, and 1 output layer. The input layer corresponds to the state representation dimension, and the corresponding number of nodes is 136; the hidden layers include a first hidden layer and a second hidden layer. The activation function of the first hidden layer is the Relu function, and the number of nodes is 64. The structure of the second hidden layer is the same as that of the first hidden layer; the output layer corresponds to the action representation dimension, and the number of nodes is 32, which is a linear activation function;

[0021] Step 4: Set the reward function:

[0022] Step 5: Set the optimization execution plan:

[0023] The optimizer is Adam, the learning rate is preset to v, the capacity of the experience replay pool is preset to u, it is preset to update the Q-network every x steps, and the ε-greedy strategy is used to select actions; v, u, and x are all positive real numbers.

[0024] Further, the design state representation dimensions include the discrete state of design churn probability, the discrete state of payment potential, the discrete state of level number, and the discrete state of level information; the churn probability includes low, medium, and high; the payment potential includes low, medium, and high; the level information includes the difficulty parameter level and the reward value level; the difficulty parameter level is divided into easy, medium, and difficult in ascending order; the reward value level is divided into first level, second level, and third level in ascending order; the discrete state of level information includes the discrete state of difficulty parameter level and the discrete state of reward value level;

[0025] The discrete state of the churn probability is: low corresponds to the vector [1, 0, 0], medium corresponds to the vector [0, 1, 0], and high corresponds to the vector [0, 0, 1]; One-hot encoding is used;

[0026] The discrete state of the payment potential is: low corresponds to the vector [1, 0, 0], medium corresponds to the vector [0, 1, 0], and high corresponds to the vector [0, 0, 1]; One-hot encoding is used;

[0027] The discrete state of the level number is a finite number set, and the finite number set is from 01 to 09, and One-hot encoding is used; the discrete state of the difficulty parameter level is: easy corresponds to the vector [1, 0, 0], medium corresponds to the vector [0, 1, 0], and difficult corresponds to the vector [0, 0, 1]; One-hot encoding is used; the discrete state of the reward value level is: first level corresponds to the vector [1, 0, 0], second level corresponds to the vector [0, 1, 0], and third level corresponds to the vector [0, 0, 1]; One-hot encoding is used.

[0028] Further, the design action representation dimensions include the adjustment action range of the design difficulty parameter level and the adjustment action range of the reward value level;

[0029] The adjustment action range of the difficulty parameter level is: -1, 0, and 1, and One-hot encoding is used;

[0030] The adjustment action range of the reward value level is: -1, 0, and 1, and One-hot encoding is used.

[0031] Further, the reward function formula is: multiply the discount rate by the maximum target Q value of the optimal action in the next state, and sum it with the immediate reward value to calculate the target Q value of the current action in the current state.

[0032] Further, the calculation method of the immediate reward value R includes:

[0033] Subtract the churn probability before adjustment from the churn probability after adjustment to obtain the change value of the churn probability; subtract the payment potential before adjustment from the payment potential after adjustment to obtain the change value of the payment potential; perform weighted calculation on the change value of the churn probability, the change value of the payment potential, the adjustment action amplitude value of the churn probability, and the adjustment action amplitude value of the payment potential to obtain the immediate reward value.

[0034] Further, the training optimization method of the reinforcement learning model includes:

[0035] Obtain c sets of historical training data, where c is a positive integer greater than 1, and the historical training data includes the churn probability, the payment potential, and the optimal adjustment strategy; divide the c sets of historical training data into a training set and a test set with a ratio of 4:1; initialize the experience replay pool, start from a random initial state, select actions using the ε-greedy strategy, execute the actions, observe the rewards and the next state, store the experiences in the replay pool, and every δ steps, where δ is a preset value, randomly sample experience samples from the replay pool for learning. The learning method includes: sampling a (s, a, a', s'), as the current state; as the current action; as the next state; as the next state the optimal action under; calculate the target Q value of the current action; and calculate the loss function based on the target Q value of the current action and the actually preset reward value.

[0036] An intelligent management platform system for implementing the intelligent management method includes:

[0037] A first acquisition module for acquiring the user ID and the corresponding user behavior data of the user terminal within adjacent unit time;

[0038] A preprocessing module for preprocessing the user behavior data to obtain preprocessed data;

[0039] A behavior prediction module for predicting the churn probability and the payment potential of the user in the next unit time based on the preprocessed data and inputting them into a pre-trained behavior prediction model;

[0040] A second acquisition module for acquiring the level number and the corresponding level information in the game;

[0041] A dynamic adjustment module for inputting the predicted churn probability and payment potential of the user in the next unit time into a pre-constructed level adjustment model based on the level number and the corresponding level information, and outputting an optimal adjustment strategy;

[0042] An execution module for generating an adjustment instruction according to the optimal adjustment strategy and adjusting the levels in the corresponding user ID.

[0043] Technical effects and advantages of an intelligent management platform system and a management method of the present invention:

[0044] The present invention first obtains a user ID and corresponding user behavior data, and after a preprocessing step, inputs them into a behavior prediction model to predict the user's churn probability and payment potential within the next unit of time; then obtains the level number in the game and corresponding level information, and through a pre-constructed level adjustment model, outputs an optimal adjustment strategy and adjusts the levels in the corresponding user ID. This optimizes the user's gaming experience and further improves the continuous operation efficiency of the game.

[0045] The experience differences of different users in the same level are large. For example, it is too difficult for novice users, or lacks challenge for high-level users; the present invention uses a reinforcement learning model to output a targeted level adjustment strategy according to the behavioral data characteristics of individual users, and then dynamically adjusts the level difficulty parameter level and reward value level according to the level number to improve the personalization and adaptability of the level.

[0046] By predicting the user's churn probability and dynamically adjusting the level difficulty and rewards, the probability of user churn due to frustration or insufficient attraction is reduced, and the user retention rate is enhanced; for example, for users with a high churn probability, the level difficulty is appropriately reduced, the clearance success rate is increased, and the user stickiness is enhanced; for users with a medium or low churn probability, the user's sense of achievement is improved by optimizing the rewards, and the churn risk is further reduced.

[0047] By predicting the user's payment potential and dynamically adjusting the level rewards, the consumption willingness of high-potential users is enhanced, thereby effectively improving the operation revenue of the game. For example, for users with high payment potential, the reward value level (such as rare items, currency rewards) is appropriately increased. Brief Description of the Drawings

[0048] Figure 1 It is a schematic diagram of an intelligent management platform system according to Embodiment 1 of the present invention;

[0049] Figure 2 It is a flowchart of an intelligent management method according to Embodiment 2 of the present invention;

[0050] Figure 3 It is a flowchart for constructing a level adjustment model according to Embodiment 1 of the present invention;

[0051] Figure 4 It is a schematic diagram of the Q-network structure according to Embodiment 1 of the present invention. Detailed Embodiments

[0052] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0053] Embodiment 1

[0054] Please refer to Figure 1 As shown, this embodiment discloses an intelligent management platform system, including: a first acquisition module, a preprocessing module, a behavior prediction module, a second acquisition module, a dynamic adjustment module, and an execution module. Each module is connected by wire and / or wireless.

[0055] The first acquisition module is used to acquire the user ID and the corresponding user behavior data of the user terminal within adjacent unit time; the user behavior data includes the total login duration, total activity, total consumption record, and level progress; the level progress is the level number where the user is located at the end of the unit time.

[0056] The user ID is used to uniquely represent each user, which is the binding object of each user behavior data. At the same time, it is convenient to adjust the user ID corresponding to the optimal adjustment strategy output by the subsequent adjustment model, avoiding errors in the adjustment object; the total login duration directly reflects the user's participation and stickiness; the longer the total login duration, the lower the loss probability; the total activity comprehensively reflects the interaction frequency between the user and the game and is an important indicator to measure whether the user is interested in the game; the total consumption record reflects the user's historical payment behavior and is used as the direct basis for the subsequent behavior prediction model to predict the payment potential; the level progress reflects the user's position and challenge situation in the game, which helps to determine the adjustment target level corresponding to the optimal adjustment strategy output by the subsequent level adjustment model.

[0057] The user ID is obtained through the user account bound when the user registers and logs in to the game.

[0058] The method for obtaining the total login duration includes:

[0059] The server receives the login and logout timestamps of the client, calculates and obtains the duration of each session, which is recorded as the single login duration, and accumulates the single login durations within the unit time to obtain the total login duration.

[0060] The total activity is obtained through the server log to record the total number of tasks completed by the user; after the total number of tasks completed is recorded by the user client, it is uploaded to the server in real time using the data embedding technology.

[0061] The total consumption record is obtained by the server by counting the recharge and consumption records within the unit time.

[0062] The level progress, that is, the level number at which the user was last offline, is obtained through the server.

[0063] A preprocessing module preprocesses the user behavior data to obtain preprocessed data; the preprocessed data includes the difference in total login duration, the difference in total activity, the difference in total consumption records, and the difference in level progress.

[0064] The method for obtaining the difference in total login duration includes: obtaining the total login duration within adjacent unit time, subtracting the total login duration in the latter unit time of the adjacent unit time from the total login duration in the former unit time, and calculating to obtain the difference in total login duration.

[0065] Referring to the method for obtaining the difference in total login duration, obtain the difference in total activity, the difference in total consumption records, and the difference in level progress.

[0066] A behavior prediction module, based on the preprocessed data, inputs a pre-trained behavior prediction model to predict the churn probability and payment potential of the user in the next unit time; the churn probability includes low, medium, and high; the payment potential includes low, medium, and high.

[0067] It should be noted that for the churn probability, the low corresponding probability range is 0 - 20%; the medium corresponding probability range is 20% - 60%; the high corresponding probability range is 60% - 100%.

[0068] It should be noted that for the payment potential, the low corresponding payment range is 0 - 100 yuan; the medium corresponding payment range is 100 - 500 yuan; the high corresponding payment range is above 500 yuan.

[0069] The training method of the behavior prediction model includes:

[0070] Obtain v groups of behavior data sets, where v is a positive integer greater than 1. The behavior data set includes the difference in total login duration, the difference in total activity, the difference in total consumption records, the difference in level progress, the corresponding churn probability, and payment potential; use the behavior data set as a sample set, divide the sample set into a training set and a test set, construct an RNN classifier, use the difference in total login duration, the difference in total activity, the difference in total consumption records, and the difference in level progress in the training set as input data, use the churn probability and payment potential in the training set as output data, perform iterative training on the classifier to obtain an initial prediction model, use the test set to test the initial prediction model, and if the accuracy of the initial prediction model meets the preset accuracy, then use the initial prediction model as the behavior prediction model, and the classifier is an RNN neural network model.

[0071] The method for obtaining the behavior data set includes:

[0072] Collect the user behavior data of multiple different user IDs corresponding to the historical adjacent unit time, as well as the corresponding churn probability and payment potential in the next unit time; the churn probability and payment potential are marked and distinguished by the game background control personnel with reference to the user behavior data of the next unit time at the end of the latter unit time in the adjacent unit time.

[0073] A second acquisition module, configured to acquire the level numbers in the game and the corresponding level information; the level information includes a difficulty parameter level and a reward value level; the difficulty parameter level is divided into easy, medium, and difficult in ascending order; the reward value level is divided into first level, second level, and third level in ascending order.

[0074] The level number is used to uniquely identify the levels in the game and is the core identifier of the level information; the difficulty parameter level reflects the difficulty level of the level design, and the reward value level represents the reward level after clearing the level. It is used to adjust the difficulty parameter level and the reward value level in the subsequent level adjustment model, reduce the user churn probability in the next unit time, and appropriately increase the payment potential, thereby effectively improving the efficiency of game operation.

[0075] Both the difficulty parameter level and the reward value level are obtained through the configuration file corresponding to the level.

[0076] A dynamic adjustment module, based on the level number and the corresponding level information, inputs the predicted churn probability and payment potential of users in the next unit time into a pre-constructed level adjustment model, and outputs an optimal adjustment strategy.

[0077] The level adjustment model is a reinforcement learning model using a Q-network structure. Please refer to Figure 3 , and the construction steps are as follows:

[0078] Step 1: Design the state representation dimension.

[0079] The design of the state representation dimension includes the design of the discrete state of the churn probability, the discrete state of the payment potential, and the discrete state of the level information.

[0080] The discrete state of the churn probability is: low corresponds to the vector [1, 0, 0], medium corresponds to the vector [0, 1, 0], and high corresponds to the vector [0, 0, 1]; One-hot encoding is used.

[0081] The discrete state of the payment potential is: low corresponds to the vector [1, 0, 0], medium corresponds to the vector [0, 1, 0], and high corresponds to the vector [0, 0, 1]; One-hot encoding is used.

[0082] In the said level information, the level number status is discretized into a finite number set, and the finite number set is from 01 to 09, using One-hot encoding; the status discretization of the level information includes the discretization of the difficulty parameter level status and the discretization of the reward value level status; the discretization of the difficulty parameter level status is: easy corresponding to the vector [1, 0, 0], medium corresponding to the vector [0, 1, 0], and difficult corresponding to the vector [0, 0, 1]; using One-hot encoding; the discretization of the reward value level status is: first level corresponding to the vector [1, 0, 0], second level corresponding to the vector [0, 1, 0], and third level corresponding to the vector [0, 0, 1]; using One-hot encoding.

[0083] The status representation dimension is One-hot encoding, and it is the sum of the One-hot dimensions of the churn probability status, the One-hot dimension of the payment potential status, the One-hot dimension of the level number status, the One-hot dimension of the difficulty parameter level status, and the One-hot dimension of the reward value level status

[0084] Step 2: Design the action representation dimension.

[0085] The said design of the action representation dimension includes designing the adjustment action range of the difficulty parameter level and the adjustment action range of the reward value level.

[0086] The adjustment action range of the difficulty parameter level is: -1, 0, and 1, using One-hot encoding.

[0087] The adjustment action range of the reward value level is: -1, 0, and 1, using One-hot encoding.

[0088] The action representation dimension is One-hot encoding, and it is the sum of the One-hot dimension of the difficulty parameter level action and the One-hot dimension of the reward value level action.

[0089] It should be specifically noted that: the status representation dimension and the action representation dimension are not the status and action in the normal state, but are improvements made according to the particularity of the status and action of the present invention; according to the particularity of the present invention, the division of the status and action takes into account the strong correlation between user churn and payment, and through discretization and One-hot encoding, the calculation process is simplified, and at the same time, the influence of unit differences is avoided.

[0090] Meanwhile, although the payment potential and the churn probability are independent inputs, they are highly correlated with the decisions of level design; the level adjustment model needs to comprehensively consider the trade-off between the two during policy training; One-hot encoding makes the input state and output action representations clear and easy to process, and at the same time provides a standardized input-output space for subsequent reward value setting and policy optimization; in the form of One-hot encoding, the state representation dimension refers to the total feature space dimension occupied by each environmental state variable after One-hot encoding, which serves as the dimension of the feature vector representing the environmental state; for example, after the churn probability state is One-hot encoded, each state is represented by a One-hot vector. For example, low corresponds to the vector [1, 0, 0], medium corresponds to the vector [0, 1, 0], and high corresponds to the vector [0, 0, 1]. Then the sum of the One-hot dimensions of the churn probability state is 9; the payment potential state, the difficulty parameter level state, and the reward value level state are also represented by One-hot vectors with the same rule after One-hot encoding; the level number is a finite number set from 01 to 09, and the sum of the One-hot dimensions of the level number state is 100; the state representation dimension = the sum of the One-hot dimensions of the churn probability state + the sum of the One-hot dimensions of the payment potential state + the sum of the One-hot dimensions of the level number state + the sum of the One-hot dimensions of the difficulty parameter level state + the sum of the One-hot dimensions of the reward value level state = 9 + 9 + 100 + 9 + 9 = 136; the action representation dimension is the same.

[0091] Step 3, construct the Q network.

[0092] Please refer to Figure 4 , the Q network includes 1 input layer, 2 hidden layers and 1 output layer. The input layer corresponds to the state representation dimension, and the corresponding number of nodes is 136; the hidden layer includes the first hidden layer and the second hidden layer. The activation function of the first hidden layer is the Relu function, and the number of nodes is 64. The structure of the second hidden layer is the same as that of the first hidden layer, and is used to learn the complex relationship between the input state and the output action; the output layer corresponds to the action representation dimension, and the number of nodes is 32, which is a linear activation function.

[0093] Step 4, set the reward function.

[0094] The formula of the reward function is:

[0095] ;

[0096] In the formula, is the target Q value, which is used to evaluate the value of the current state and action, combines with the immediate reward and the value of the next state, and guides the level dynamic adjustment strategy; the core of reinforcement learning is to train the network to gradually approximate the predicted Q value to the target Q value, so as to optimize the decision-making process. The immediate reward value of the adjustment strategy for the level, reflecting the immediate effect of the adjustment strategy for the level; Is the next state, and the next state is the churn probability, payment potential, and game information after adjustment; Is the next state The optimal action under the next state, and the next state The optimal action under the next state is the optimal adjustment strategy to be executed at the beginning of the next state; Is the discount rate, used to balance the importance of immediate rewards and future rewards, and the initial value is preset to p; the higher p is, the more the solution focuses on long-term effects, such as the continuous impact of the optimized level on user retention and payment behavior; p is a positive number less than 1; Is the maximum target Q value of the optimal action of the next state, combined with the current immediate reward R, so as to calculate the overall value of the adjustment strategy in the current state.

[0097] The calculation method of the immediate reward value R includes:

[0098] ;

[0099] In the formula, Is the value obtained by subtracting the churn probability before adjustment from the churn probability after adjustment.

[0100] Is the value obtained by subtracting the payment potential before adjustment from the payment potential after adjustment; Is the adjustment action amplitude value of the churn probability; Is the adjustment action amplitude value of the payment potential; , , , And Are the corresponding weights.

[0101] Exemplarily, if the churn probability after adjustment decreases by 10%, that is, The value of is -0.1; if the payment potential after adjustment increases by 100 yuan, that is, [[ID=4�]] The value of is 100; the adjustment action amplitude value of the difficulty parameter level is -1, and the adjustment action amplitude value of the reward numerical level is 1; the weights , , , And Are 5, 0.005, 0.5, 0.8, and 0.2 respectively, then the immediate reward value R at the current moment is 0.5, indicating that the result after adjustment is generally effective.

[0102] Step 5, set the optimization execution plan.

[0103] The optimizer is Adam, the learning rate is preset to v, the capacity of the experience replay pool is preset to u, the Q network is updated every x steps, and the ε-greedy strategy is used to select actions; both v, u, and x are positive real numbers.

[0104] The training optimization method of the reinforcement learning model includes:

[0105] Obtain c sets of historical training data, where c is a positive integer greater than 1. The historical training data includes churn probability, payment potential, and optimal adjustment strategies; divide the c sets of historical training data into a training set and a test set with a ratio of 4:1; initialize the experience replay pool, start from a random initial state, select actions using the ε-greedy strategy, execute the actions, observe the rewards and the next state, store the experiences in the replay pool, and every δ steps, where δ is a preset value determined by those skilled in the art according to data fitting, randomly sample experience samples from the replay pool for learning. The learning method includes: sampling a (s, a, a', s'), is the current state, and the current state is the current churn probability, payment potential, level number, and level information. is the current action, and the current action is the adjustment strategy executed in the current state.

[0106] Calculate , calculate the loss function, and the loss function The formula is:

[0107] ;

[0108] In the formula, are the Q network parameters, initially 0; is the actual preset reward value obtained using the current action according to the current Q network parameters in the current state.

[0109] It should be noted particularly that: in the training stage is determined by those skilled in the art. As a value orientation, it gives the training direction, making the model change direction towards the development direction of pre-solving the problems of the present invention.

[0110] Perform backpropagation to optimize the Q network parameters, use the Adam optimizer, ε gradually decays from 1 to the minimum value, enhance the utilization of the Q network parameters obtained from training, test the Q network effect every δ steps, continuously train until the loss function converges, output the Q network parameters, and use the Q network parameters to evaluate the control effect on the test set. When the average reward value is higher than the preset threshold of 0.9 for N consecutive evaluation cycles, the training can be terminated; the ε is the exploration rate.

[0111] It should be noted that in the initial stage of training, since the Q-network has not been fully trained, more exploration (i.e., ε is 1, indicating full exploration) is required to collect diverse data and cover different situations in the state space. As the training progresses, the Q-network's prediction of action values becomes gradually accurate, so the necessity of exploration decreases. At this time, ε is gradually reduced to improve the training efficiency.

[0112] An execution module, configured to generate an adjustment instruction according to the optimal adjustment strategy and adjust the level information in the corresponding user ID.

[0113] Embodiment 2

[0114] Please refer to Figure 2 As shown, this embodiment provides an intelligent management method, including:

[0115] Obtain the user ID of the user side and the corresponding user behavior data within adjacent unit time;

[0116] Preprocess the user behavior data to obtain preprocessed data;

[0117] Based on the preprocessed data, input it into a pre-trained behavior prediction model to predict the user's churn probability and payment potential in the next unit time;

[0118] Obtain the level number in the game and the corresponding level information;

[0119] Based on the level number and the corresponding level information, input the predicted user's churn probability and payment potential in the next unit time into a pre-constructed level adjustment model to output the optimal adjustment strategy;

[0120] Generate an adjustment instruction according to the optimal adjustment strategy and adjust the level in the corresponding user ID.

[0121] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0122] In addition, in each embodiment of this application, the functional units can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated units can be implemented in the form of hardware or in the form of software functional units.

[0123] The unit described as a separation component may or may not be physically separated. The component shown as a unit may or may not be a physical unit, that is, it may be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0124] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, to implement all or part of the processes in the above-mentioned embodiment methods of this application, it can also be completed by instructing relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above-mentioned method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file or some intermediate form, etc. The computer-readable medium can include: any entity or device that can carry the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disc, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal, and software distribution medium, etc.

[0125] The above-mentioned embodiments are only used to illustrate the technical solutions of this application, rather than limiting them; although this application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of each embodiment of this application, and should all be included in the protection scope of this application.

Claims

1. An intelligent management method, characterized in that, Including: Obtain the user ID of the client and the corresponding user behavior data within adjacent unit time; Preprocess the user behavior data to obtain preprocessed data; Based on the preprocessed data, input it into a pre-trained behavior prediction model to predict the churn probability and payment potential of the user in the next unit time; Obtain the level number in the game and the corresponding level information; Based on the level number and the corresponding level information, input the predicted churn probability and payment potential of the user in the next unit time into a pre-constructed level adjustment model, and output the optimal adjustment strategy; Generate an adjustment instruction according to the optimal adjustment strategy to adjust the level in the corresponding user ID.

2. The intelligent management method according to claim 1, characterized in that, The user behavior data includes total login duration, total activity, total consumption record, and level progress; the level progress is the level number where the user is at the end of the unit time; The preprocessed data includes the difference in total login duration, the difference in total activity, the difference in total consumption record, and the difference in level progress; The training method of the behavior prediction model includes: Obtain v groups of behavior data sets, where v is a positive integer greater than 1. The behavior data set includes the difference in total login duration, the difference in total activity, the difference in total consumption record, the difference in level progress, the corresponding churn probability, and payment potential; Use the behavior data set as a sample set, divide the sample set into a training set and a test set, construct an RNN classifier, use the difference in total login duration, the difference in total activity, the difference in total consumption record, and the difference in level progress in the training set as input data, and use the churn probability and payment potential in the training set as output data, and perform iterative training on the classifier to obtain an initial prediction model. Use the test set to test the initial prediction model. If the accuracy of the initial prediction model meets the preset accuracy, use the initial prediction model as the behavior prediction model, and the classifier is an RNN neural network model.

3. The intelligent management method according to claim 1, wherein The level adjustment model is a reinforcement learning model, using a Q-network structure, and the construction steps are as follows: Step 1: Design the state representation dimension; Step 2: Design the action representation dimension; Step 3: Construct a Q-network; The Q-network includes 1 input layer, 2 hidden layers, and 1 output layer. The input layer corresponds to the state representation dimension, and the corresponding number of nodes is 136; the hidden layer includes a first hidden layer and a second hidden layer. The activation function of the first hidden layer is the Relu function, and the number of nodes is 64. The structure of the second hidden layer is the same as that of the first hidden layer; the output layer corresponds to the action representation dimension, and the number of nodes is 32, which is a linear activation function; Step 4: Set the reward function: Step 5: Set the optimization execution plan: The optimizer is Adam, the learning rate is preset as v, the capacity of the experience replay pool is preset as u, it is preset to update the Q-network every x steps, and the ε-greedy strategy is used to select actions; v, u, and x are all positive real numbers.

4. The intelligent management method according to claim 3, wherein, The design state representation dimensions include discrete states of design churn probability, discrete states of payment potential, discrete states of level number, and discrete states of level information; the churn probability includes low, medium, and high; the payment potential includes low, medium, and high; the level information includes difficulty parameter level and reward value level; the difficulty parameter level is divided into easy, medium, and difficult in ascending order; the reward value level is divided into first level, second level, and third level in ascending order; the discrete states of level information include discrete states of difficulty parameter level and discrete states of reward value level; The discrete states of churn probability are: low corresponds to the vector [1, 0, 0], medium corresponds to the vector [0, 1, 0], and high corresponds to the vector [0, 0, 1]; One-hot encoding is used; The discrete states of payment potential are: low corresponds to the vector [1, 0, 0], medium corresponds to the vector [0, 1, 0], and high corresponds to the vector [0, 0, 1]; One-hot encoding is used; The discrete states of level number are a finite number set, and the finite number set is from 01 to 09, and One-hot encoding is used; the discrete states of difficulty parameter level are: easy corresponds to the vector [1, 0, 0], medium corresponds to the vector [0, 1, 0], and difficult corresponds to the vector [0, 0, 1]; One-hot encoding is used; The discrete states of reward value level are: first level corresponds to the vector [1, 0, 0], second level corresponds to the vector [0, 1, 0], and third level corresponds to the vector [0, 0, 1]; One-hot encoding is used.

5. An intelligent management method according to claim 3, characterized in that, The design action representation dimensions include the adjustment action range of the difficulty parameter level and the adjustment action range of the reward value level; The adjustment action range of the difficulty parameter level: -1, 0, and 1, and One-hot encoding is used; The adjustment action range of the reward value level: -1, 0, and 1, and One-hot encoding is used.

6. The intelligent management method according to claim 3, wherein The reward function formula is: multiply the discount rate by the maximum target Q value of the optimal action in the next state, and sum it with the immediate reward value to calculate the target Q value of the current action in the current state.

7. An intelligent management method according to claim 6, characterized in that The calculation method of the immediate reward value R includes: Subtract the adjusted churn probability from the pre-adjusted churn probability to obtain the change value of the churn probability; subtract the adjusted payment potential from the pre-adjusted payment potential to obtain the change value of the payment potential; perform weighted calculation on the change value of the churn probability, the change value of the payment potential, the adjustment action amplitude value of the churn probability, and the adjustment action amplitude value of the payment potential to obtain the immediate reward value.

8. An intelligent management method according to claim 6, characterized in that The training optimization method of the reinforcement learning model includes: Obtain c sets of historical training data, where c is a positive integer greater than 1, and the historical training data includes churn probability, payment potential, and optimal adjustment strategies; divide the c sets of historical training data into a training set and a test set with a ratio of 4:1; initialize the experience replay pool, start from a random initial state, select an action using the ε-greedy strategy, execute the action, observe the reward and the next state, store the experience in the replay pool, and every δ steps, where δ is a preset value, randomly sample experience samples from the replay pool for learning; the learning method includes: sampling an (s, a, a', s'), as the current state; as the current action; as the next state; as the next state the optimal action under; calculate the target Q-value of the current action; and calculate the loss function based on the target Q-value of the current action and the actually preset reward value.

9. An intelligent management platform system for implementing the intelligent management method according to any one of claims 1-8, characterized in that, including: The first acquisition module is used to acquire the user ID of the user side and the corresponding user behavior data within adjacent unit time; The preprocessing module preprocesses the user behavior data to obtain preprocessed data; The behavior prediction module, based on the preprocessed data, inputs a pre-trained behavior prediction model to predict the churn probability and payment potential of the user in the next unit time; The second acquisition module is used to acquire the level number in the game and the corresponding level information; The dynamic adjustment module inputs the predicted user churn probability and payment potential within the next unit time into a pre-constructed level adjustment model based on the level number and the corresponding level information, and outputs the optimal adjustment strategy. The execution module generates an adjustment instruction according to the optimal adjustment strategy and adjusts the levels in the corresponding user ID.

Images