Game data processing method and device, storage medium and electronic equipment
By using multiple unique traffic hierarchy and configuration parameters in game products, dividing the sample set of experimental objects and running different game contents, the problem of low accuracy of A/B experimental results in game products in the existing technology is solved, and the problem of low accuracy of A/B experimental results is achieved. Higher accuracy and applicability of experimental results.
Patent Information
- Application Number
- CN202310594904.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-23
- Publication Date
- 2024-11-26
AI Technical Summary
The existing A/B experimental technology has low accuracy in the results due to complexity and social attribute characteristics in game products, and the existing shunt units and analysis units need to be consistent, which cannot adapt to individual user differences, affecting the accuracy of experimental results.
By preconfiguring multiple traffic layers, each layer has unique shunt configuration parameters and analysis configuration parameters, the target traffic layer is determined based on the game content to be tested, the experimental object sample set is divided into different groups, and runs in each group Different game content to obtain game value data.
It improves the accuracy of experimental results, avoids unfair resource allocation and social public opinion pressure, is suitable for complex content in game products, takes into account individual user differences, and enhances the reliability of experimental results.
Smart Images

Figure CN119015713A_ABST
Abstract
Description
Background Art
[0002] With the growing demand for application updates on the Internet, when developers add new features or develop new strategies for applications, the applications need to conduct local verification of the new features or new strategies added by the above applications with a small sample of traffic through control experiments (A / B experiments) before the full traffic is launched, so as to analyze the experimental results of the control experiments and adjust or determine whether to launch the new features or new strategies with full traffic based on the experimental analysis results.
[0003] In the relevant technical scheme, in the A / B experiment designed according to the game content to be tested for the game product, different control experiments are usually directly and randomly assigned to different traffic layers for parallel execution, and the experimental samples are only diverted into the control group and the experimental group on a device basis.
[0004] However, due to the complexity, diversity and social attribute characteristics of the game content corresponding to game products, the above-mentioned existing technical solutions are not applicable to game products, and are very likely to introduce user behavior differences caused by non-experimental content itself, resulting in low accuracy of the results of testing game content.
[0005] Therefore, there is an urgent need for a game data processing method for game products. Summary of the invention
[0006] The present disclosure provides a game data processing method, a game data processing device, a computer storage medium and an electronic device, thereby improving the accuracy of the results of verifying game content.
[0007] In a first aspect, an embodiment of the present disclosure provides a game data processing method, the method comprising: determining a target traffic layer matching the game content to be tested from a plurality of pre-configured traffic layers according to the game content to be tested corresponding to a game product; dividing an experimental object sample set into a first object group and a second object group based on a diversion configuration parameter corresponding to the target traffic layer; running a game product containing the game content to be tested in the experimental objects of the first object group to obtain first game data corresponding to the first object group, and running a game product not containing the game content to be tested in the experimental objects of the second object group to obtain second game data corresponding to the second object group; and comparing the first game data and the second game data based on the analysis configuration parameters corresponding to the target traffic layer to obtain game value data corresponding to the game content to be tested in the game product.
[0008] In a second aspect, an embodiment of the present disclosure provides a game data processing device, which includes: a traffic layer determination module is used to determine a target traffic layer that matches the game content to be tested from a plurality of pre-configured traffic layers according to the game content to be tested corresponding to the game product; a sample division module is used to divide the experimental object sample set into a first object group and a second object group based on the diversion configuration parameters corresponding to the target traffic layer; a game running module is used to run the game product containing the game content to be tested in the experimental objects of the first object group to obtain first game data corresponding to the first object group, and to run the game product not containing the game content to be tested in the experimental objects of the second object group to obtain second game data corresponding to the second object group; and an experimental analysis module is used to compare the first game data and the second game data based on the analysis configuration parameters corresponding to the target traffic layer to obtain game value data corresponding to the game content to be tested in the game product.
[0009] In a third aspect, an embodiment of the present disclosure provides a computer-readable storage medium having a computer program stored thereon, which implements the above-mentioned game data processing method when executed by a processor.
[0010] In a fourth aspect, an embodiment of the present disclosure provides an electronic device, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the above-mentioned game data processing method by executing the executable instructions.
[0011] The technical solution disclosed in this disclosure has the following beneficial effects:
[0012] The above-mentioned game data processing method, on the one hand, allocates different control experiments to corresponding traffic layers according to the game content to be tested, that is, the traffic layers and the control experiments have a one-to-one correspondence, and different traffic layers have different diversion configuration parameters and analysis configuration parameters, so that the diversion configuration parameters and analysis configuration parameters that match the different contents to be tested are determined, so the flexibility and diversity of the diversion configuration parameters are more suitable for game products with complex game content, avoiding the problem that the related technology must use users or terminal devices as experimental diversion units, resulting in unfair resource allocation of game products and social public opinion pressure, and then introducing variables of non-experimental content to cause low accuracy of experimental results, thereby achieving the technical effect of improving the accuracy of experimental results. On the other hand, since the diversion configuration parameters and the analysis configuration parameters can be the same or different, taking into account the game characteristics of individual differences in users in game products, it avoids the technical problem that the diversion configuration parameters and the analysis configuration parameters in the related technology must be kept consistent, resulting in limited scope of application of the experiment and affecting the accuracy of game content testing, and achieving a technical effect that is more suitable for game products and further improves the accuracy of experimental results.
[0013] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS
[0014] The drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and are used together with the specification to explain the principles of the present disclosure. Obviously, the drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0015] Figure 1 A system architecture diagram of a game data processing system in this exemplary embodiment is schematically shown;
[0016] Figure 2 A flowchart schematically shows a method for processing game data in this exemplary embodiment;
[0017] Figure 3 A schematic diagram of dividing multiple traffic layers in this exemplary embodiment is schematically shown;
[0018] Figure 4 A flowchart for determining the sample size in the experimental subject sample set is schematically shown in this exemplary embodiment;
[0019] Figure 5 A flowchart of dividing experimental subjects in an experimental subject sample set into experimental groups in this exemplary embodiment is schematically shown;
[0020] Figure 6 A flowchart schematically shows a complete method for processing game data in this exemplary embodiment;
[0021] Figure 7 A schematic diagram of the structure of a game data processing device in this exemplary embodiment is shown;
[0022] Figure 8 The structure diagram of an electronic device in this exemplary embodiment is schematically shown. DETAILED DESCRIPTION
[0023] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, exemplary embodiments can be implemented in a variety of forms and should not be construed as being limited to the examples set forth herein; on the contrary, these embodiments are provided so that the present disclosure will be more comprehensive and complete, and the concepts of the exemplary embodiments are fully conveyed to those skilled in the art. The described features, structures, or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced while omitting one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known technical solutions are not shown or described in detail to avoid obscuring various aspects of the present disclosure.
[0024] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.
[0025] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the steps. For example, some steps may be decomposed, while some steps may be combined or partially combined, so the actual execution order may change according to the actual situation.
[0026] In order to help those skilled in the art better understand the technical solution of the present disclosure, the relevant contents involved in the technical solution of the present disclosure are introduced below.
[0027] (1) Control experiment: Also known as A / B experiment, it is one of the methods to test whether there is a systematic error in the measurement method. A / B experiment is an experiment to compare different versions, different styles, different rules, etc. of new functions or new strategies. One traffic experiment corresponds to at least two control experiments, namely the experimental group and the control group. Its core principle is to create an experimental environment through random diversion, so that the only difference between the experimental group and the control group is the part to be tested, and then confirm the relationship with the experimental results based on the difference between the groups through a large amount of data.
[0028] (2) Daily Active User (DAU): used to represent the total number of users who log in daily.
[0029] (3) User traffic: refers to the number of users visiting a product.
[0030] (4) A / B experiment diversion: It is a method of presetting multiple traffic groups for the upcoming experiment, and using a specific strategy to make part of the traffic "flow" into one or some traffic groups, and the other part of the traffic "flow" into other traffic groups. Experiment diversion is to ensure the scientific nature of the A / B experiment, that is, each experiment has only one variable (i.e., the element to be tested), and all other elements of each traffic group remain consistent.
[0031] (4) Unit of Diversion: It is used to define the unit or object for grouping traffic when conducting A / B experiment diversion. Taking game products as an example, the unit of diversion can be a terminal device, a natural person, a user account, or other items.
[0032] (5) Unit of Analysis: refers to the unit or object for conducting experimental analysis on the data collected after completing the A / B experiment. Taking game products as an example, the unit of analysis can also be a terminal device, a natural person, a user account, or other items.
[0033] (6) Traffic stratification: A stratified experiment is to establish a multi-layer experimental structure (traffic layer) by combining multiple experiments. The user traffic used in each experimental structure can continue to flow into the next experimental structure. The existing traffic stratification methods include orthogonal experiments and mutually exclusive experiments.
[0034] Orthogonal experiment: Each experiment occupies a traffic layer exclusively, that is, different experiments are placed in different traffic layers to ensure the accuracy of the experiment. The traffic layers are orthogonal to each other, and the experimental traffic can penetrate and overlap.
[0035] Mutually exclusive experiments: Experiments divide the experimental traffic in the same traffic layer, and the traffic between different experiments does not overlap.
[0036] (7) Significance test: Make an assumption about the parameters of the population (random variable) or the distribution form of the population in advance, and then use the sample information to determine whether this assumption (alternative hypothesis) is reasonable, that is, to determine whether the actual situation of the population is significantly different from the original hypothesis.
[0037] A / B experiments are now widely used by major Internet companies and have gradually penetrated into the entire life cycle of their product and service iterations. The core statistical principle of A / B experiments is to verify whether the difference in sample means between the experimental group and the control group is large enough relative to the sample variance through hypothesis testing, so that it can be said with certainty that the difference in sample means between the experimental group and the control group is stable rather than caused by fluctuations.
[0038] The game data processing method provided by the exemplary embodiment of the present disclosure can be applied to the application scenario of determining the causal relationship between the factors to be tested and the experimental results. For example, in the full life cycle of the iteration of Internet products and services, in order to analyze whether a factor to be tested will lead to an experimental result. Assume that in the following exemplary scenario, game product A is launched during the New Year's special sale, and a large number of people participate in the game product A on the same day, but a few days later, the DAU of the game product A drops sharply (the experimental result of the A / B experiment). Therefore, it is necessary to determine through an A / B experiment whether it is caused by the natural decline in traffic during the Chinese New Year (a factor to be tested in the A / B experiment), or the game content problem of game product A (another factor to be tested in the A / B experiment), or both of the above factors. At this time, two A / B experiments can be designed according to the above two factors to be tested for determination.
[0039] In the existing relevant technical solutions, a complete A / B experiment includes: experimental design stage - A / A experimental stage - A / B experimental stage - data recovery and analysis stage.
[0040] 1) Experimental design phase: clarify the experimental objectives - determine the experimental diversion unit - select the core indicators of the experiment (retention rate / paying / active users or other performance indicator parameters, etc.) - calculate user traffic and the effective duration of the experiment.
[0041] 2) A / A experiment stage: Experimental groups are performed according to the experimental design and traffic layers are selected. Versions (not containing any new content) are found in each traffic group. The experiment is run for a period of time and data is collected to determine whether there are natural differences between the experimental groups based on the data. If there are natural differences, the groups are regrouped. If there are no natural differences, the grouping is reasonable and the next A / B experimental steps can be carried out.
[0042] 3) A / B experiment stage: According to the experimental design, the content is distributed according to the groups in the A / A experiment. The control group is distributed with the original version without any changes (i.e., without any new content), and the experimental group is distributed with the target version containing new content;
[0043] 4) Data recovery and analysis phase: Based on the distribution of core indicators of the experimental and control groups, determine whether the experiment has a significant effect (i.e., whether the iteration / new version has business value).
[0044] For game products, the above-mentioned existing related technical solutions have the following technical problems:
[0045] A. Existing diversion units are not suitable for game products: In related technologies, grouping the collected experimental object samples using users or terminal devices as experimental diversion units will cause unfair resource allocation and public opinion pressure.
[0046] For example, if a developer adds a new character skill to a game product in the game scene, and the user retention rate of the game product increases significantly on the day after it goes online, the A / B experiment can be used to analyze whether the new game skill is the reason for the significant increase in user retention rate. Assuming that adding new character skills is the new game version of the game product, and not adding new character skills is the old game version of the game product, for players A and B on the same server, if player A experiences the new game version and gets a reward, but player B on the same server cannot experience the content of the above new game version and cannot get the corresponding reward. On the one hand, since players on the same server often have a competitive relationship, it will be unfair to player B at this time, resulting in differences in user behavior caused by non-experimental content itself (that is, player B quits because he feels unfair, not because the new character skills are not good), thereby affecting the accuracy of the experimental results. On the other hand, due to the strong social attributes of game products, the unfair treatment experience of players A and B will spread quickly among game players through communication methods such as friend chats, public channel speeches, and game forums, which will bring huge public opinion risks.
[0047] B. The existing traffic stratification method is not suitable for game products: In most Internet products, a large number of A / B experiments with different game content are usually required to be launched at the same time. In order to prevent the experiments from interfering with each other and keep them independent, traffic stratification is technically performed through orthogonal and mutually exclusive methods, placing different experiments in different traffic layers to ensure experimental accuracy.
[0048] However, in the game scenario, since the production and implementation of game content is more complicated (many contents involve not only programs and algorithms, but also art and copywriting), it is difficult to implement a large number of A / B experiments online at the same time in actual operation. Although the demand for a large number of parallel experiments in game products is not high, a scientific layered approach is still needed to standardize the production process of experiments and test content.
[0049] C. The existing experimental diversion units and experimental analysis units need to be consistent, and are not applicable to game products: For example, in game products, if the server is used as the experimental diversion unit to distribute experimental content a (that is, the same game content is distributed to all players in the server), when conducting experimental analysis, the server must also be used as the experimental analysis unit for analysis.
[0050] However, in the game, there are differences between players on different servers. When the game content is distributed using the server as the experimental diversion unit, it is impossible to determine the number of servers used for the experiment. At the same time, the above conditions still require the analysis of the experimental effect according to a single user (that is, the experimental analysis unit needs to be located at the user). For example, commonly used operational indicators are average online time per person, average payment per person, etc., rather than performance indicators such as server payment and total server time (because such performance indicators have a lower reference significance, and the game pays more attention to the individual differences of game players), and there is currently no relevant technical solution for reference.
[0051] Taking the above problems into consideration, the exemplary embodiment of the present disclosure proposes a method for processing game data, which is to pre-divide a plurality of traffic layers, each of which has its corresponding diversion configuration parameters and analysis configuration parameters, that is, there are differences in the diversion configuration parameters and analysis configuration parameters of different traffic layers, and there are also differences in the diversion configuration parameters and analysis configuration parameters of the same traffic layer, so that it can be more accurately applied to game products with more complex game content. It avoids the problem that the related technology uses fixed diversion configuration parameters for diversion, which leads to the introduction of user behavior differences caused by non-experimental content itself, which affects the accuracy of the results of testing game content, thereby improving the accuracy of the experimental results. At the same time, the experimental diversion unit and experimental analysis unit corresponding to each traffic layer do not need to be consistent, thereby taking into account the differences between players and further improving the accuracy of the experimental results of the control experiment.
[0052] In order to solve the above problems, the present invention proposes a game data processing method and device, which can be applied to Figure 1 In the system architecture of the exemplary application environment shown.
[0053] like Figure 1 As shown, the system architecture 100 may include one or more of terminal devices 101, 102, 103, 104, a network 105, and a server 106. The network 105 is used to provide a medium for communication links between the terminal devices 101, 102, 103, 104 and the server 106. The network 105 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc. The terminal devices 101, 102, 103, 104 may be, for example, smart phones, personal digital assistants (PDAs), laptop computers, servers, desktop computers, or any other computing devices with networking capabilities, but are not limited thereto.
[0054] It should be understood that Figure 1The number of terminal devices, networks and servers in the example is only illustrative. Any number of terminal devices, networks and servers may be provided as required. For example, server 106 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.
[0055] The game data processing method provided in the embodiment of the present disclosure can be executed on the server 106, and accordingly, the game data processing device is generally set in the server 106. The game data processing method provided in the embodiment of the present disclosure can also be executed in the terminal device, and accordingly, the game data processing device can also be set in the terminal device. The game data processing method provided in the embodiment of the present disclosure can also be partially executed in the server 106 and partially executed in the terminal, and accordingly, some modules of the game data processing device can be set in the server 106, and some modules can be set in the terminal device.
[0056] For example, in an exemplary embodiment, the experimental designer may determine the experimental content and experimental objectives of the control experiment in advance during the experimental design phase. Multiple users may play games through terminal devices 101, 102, 103, and 104, respectively. The server 106 may collect game data of the terminal devices 101, 102, 103, and 104 as a sample set of experimental objects. The server 106 determines the target traffic layer that matches the game content to be tested from the pre-configured multiple traffic layers according to the game content to be tested corresponding to the game product; and divides the sample set of experimental objects into a first object group and a second object group based on the diversion configuration parameters corresponding to the target traffic layer. Then, the server 106 may run the game product containing the game content to be tested in the experimental object of the first object group to obtain the first game data corresponding to the first object group, and run the game product that does not contain the game content to be tested in the experimental object of the second object group to obtain the second game data corresponding to the second object group; based on the analysis configuration parameters corresponding to the target traffic layer, the first game data and the second game data are compared to obtain the game value data corresponding to the game content to be tested in the game product.
[0057] However, it is easy for those skilled in the art to understand that the above application scenarios are only used as examples and are not limited to this in the present exemplary embodiment.
[0058] The following uses the above-mentioned server as the execution subject and illustrates the application of the game data processing method to the above-mentioned server as an example. Figure 2A flowchart schematically shows a method for processing game data in this exemplary embodiment, see Figure 2 The game data processing method provided by the embodiment of the present disclosure includes the following steps S201 to S204:
[0059] Step S201: According to the game content to be checked corresponding to the game product, a target traffic layer matching the game content to be checked is determined from a plurality of pre-configured traffic layers.
[0060] Step S202: Divide the experimental object sample set into a first object group and a second object group based on the diversion configuration parameters corresponding to the target traffic layer.
[0061] Step S203: Run the game product containing the game content to be tested in the experimental objects of the first object group to obtain the first game data corresponding to the first object group, and run the game product not containing the game content to be tested in the experimental objects of the second object group to obtain the second game data corresponding to the second object group.
[0062] Step S204: Based on the analysis configuration parameters corresponding to the target traffic layer, the first game data and the second game data are compared to obtain game value data corresponding to the game content to be tested in the game product.
[0063] exist Figure 2 In the technical scheme shown, the method determines a target traffic layer that matches the game content to be tested from multiple pre-configured traffic layers according to the game content to be tested corresponding to the game product; divides the experimental object sample set into a first object group and a second object group based on the diversion configuration parameters corresponding to the target traffic layer; runs the game product containing the game content to be tested in the experimental objects of the first object group to obtain first game data corresponding to the first object group, and runs the game product that does not contain the game content to be tested in the experimental objects of the second object group to obtain second game data corresponding to the second object group; compares the first game data and the second game data based on the analysis configuration parameters corresponding to the target traffic layer to obtain game value data corresponding to the game content to be tested in the game product.
[0064] On the one hand, different control experiments are allocated to corresponding traffic layers according to the game content to be tested, that is, there is a one-to-one correspondence between the traffic layers and the control experiments, and different traffic layers have different diversion configuration parameters and analysis configuration parameters, so that the diversion configuration parameters and analysis configuration parameters that match the different content to be tested are determined. Therefore, the flexibility and diversity of the diversion configuration parameters are more suitable for game products with complex game content, avoiding the problem that the related technology must use users or terminal devices as experimental diversion units, resulting in unfair resource allocation of game products and social public opinion pressure, and then introducing variables of non-experimental content to cause low accuracy of experimental results, thereby achieving the technical effect of improving the accuracy of experimental results. On the other hand, since the diversion configuration parameters and the analysis configuration parameters can be the same or different, taking into account the game characteristics of individual users in the game product, it avoids the technical problem that the diversion configuration parameters and the analysis configuration parameters in the related technology must be kept consistent, resulting in limited scope of application of the experiment and affecting the accuracy of game content testing, and achieving a technical effect that is more suitable for game products and further improves the accuracy of experimental results.
[0065] Next, we will combine Figure 2 The specific implementation methods of each step in the embodiment of the game data processing method are described in detail:
[0066] In step S201, according to the game content to be checked corresponding to the game product, a target traffic layer matching the game content to be checked is determined from a plurality of pre-configured traffic layers.
[0067] Among them, the content to be tested is predetermined by the experimental designers during the game design stage based on the market performance of the game product.
[0068] Exemplarily, the experimental designer can design a control experiment for the game content to be tested. For example, a game product A simultaneously launches equipment recommendation algorithm X and pet recommendation algorithm Y, and the payment rate of game product A on the same day after the launch increases. In order to determine whether the result of the increase in the payment rate of game product A is caused by equipment recommendation algorithm X or pet recommendation algorithm Y. At this time, it can be determined through A / B experiments. In the experimental design stage, the experimental goal of an A / B experiment can be determined as follows: whether the experimental result that causes the increase in the payment rate of game product A is caused by equipment recommendation algorithm X, and the corresponding experimental content is the initial game product that does not contain equipment recommendation algorithm X and the target game product that contains equipment recommendation algorithm X; the experimental goal of another A / B experiment is: whether the experimental result that causes the increase in the payment rate of game product A is caused by pet recommendation algorithm Y, and the corresponding experimental content is the initial game product that does not contain pet recommendation algorithm Y and the target game product that contains pet recommendation algorithm Y.
[0069] Among them, the game content to be tested in the control experiment corresponds one to one to the traffic stratification, that is, based on the correspondence between the game content to be tested and the traffic stratification, the target traffic stratification that can be divided into the control experiment can be determined according to the game content to be tested.
[0070] For example, the target traffic layer corresponding to the A / B experiment can be determined from multiple traffic layers pre-divided in the experimental traffic. Determining the target traffic layer that can be allocated to the A / B experiment through the game content to be tested in the A / B experiment makes it easier to design and manage the experiment.
[0071] In some example embodiments of the present disclosure, before determining a target traffic layer that matches the game content to be tested from a plurality of pre-configured traffic layers, it is necessary to divide the experimental traffic into a plurality of traffic layers in advance.
[0072] Dividing the experimental traffic into multiple traffic layers makes it easier to execute multiple A / B experiments in parallel in different traffic layers, thereby improving the efficiency of executing multiple A / B experiments and ensuring the accuracy of the experiments.
[0073] In the relevant technical solutions, the existing traffic stratification mechanism mainly stratifies the experimental traffic (experimental traffic can also be called the total traffic pool) in an orthogonal and mutually exclusive manner, and randomly allocates a large number of A / B experiments to different traffic strata. However, for game products, since the production and implementation of game content is also more complicated (for example, game products not only involve the programs and algorithms for implementing the game, but also involve art production and game copywriting), it is difficult to implement a large number of A / B experiments online at the same time in practice, and the method of randomly allocating A / B experiments is not suitable for game products. Therefore, it is extremely necessary to implement a method for traffic stratification of experimental traffic based on the characteristics of game products.
[0074] In some example embodiments of the present disclosure, multiple traffic layers are divided based on the game stages included in the game product. When executing step S201, according to the game content to be tested corresponding to the game product, a target traffic layer that matches the game content to be tested is determined from the pre-configured multiple traffic layers based on the game stages corresponding to the game content to be tested.
[0075] Exemplarily, multiple traffic layers are mutually orthogonal. Among them, multiple traffic layers that are orthogonal to each other can also be called orthogonal layers, which can be understood as traffic layers that do not interfere with each other vertically, that is, do not affect each other, so as to test the effect of traffic reuse, specifically, the impact of each experiment in the previous traffic layer is evenly distributed to the next traffic layer, so that the experimental effects cancel each other out, thereby not affecting the experimental evaluation of the next traffic layer.
[0076] Exemplarily, based on different game stages, the total traffic pool can be divided into multiple orthogonal traffic layers, so that the target traffic layer is determined according to the game stage corresponding to the game content to be tested.
[0077] The following will be combined Figure 3 An exemplary description is given of the process of dividing the total traffic pool into multiple traffic layers based on different game stages.
[0078] Figure 3 A schematic diagram of dividing multiple traffic layers in this exemplary embodiment is schematically shown. Figure 3 As shown, the experimental traffic can be divided into four traffic layers, namely the service launch strategy traffic layer, the client traffic layer, the server traffic layer, and the cross-server strategy traffic layer, and each of the above traffic layers is applicable to the experimental content of the game product at different game stages. The above four traffic layers are displayed in the logical order of the time when the players play the game, and there is no priority order in reality.
[0079] The following is an explanation of the experimental contents of the four traffic layers divided above and applicable to different game stages:
[0080] (1) Service launch strategy traffic layer: Applicable to server-related AB experiments (for example, new server launch conditions, new server launch time, new server naming method, etc.) and related content of the purchase volume diversion strategy. Among them, the purchase volume diversion strategy is a strategy that determines the allocation of users purchased in different ways to different servers.
[0081] For example, the server launch strategy traffic layer applies to the game stage before the server is launched, when the player opens the game client but exits the game client before creating a character.
[0082] (2) Client traffic layer: Applicable to the character creation stage and all game content before the player officially enters the game. For example, all game content before the player officially enters the game can include: the interface for creating game characters, the display form of game characters, the opening animation of the game product, etc.
[0083] (3) Server traffic layer: Applicable to all server-related content after the game character enters the game, that is, all control experiments that do not involve multi-server scenarios can be assigned to the server traffic layer. For example, it includes but is not limited to the gameplay of game products in the server and the specific content and methods of activities, player recharge and purchase content, and other user interface (UI) related content in the game.
[0084] (4) Cross-server strategy traffic layer: Applicable to all strategies in multiple server scenarios. That is, if the experimental content of the control experiment involves multi-server related algorithm strategies, the control experiment will be classified into the cross-server strategy traffic layer. For example, it includes but is not limited to cross-server player team matching, cross-server product sales recommendation, cross-server battle player matching and other strategies.
[0085] Compared with related technical solutions, dividing the experimental content of game products at different game stages into multiple mutually orthogonal traffic layers is more intuitive and convenient for A / B experiment management. It is also suitable for the complexity of game content of game products. Since the game content at different game stages has different characteristics, it is convenient to select the diversion configuration parameters and analysis configuration parameters that are more suitable for the game stage based on different game stages, thereby improving the accuracy of the experimental results of the control experiment designed for the game content to be tested.
[0086] In step S202, based on the diversion configuration parameters corresponding to the target traffic layer, the experimental object sample set is divided into a first object group and a second object group;
[0087] The diversion configuration parameter, namely the experimental diversion unit, is used to characterize the unit for dividing all experimental objects in the experimental object sample set.
[0088] For example, since the player plays the game through a terminal device, each experimental object in the experimental object sample set is usually a terminal device used by the player. Assuming that the experimental object sample set composed of terminal devices is grouped into groups based on the server as a unit, the diversion configuration parameter is the server.
[0089] The traffic diversion configuration parameters corresponding to the traffic stratification are related to the game stage, so the traffic diversion configuration parameters corresponding to different traffic stratifications are different.
[0090] For example, Figure 3 Taking the four traffic layers shown as an example, the diversion configuration parameters of each traffic layer (ie, the experimental diversion unit) are explained.
[0091] A. Service launch strategy traffic layer: Since the experimental content of the control experiment assigned in this traffic layer is the game stage where the player opens the game client but exits the game client before creating a character, it can be seen that the characteristic information of the experimental content is irrelevant to the game character, and it is usually necessary to distinguish each device that starts the client. Therefore, in the service launch strategy traffic layer, the device is used as the experimental diversion unit, so that the large number of collected device samples are divided into multiple different groups in the service launch strategy traffic layer.
[0092] B. Client traffic layer: usually the terminal device is used as the experimental diversion unit.
[0093] C. Server traffic layer: This traffic layer includes various user interaction interfaces, game characters, game play methods, rules, etc. When the experimental content involves fairness characteristics, the server is used as the experimental diversion unit, and all players in a single server can be taken as a whole, thereby dividing a large number of server samples into multiple different groups; if the experimental content does not involve fairness characteristics, for example, the experimental content involves user interaction interfaces, the game characters are used as the experimental diversion units.
[0094] For example, the server traffic layer is allocated for a control experiment in the formal game phase. Assuming that the content of a control experiment is to test the game effect of a new game scene, if player A experiences the new game version and gets a reward, while player B on the same server as player A does not experience the new game version (that is, uses the original game version), and thus cannot get the corresponding reward, then the experimental content of the control experiment involves fairness characteristics. Therefore, it is necessary to ensure that all players on a server experience the same game version (that is, all players on a server experience the new game version, or all experience the original game version). For this control experiment, the server is used as the experimental diversion unit.
[0095] D. Cross-server strategy traffic layer: This traffic layer is applicable to the game stage across servers, and usually uses the server as the experimental diversion unit.
[0096] In an optional embodiment of the present disclosure, in the process of executing step S202 to divide the experimental samples into different groups based on the experimental diversion unit to obtain multiple experimental groups, that is, in the process of diverting the experimental samples into different groups, the diversion is usually based on the identification information of the experimental samples (for example, identity document (ID)). For example, when the server is used as the experimental diversion unit, the diversion is performed by the server ID.
[0097] Exemplarily, assuming that there are N experimental groups in total, the device ID or game character ID and server ID can be calculated by double Xiha. Since the calculation result may be an integer or a decimal, in order to prevent the calculation result from being greater than N and accurately divide the groups, the above calculation result can be processed modulo N, so that different terminal devices can be divided into different groups based on the modulo result. For example, the device ID or game character ID and server ID with a modulo result of 1 are divided into experimental group 1, and the device ID or game character ID and server ID with a modulo result of 2 are divided into experimental group 2, until all experimental samples in the experimental object sample set are divided.
[0098] In some example embodiments of the present disclosure, when executing step S202 to divide the experimental object sample set into a first object group and a second object group based on the diversion configuration parameters corresponding to the target traffic layer, it is necessary to pre-calculate the sample size and total experimental period of the experimental object sample set.
[0099] The sample size of the experimental object sample set is used to determine the number of devices scheduled to participate in the control experiment every day; and the total experimental period is used to determine the total duration of the control experiment.
[0100] The total experimental period is related to the sample size of the experimental object sample set. For example, a control experiment requires a sample size of 1000 for the experimental object sample set. If only 100 users log in to the game every day, the total duration of the control experiment is at least 10 days. The above process of determining the sample size of the experimental object sample set and the total experimental period is determined in the experimental design stage.
[0101] The process of determining the sample size of a sample set of real objects will be exemplarily described below in conjunction with specific embodiments.
[0102] Figure 4 A flow chart for determining the sample size in the experimental subject sample set in this exemplary embodiment is schematically shown; see Figure 4 In an optional embodiment of the present disclosure, the above step S202 divides the experimental object sample set into a first object group and a second object group based on the diversion configuration parameters corresponding to the target traffic layer, and includes the following steps S401 to S406:
[0103] Step S401: Determine corresponding diversion configuration parameters and analysis configuration parameters according to the target traffic layer.
[0104] The analysis configuration parameter is a unit for analyzing the experimental data obtained after running the A / B experiment.
[0105] In the experimental design stage, after determining the target traffic layer that can be allocated to the control experiment, the diversion configuration parameters and analysis diversion configuration parameters can be determined according to the target traffic layer. For example, if the experiment is to determine the impact of the pet recommendation algorithm added to a game product on the average online time per person, the diversion configuration parameter is the server, and the analysis configuration parameter is the game character (because the control experiment is to determine the data change process of the game character's online time after using the pet recommendation algorithm, the experimental analysis unit is determined to be the game character).
[0106] Combination Figure 3The four traffic layers shown, when it is determined based on the game content to be tested that the control experiment can be allocated to the service launch strategy experiment layer, the client experiment layer, or the control experiment is allocated to the server experiment layer and does not involve the principle of fairness, the game character ID and the terminal device ID can be used as the experimental diversion unit and the experimental analysis unit (that is, the control experiment uses the game character or terminal device as a unit to execute different experimental contents, and the performance indicators of the game character or terminal device are also analyzed during the data analysis stage). When the control experiment uses the server as the experimental diversion unit, the game character is still used as the experimental analysis unit for data analysis during the data analysis stage (that is, focus on the data changes of the game character on specific performance indicators).
[0107] Step S402: determine whether the diversion configuration parameters are the same as the analysis configuration parameters;
[0108] If the diversion configuration parameters corresponding to the target traffic layer are the same as the analysis configuration parameters, step S403 is executed to determine the first sample size threshold corresponding to the experimental object sample set.
[0109] In some example embodiments of the present disclosure, a first sample size threshold corresponding to the experimental object sample set is determined based on the total variance of the experimental objects on the target performance indicator, the significance threshold and the standard deviation of the mean of the experimental objects on the target performance indicator, the statistical power and the standard deviation of the mean of the experimental objects on the target performance indicator, and the minimum gap between the first object group and the second object group for the target performance indicator.
[0110] The first sample size threshold is used to represent the minimum sample size of the experimental object sample set required for a control experiment in which the experimental diversion unit is consistent with the experimental analysis unit.
[0111] Exemplarily, when determining the first sample size threshold that can be used in the control experiment, it can be determined according to the following formula (1):
[0112] N=2S Y 2 (Z a +Z b ) 2 / x 2 (Formula 1)
[0113] In the above (Formula 1), N is the minimum sample size of the experimental subject sample set required for the control experiment, S Y 2 is the total variance of the experimental object on the target performance indicator (for all samples of all experimental groups); a is the significance threshold of the experimental effect, b is the statistical power, and Z a , Z bThey represent the standard deviation of the mean of the target performance indicator of the experimental objects corresponding to the a and b quantiles in the normal distribution respectively; x is the experimental sensitivity, which is used to characterize the minimum gap between the first object grouping and the second object grouping for the target performance indicator.
[0114] Assume that two experimental groups are divided for the control experiment. The experimental sensitivity represented by x refers to the minimum gap between the two experimental groups for performance indicator Y that can be effectively tested by the experiment. The performance indicator Y can be, for example, the average daily online time, the average daily user payment, or other performance indicators including but not limited to operational indicators. The statistical power refers to the probability of accepting a correct replacement hypothesis when the null hypothesis is wrong in the hypothesis test.
[0115] Therefore, when the diversion configuration parameters are consistent with the analysis configuration parameters, the minimum sample size required when performing a control experiment can be quickly determined based on Formula 1, so as to determine the sample size of the experimental object sample set based on the minimum sample size.
[0116] After determining the first sample size threshold, step S404 may be executed to divide the experimental object sample set whose sample quantity is not less than the first sample size threshold into a first object group and a second object group based on the diversion configuration parameters corresponding to the target traffic layer.
[0117] If the diversion configuration parameters corresponding to the target traffic layer are different from the analysis configuration parameters, step S405 is executed to determine a second sample size threshold corresponding to the experimental object sample set.
[0118] In case the diversion configuration parameters are inconsistent with the analysis configuration parameters:
[0119] In another optional embodiment of the present disclosure, if the experimental analysis unit is different from the experimental diversion unit, the target mean is determined based on the mean and variance of the number of experimental sub-objects in the experimental object, as well as the mean and variance of the target performance indicator and the variance of the target performance indicator of each experimental object in the experimental object sample set; and the second sample size threshold corresponding to the experimental object sample set is determined based on the target mean and the first sample size threshold.
[0120] The experimental object may be a server, and the experimental sub-object may be a game character in the same server.
[0121] The second sample size threshold is used to represent the minimum sample size of the experimental object sample set required for a control experiment for the case where the experimental diversion unit is inconsistent with the experimental analysis unit.
[0122] In an optional embodiment of the present disclosure, if the analysis configuration parameter is different from the diversion configuration parameter, then.
[0123] Taking the experimental diversion unit as the server ID and the experimental analysis unit as the game character ID as an example, assuming that there are k servers in total, the mean of the individual game characters on a performance indicator Y is approximately calculated based on the Delta-method (ignoring higher-order terms above the third order):
[0124]
[0125] In the above (Formula 2), is the mean value of the individual game character on a performance indicator Y (i.e., the target mean value); k is the total number of servers opened; m n , S n are the mean and variance of the number of roles in the server (i.e., the mean and variance of the number of experimental sub-objects in the above experimental object); m s , S s are the mean and variance of the server's performance indicators Y and (i.e., the mean and variance of the target performance indicator); S sn It is the variance of the performance index Y of all experimental subjects in the experimental subject sample (the variance of the target performance index of each experimental subject in the experimental subject sample set).
[0126] Based on the above (Formula 2), the minimum number of servers required can be determined, that is, the second sample size is:
[0127]
[0128] In the above (Formula 3), k is the minimum number of servers required to be determined, that is, the second sample size; a is the significance threshold of the experimental effect, b is the statistical power, and Z a , Z b They represent the Z values corresponding to the a and b quantiles in the normal distribution, respectively, to represent the standard deviation of the quantile distance from the sample mean corresponding to the significance threshold and the standard deviation of the quantile distance from the sample mean corresponding to the statistical power; x is the experimental sensitivity; m n , S n are the mean and variance of the number of roles in the server respectively; m s , S s are the mean and variance of Y and in the performance index of the server respectively; S sn is the variance of the performance indicator Y of all experimental subjects in the experimental subject sample.
[0129] After determining the second sample size threshold, step S406 may be executed to divide the experimental object sample set whose sample quantity is not less than the second sample size threshold into a first object group and a second object group based on the diversion configuration parameters corresponding to the target traffic layer.
[0130] The related art does not involve a technical solution for conducting a control experiment when the experimental diversion unit and the experimental analysis unit are inconsistent, for example, a process of dividing multiple experimental groups using a server as an experimental diversion unit and conducting experimental analysis using a game character as an experimental analysis unit. The exemplary technical solution provided by the present disclosure proposes a technical solution for this, thereby further improving the accuracy of the experimental results.
[0131] In step S203, the initial game product is run in a first target group among multiple first experimental groups to obtain first experimental data for the first target group, and the target game product is run in a second target group among multiple first experimental groups to obtain second experimental data for the second target group.
[0132] Exemplarily, the first target group is used to run the initial game product without the factor to be tested, which can be called the control group; while the second target group is used to run the target game product with the factor to be tested, which can be called the experimental group. It can be seen that the first target group running the initial game product and the second target group running the target game product can ensure that there is a variable with the factor to be tested. The experimental data of the first target group can provide a comparable benchmark for the experiment conducted by the second target group, so that subsequent experimental designers can perform data analysis based on the experimental data of the control group and the experimental group.
[0133] In step S204, based on the analysis configuration parameters corresponding to the target traffic layer, the first game data and the second game data are compared to obtain game value data corresponding to the game content to be tested in the game product.
[0134] Exemplarily, after obtaining the first game data corresponding to the first object grouping and the second game data corresponding to the second object grouping, the game data may be analyzed to obtain valuable experimental results.
[0135] In some example embodiments of the present disclosure, based on the analysis configuration parameters corresponding to the target traffic layer, a significance test is performed on the first game data corresponding to the first object group and the second game data corresponding to the second object group; based on the test results of the significance test, the game value data corresponding to the game content to be tested in the game product is obtained.
[0136] Exemplarily, when the experimental analysis unit corresponding to the target traffic layer performs an analysis on the first game data and the second data, a significance test may be performed on the above data.
[0137] That is, by comparing whether the gap between the first game data and the second game data is significant, the game value data can be obtained.
[0138] In an optional embodiment of the present disclosure, if the difference between the first game data and the second game data is significant, it indicates that the game product containing the game content to be tested is significantly more effective than the game product not containing the game content to be tested, and the game product containing the game content to be tested can be used to replace the game product not containing the game content to be tested for operation. That is, all experimental subjects in the experimental subject sample set are assigned to the game group corresponding to the game product containing the game content to be tested.
[0139] In another optional embodiment of the present disclosure, if the gap between the first game data and the second game data is not significant, it indicates that the effect of the game product containing the game content to be tested is not significant compared to the effect of the game product not containing the game content to be tested, and there is no need to replace the game product, that is, all experimental subjects in the experimental subject sample set are assigned to the game group corresponding to the game product that does not contain the game content to be tested.
[0140] The significance test can be used to quickly determine the contribution of the game content to be tested to the game product, thereby assisting the experimental designer in determining whether to replace the game product that does not contain the game content to be tested with a game product that contains the game content to be tested, thereby improving the efficiency of testing the game content.
[0141] Furthermore, before executing the above step S202 to run the game product containing the game content to be tested in the experimental objects of the first object group to obtain the first game data corresponding to the first object group, and running the game product not containing the game content to be tested in the experimental objects of the second object group to obtain the second game data corresponding to the second object group, since the game content of the game product is more complex and the experiment consumes a long time, in order to facilitate rapid iteration, it is also necessary to determine whether the current grouping is reasonable.
[0142] If the grouping is reasonable, step S203 can be executed to run the game product containing the game content to be tested in the experimental objects of the first object group to obtain the first game data corresponding to the first object group, and to run the game product not containing the game content to be tested in the experimental objects of the second object group to obtain the second game data corresponding to the second object group; if the current grouping is unreasonable, re-grouping is required.
[0143] Exemplarily, when performing data analysis on the first game data and the second game data, a significance test may be performed on the game data to determine whether the experimental grouping is reasonable based on the significance test result.
[0144] This disclosure will combine Figure 5 The process of judging whether the experimental grouping is reasonable is illustrated by way of example.
[0145] Figure 5A flowchart of dividing the experimental subjects in the experimental subject sample set into various experimental groups in this exemplary embodiment is schematically shown. Figure 5 In an optional embodiment of the present disclosure, when determining whether the experimental grouping is reasonable, the following steps S501 to S504 are included:
[0146] S501. Based on a preset period, obtain a plurality of first mean value sequences of a first object group with respect to a target performance indicator, and a plurality of second mean value sequences of a second object group with respect to a target performance indicator.
[0147] The first object grouping may include multiple object groups, and each object grouping corresponds to a mean value sequence.
[0148] In an optional embodiment of the present disclosure, the target performance indicator includes at least one of a user retention rate, a payment rate, a user activity rate, and a user online time.
[0149] The above-mentioned target performance indicators make it easy for experimental designers to quickly determine the experimental results based on data comparison, thereby improving the accuracy of the experimental results.
[0150] Assuming that the first object group and the second object group both contain 100 experimental objects, the preset period is 10 days, and the target performance indicator is the online duration of users, then for each day, the average online duration of users T = (t1 + t2 + ... + t100) / 100 calculated from the online durations t1, t2, ... t100 of 100 users corresponding to the 100 experimental objects can be obtained. The average online duration of users for the above 10 days can form a mean sequence.
[0151] When the first mean sequence differs significantly from the second mean sequence, S502 is performed to determine whether the difference between the first mean sequence and the second mean sequence satisfies a parallel trend.
[0152] The parallel trend refers to the fact that the mean sequence differences between experimental groups within a period of time do not expand or shrink over time but tend to be parallel, that is, the difference in the target performance index within a period of time tends to be stable.
[0153] If the parallel trend is not satisfied, step S503 is executed to re-divide the experimental objects in the first object group and the second object group to obtain an updated first object group and an updated second object group.
[0154] Exemplarily, after obtaining the updated first object grouping and the updated second object grouping, it is necessary to judge again whether the updated grouping is reasonable until the difference between the multiple mean sequences corresponding to the updated first object grouping and the multiple mean sequences corresponding to the updated second object grouping is significant and satisfies the parallel trend.
[0155] If the parallel trend is met, step S504 is executed, the game product containing the game content to be tested is run in the experimental objects of the first object group to obtain the first game data corresponding to the first object group, and the game product not containing the game content to be tested is run in the experimental objects of the second object group to obtain the second game data corresponding to the second object group.
[0156] By conducting a significance test before the formal execution of the control experiment, the experimental groups with significant differences in target performance indicators and not satisfying the parallel trend can be regrouped after grouping, thereby improving the accuracy of the experimental results.
[0157] In the related art, A / A experiments are usually conducted to determine whether the experimental grouping is reasonable before conducting A / B experiments. However, due to the complexity of the game content corresponding to the game product, the timeliness of executing the control experiment is reduced through A / A experiments, which is not conducive to the promotion of A / B experiments in game products.
[0158] In order to solve the above problem, the present disclosure compares the game data before the control experiment is formally executed with the game data after the control experiment is formally executed without executing the A / A experiment.
[0159] The following will provide an exemplary description of the process of comparing the game data before the control experiment is formally executed with the game data after the control experiment is formally executed in conjunction with a specific embodiment.
[0160] In an optional embodiment of the present disclosure, based on a preset period, multiple first mean sequences for the target performance index of the first object grouping and multiple second mean sequences for the target performance index of the second object grouping are obtained; based on the preset period, multiple third mean sequences for the target performance index in the first game data and multiple fourth mean sequences for the target performance index in the second game database are obtained; if the difference between the first mean sequence and the second mean sequence is not significant, then it is determined whether the difference between the third mean sequence and the fourth mean sequence is significant; if the difference between the third mean sequence and the fourth mean sequence is not significant, then the result of the significance test is not significant. Conversely, in another optional embodiment of the present disclosure, if the difference between the third mean sequence and the fourth mean sequence is significant, then the result of the significance test is significant.
[0161] Exemplarily, if the difference between the third mean sequence and the fourth mean sequence is significant, the average mean difference between the third mean sequence and the fourth mean sequence is recorded as average(t2), then the result of the significance test of the control experiment is significant, and the significance effect is average(t2).
[0162] In some example embodiments of the present disclosure, if the difference between the first mean sequence and the second mean sequence is significant and the first mean sequence and the second mean sequence satisfy a parallel trend, an average mean difference is determined based on a first target difference sequence formed by the mean difference between the first mean sequence and the second mean sequence; a second target difference sequence is determined based on the mean difference between the third mean sequence and the fourth mean sequence; if the difference between the second target difference sequence and the average mean difference is significantly different from a preset value, the result of the significance test is significant.
[0163] In another optional embodiment of the present disclosure, if the difference between the second target difference sequence and the average mean difference is not significantly different from the preset value, the result of the significance test is not significant.
[0164] Exemplarily, the first target difference sequence formed by the mean difference between the first mean sequence and the second mean sequence is t1, and the average mean difference is determined to be average(t1); the second target difference sequence is determined to be t2 according to the mean difference between the third mean sequence and the fourth mean sequence, and the corresponding average mean difference is average(t2). Then:
[0165] When t2-average(t1) is significantly different from 0, the experimental effect is significant, and the significant effect is average(t2)-average(t1).
[0166] When t2-average(t1) is not significantly different from 0, the experimental effect is not significant.
[0167] In order to explain the above significance test analysis process more clearly, the following several cases are used for exemplary explanation.
[0168] 1) Before the experiment starts, the difference between the first mean sequence and the second mean sequence is not significant:
[0169] Before the first target group runs the initial game product and before the second target group runs the target game product, the difference between the first mean sequence and the second mean sequence between multiple first experimental groups (i.e., between the first target group and the second target group) is not significant.
[0170] The first mean sequence and the second mean sequence are mean sequences composed of the mean values of the target performance indicators collected during the experimental period and at the preset time intervals. For example, if the experimental period of the control experiment is 10 days, a mean value for the target performance indicator is determined every day, and the corresponding mean sequence is composed of the 10 mean values obtained during the experimental period.
[0171] a) After the experiment is started, the mean difference between the third mean sequence corresponding to the first target group and the fourth mean sequence corresponding to the second target group is not significant:
[0172] The significance test result of this control experiment was not significant.
[0173] b) After the experiment is started, the mean difference between the third mean sequence corresponding to the first target group and the fourth mean sequence corresponding to the second target group is significant:
[0174] Assume that after the start of the experiment, the first target difference sequence determined between the first target group and the second target group based on the third mean sequence and the fourth mean sequence at a preset time interval is t2, and the average difference of the differences in the corresponding first target difference sequence is average(t2) (for example, the daily mean difference), then the significance test result of the control experiment is significant, and the significant effect is average(t2).
[0175] 2) Before the experiment starts, there is a significant gap between the first mean sequence and the second mean sequence:
[0176] A) At this time, for the case where the first mean sequence and the second mean sequence satisfy a parallel trend, if it is assumed that the third target difference sequence determined based on the first mean sequence and the second mean sequence is t1, then the average difference of the differences in the corresponding third target difference sequence is average(t1).
[0177] a) After the experiment is started, the mean difference between the third mean sequence corresponding to the first target group and the fourth mean sequence corresponding to the second target group is not significant:
[0178] If t2-average(t1) is not significantly different from 0, the significance test result of the control experiment is not significant.
[0179] b) After the experiment is started, the mean difference between the third mean sequence corresponding to the first target group and the fourth mean sequence corresponding to the second target group is significant:
[0180] If t2-average(t1) is significantly different from 0, the significance test result of the control experiment is significant, and the significant effect is average(t2)-average(t1).
[0181] B) At this time, in view of the fact that the first mean sequence and the second mean sequence do not satisfy the parallel trend, the experimental object sample sets in the multiple first experimental groups are re-divided to obtain multiple second experimental groups until the gap between the mean sequences corresponding to the multiple second experimental groups is significant and satisfies the parallel trend.
[0182] exist Figure 3 Based on the four traffic layers divided, the following will refer to Figure 6The entire game data processing process of the game data processing method according to the exemplary embodiment of the present disclosure is described in detail.
[0183] In the experimental design phase, the control experiment to be conducted is determined, so as to execute step 601 and determine the target traffic layer based on the experimental content to be tested.
[0184] Exemplarily, the target traffic layer is determined from a plurality of traffic layers pre-divided based on the game content to be tested, and the plurality of traffic layers divide the experimental traffic into a plurality of orthogonal traffic layers based on the game content of the game product at different game stages.
[0185] If the service opening strategy traffic layer or client traffic layer assigned to the control experiment is determined based on the experimental content, when the user opens the service port of the server based on the terminal device, step S602 can be executed, and the control experiment can enter the service opening strategy traffic layer / client traffic layer to obtain the ID of each terminal device in the experimental object sample set.
[0186] In step S603, the terminal device IDs in the experimental object sample set are experimentally grouped to obtain a plurality of first experimental groups, each of which includes a control group and an experimental group.
[0187] Exemplarily, the terminal device ID can be calculated using a double hashing method and modulo N (assuming that the experiment has N experimental groups in total), and different terminal device IDs can be divided into corresponding experimental groups according to the modulo results (for example, the result 1 enters experimental group 1, the result 2 enters experimental group 2, until all terminal devices are grouped).
[0188] For example, if the grouping of the experimental group and the control group is unreasonable, it is only necessary to regroup until the grouping of the experimental group and the control group is reasonable. Otherwise, step S604 is executed to run the initial game product in the control group and the target game product in the experimental group.
[0189] Among them, the initial game product does not have factors to be tested, while the target game product contains factors to be tested.
[0190] In step S605, the first experimental data of the control group and the second experimental data of the experimental group are collected and the experimental data are analyzed with the device as the experimental analysis unit.
[0191] Exemplarily, when performing experimental data analysis, a significance test analysis may be performed, so as to make a decision based on the result of the significance test, that is, executing step S606 to decide whether the target game version replaces the initial game version based on the result of the experimental data analysis.
[0192] In step S607, the server makes a decision.
[0193] For example, if the significance test result of the control experiment is significant, the initial game version can be replaced with the target game version, and the group numbers corresponding to all terminal devices in the server can be changed to the experimental group corresponding to the target game version. On the contrary, if the significance test result of the control experiment is not significant, there is no need to replace the initial game version with the target game version, and the group numbers corresponding to all terminal devices in the server can be changed to the experimental group corresponding to the initial game product.
[0194] If the control experiment enters the server traffic layer / cross-server strategy traffic layer, after the player enters the client and completes the determination of the server and the game character, the server can execute step S608, the control experiment enters the server traffic layer / cross-server strategy traffic layer, and obtains the game character ID or server ID in the experimental object sample set.
[0195] Then, step S609 is executed to perform experimental grouping of the experimental object samples according to the game character ID or the server ID to obtain a plurality of first experimental groups, each of which includes a control group and an experimental group.
[0196] It should be noted that after the experimental grouping, it is necessary to analyze whether the experimental grouping is reasonable based on the significance test. If the grouping is unreasonable, it is necessary to regroup until the experimental grouping is reasonable based on the significance test analysis and then execute step S610. On the contrary, if the experimental grouping is reasonable, directly execute step S610, run the initial game product in the control group, and run the target game product in the experimental group.
[0197] In step S611, the first experimental data of the control group and the second experimental data of the experimental group are collected and the experimental data are analyzed with the game characters as the experimental analysis units.
[0198] In step S606, it is decided whether to replace the initial game product with the target game product, and the decision result is executed based on step S607. That is, if the significance test result of the control experiment is significant, the initial game version can be replaced with the target game version, and the group numbers corresponding to all terminal devices in the server are changed to the experimental group corresponding to the target game version. On the contrary, if the significance test result of the control experiment is not significant, there is no need to replace the initial game version with the target game version, and the group numbers corresponding to all terminal devices in the server are changed to the experimental group corresponding to the initial game product.
[0199] Finally, the entire experimental process can be organized into a report and archived for subsequent reference by experimental designers.
[0200] In order to implement the above-mentioned game data processing method, a game data processing device is provided in one embodiment of the present disclosure. Figure 7The schematic diagram shows a schematic architecture of a game data processing device.
[0201] The game data processing device 700 includes a traffic layer determination module 701 , a sample division module 702 , a game operation module 703 and an experiment analysis module 704 .
[0202] The traffic layer determination module 701 is used to determine the target traffic layer that matches the game content to be tested from multiple pre-configured traffic layers according to the game content to be tested corresponding to the game product; the sample division module 702 is used to divide the experimental object sample set into a first object group and a second object group based on the diversion configuration parameters corresponding to the target traffic layer; the game running module 703 is used to run the game product containing the game content to be tested in the experimental objects of the first object group to obtain the first game data corresponding to the first object group, and to run the game product that does not contain the game content to be tested in the experimental objects of the second object group to obtain the second game data corresponding to the second object group; the experimental analysis module 704 is used to compare the first game data and the second game data based on the analysis configuration parameters corresponding to the target traffic layer to obtain the game value data corresponding to the game content to be tested in the game product.
[0203] In an optional embodiment of the present disclosure, the traffic layer determination module 701 is used to determine a target traffic layer that matches the game content to be checked from a plurality of pre-configured traffic layers based on a game stage corresponding to the game content to be checked.
[0204] In an optional embodiment of the present disclosure, the sample partitioning module 702 is used to determine a first sample size threshold corresponding to the experimental object sample set based on the total variance of the experimental object on the target performance indicator, the standard deviation of the significance threshold and the mean of the experimental object on the target performance indicator, the statistical power and the standard deviation of the mean of the experimental object on the target performance indicator, and the minimum gap between the first object group and the second object group for the target performance indicator if the analysis configuration parameters are the same as the diversion configuration parameters; and divide the experimental object sample set with a sample size not less than the first sample size threshold into a first object group and a second object group based on the diversion configuration parameters corresponding to the target traffic layer.
[0205] In an optional embodiment of the present disclosure, the sample partitioning module 702 is used to determine the target mean based on the mean and variance of the number of experimental sub-objects in the experimental object, and the mean and variance of the target performance indicator and the variance of each experimental object in the experimental object sample set on the target performance indicator if the analysis configuration parameters are different from the diversion configuration parameters; determine the first sample size threshold corresponding to the experimental object sample set based on the total variance of the experimental object on the target performance indicator, the significance threshold and the standard deviation of the mean of the experimental object on the target performance indicator, the statistical power and the standard deviation of the mean of the experimental object on the target performance indicator, and the minimum gap between the first object group and the second object group on the target performance indicator; determine the second sample size threshold corresponding to the experimental object sample set based on the target mean and the first sample size threshold; and divide the experimental object sample set with a sample number not less than the second sample size threshold into a first object group and a second object group based on the diversion configuration parameters corresponding to the target traffic layer.
[0206] In an optional embodiment of the present disclosure, the experimental analysis module 704 is used to perform a significance test on the first game data corresponding to the first object group and the second game data corresponding to the second object group based on the analysis configuration parameters corresponding to the target traffic layer; and obtain the game value data corresponding to the game content to be tested in the game product according to the test result of the significance test.
[0207] In an optional embodiment of the present disclosure, the game data processing device 700 may further include a sequence acquisition module 705 and a difference determination module 706 .
[0208] Among them, the sequence acquisition module 705 is used to obtain multiple first mean sequences of the first object grouping for the target performance indicator and multiple second mean sequences of the second object grouping for the target performance indicator based on a preset period; the difference judgment module 706 is used to judge whether the difference between the multiple first mean sequences of the first object grouping and the multiple second mean sequences of the second object grouping satisfies the parallel trend if the difference between the multiple first mean sequences of the first object grouping and the multiple second mean sequences of the second object grouping is significant; the sample division module 702 is used to re-divide the experimental objects in the first object grouping and the second object grouping if the parallel trend is not satisfied, to obtain the updated first object grouping and the updated second object grouping, until the difference between the multiple first mean sequences corresponding to the updated first object grouping and the multiple second mean sequences corresponding to the updated second object grouping is significant and satisfies the parallel trend.
[0209] In an optional embodiment of the present disclosure, the experimental analysis module 704 can also be used to obtain multiple first mean sequences for the target performance indicator for the first object grouping, and multiple second mean sequences for the target performance indicator for the second object grouping based on a preset period; obtain multiple third mean sequences for the target performance indicator in the first game data, and multiple fourth mean sequences for the target performance indicator in the second game database based on a preset period; if the difference between the first mean sequence and the second mean sequence is not significant, determine whether the difference between the third mean sequence and the fourth mean sequence is significant; if the difference between the third mean sequence and the fourth mean sequence is not significant, the result of the significance test is insignificant; or, if the difference between the third mean sequence and the fourth mean sequence is significant, the result of the significance test is significant.
[0210] In an optional embodiment of the present disclosure, the experimental analysis module 704 is used to determine the average mean difference based on a first target difference sequence formed by the mean difference between the first mean sequence and the second mean sequence if the difference between the first mean sequence and the second mean sequence is significant and the first mean sequence and the second mean sequence satisfy a parallel trend; determine the second target difference sequence according to the mean difference between the third mean sequence and the fourth mean sequence; if the difference between the second target difference sequence and the average mean difference is significantly different from a preset value, the result of the significance test is significant; or, if the difference between the second target difference sequence and the average mean difference is not significantly different from the preset value, the result of the significance test is not significant.
[0211] In an optional embodiment of the present disclosure, the target performance indicator includes at least one of a user retention rate, a payment rate, a user activity rate, and a user online time.
[0212] The game data processing device 700 provided in the embodiment of the present disclosure can execute the technical solution of the game data processing method in any of the above-mentioned embodiments. Its implementation principle and beneficial effects are similar to the implementation principle and beneficial effects of the game data processing method. Please refer to the implementation principle and beneficial effects of the game data processing method, and no further details will be given here.
[0213] In an exemplary embodiment of the present disclosure, a computer-readable storage medium is also provided, on which a program product capable of implementing the above method of the present specification is stored. In some possible implementations, various aspects of the present invention may also be implemented in the form of a program product, which includes a program code, and when the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps according to various exemplary embodiments of the present invention described in the above "Exemplary Method" section of the present specification.
[0214] The program product for implementing the above method according to an embodiment of the present invention may adopt a portable compact disc read-only memory (CD-ROM) and include program code, and may be run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto, and in this document, a readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device.
[0215] The program product may adopt any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.
[0216] Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, in which readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Readable signal media may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.
[0217] The program code contained on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, radio frequency (RF), etc., or any suitable combination of the foregoing.
[0218] Program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, C++, etc., and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., via the Internet using an Internet Service Provider).
[0219] In an exemplary embodiment of the present disclosure, an electronic device capable of implementing the above method is also provided.
[0220] It will be appreciated by those skilled in the art that various aspects of the present invention may be implemented as a system, method or program product. Therefore, various aspects of the present invention may be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to herein as a "circuit", "module" or "system".
[0221] Refer to the following Figure 8 An electronic device 800 according to this embodiment of the present invention will be described. Figure 8 The electronic device 800 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.
[0222] like Figure 8 As shown, the electronic device 800 is in the form of a general computing device. The components of the electronic device 800 may include, but are not limited to: the at least one processing unit 810, the at least one storage unit 820, a bus 830 connecting different system components (including the storage unit 820 and the processing unit 810), and a display unit 840.
[0223] The storage unit stores program codes, which can be executed by the processing unit 810, so that the processing unit 810 performs the steps of various exemplary embodiments of the present invention described in the above “Exemplary Method” section of this specification. For example, the processing unit 810 can perform the following steps: Figure 2 Steps S201 to S204 shown in FIG.
[0224] The storage unit 820 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 8201 and / or a cache memory unit 8202 , and may further include a read-only memory unit (ROM) 8203 .
[0225] The storage unit 820 may also include a program / utility 8204 having a set (at least one) of program modules 8205, such program modules 8205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.
[0226] Bus 830 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.
[0227] The electronic device 800 may also communicate with one or more external devices 1000 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 800, and / or communicate with any device that enables the electronic device 800 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 850. Furthermore, the electronic device 800 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 860. As shown, the network adapter 860 communicates with other modules of the electronic device 800 via a bus 830. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 800, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, disk arrays (Redundant Arrays of Independent Disks, RAID) systems, tape drives, and data backup storage systems.
[0228] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the implementation of the present disclosure.
[0229] In addition, the above-mentioned figures are only schematic illustrations of the processes included in the method according to an exemplary embodiment of the present invention, and are not intended to be limiting. It is easy to understand that the processes shown in the above-mentioned figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.
[0230] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.
[0231] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The specification and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.
[0232] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.
Claims
1. A method for processing game data, characterized in that: include: According to the game content to be checked corresponding to the game product, determining a target traffic layer matching the game content to be checked from a plurality of pre-configured traffic layers; Based on the diversion configuration parameters corresponding to the target traffic layer, the experimental object sample set is divided into a first object group and a second object group; Running the game product containing the to-be-tested game content in the experimental objects of the first object group to obtain first game data corresponding to the first object group, and running the game product not containing the to-be-tested game content in the experimental objects of the second object group to obtain second game data corresponding to the second object group; Based on the analysis configuration parameters corresponding to the target traffic layer, the first game data and the second game data are compared to obtain game value data corresponding to the game content to be tested in the game product.
2. The game data processing method according to claim 1, characterized in that: The multiple traffic layers are divided based on the game stages included in the game product; The step of determining a target traffic layer matching the game content to be checked from a plurality of pre-configured traffic layers according to the game content to be checked corresponding to the game product includes: Based on the game stage corresponding to the game content to be checked, a target traffic layer matching the game content to be checked is determined from a plurality of pre-configured traffic layers.
3. The game data processing method according to claim 1, characterized in that: The dividing the experimental object sample set into a first object group and a second object group based on the diversion configuration parameters corresponding to the target traffic layer includes: If the analysis configuration parameter is the same as the diversion configuration parameter, determining a first sample size threshold corresponding to the experimental object sample set based on the total variance of the experimental object on the target performance indicator, the standard deviation of the significance threshold and the mean of the experimental object on the target performance indicator, the statistical power and the standard deviation of the mean of the experimental object on the target performance indicator, and the minimum gap between the first object grouping and the second object grouping for the target performance indicator; Based on the diversion configuration parameters corresponding to the target traffic layer, the experimental object sample set whose sample quantity is not less than the first sample quantity threshold is divided into the first object group and the second object group.
4. The game data processing method according to claim 1, characterized in that: The dividing the experimental object sample set into a first object group and a second object group based on the diversion configuration parameters corresponding to the target traffic layer includes: If the analysis configuration parameters are different from the diversion configuration parameters, determining a target mean based on the mean and variance of the number of experimental sub-objects in the experimental object and the mean and variance of the target performance indicator and the variance of each experimental object in the experimental object sample set on the target performance indicator; Determine a first sample size threshold corresponding to the experimental object sample set based on the total variance of the experimental object on the target performance indicator, the standard deviation of the significance threshold and the mean of the experimental object on the target performance indicator, the statistical power and the standard deviation of the mean of the experimental object on the target performance indicator, and the minimum gap between the first object group and the second object group for the target performance indicator; Determine a second sample size threshold corresponding to the experimental object sample set based on the target mean and the first sample size threshold; Based on the diversion configuration parameters corresponding to the target traffic layer, the experimental object sample set whose sample quantity is not less than the second sample quantity threshold is divided into the first object group and the second object group.
5. The game data processing method according to claim 1, characterized in that: The comparing the first game data and the second game data based on the analysis configuration parameters corresponding to the target traffic layer to obtain game value data corresponding to the game content to be tested in the game product includes: Based on the analysis configuration parameters corresponding to the target traffic layer, a significance test is performed on the first game data corresponding to the first object grouping and the second game data corresponding to the second object grouping; based on the test result of the significance test, the game value data corresponding to the game content to be tested in the game product is obtained.
6. The game data processing method according to claim 5, characterized in that: Before running the game product containing the to-be-tested game content in the experimental objects of the first object group to obtain the first game data corresponding to the first object group, and running the game product not containing the to-be-tested game content in the experimental objects of the second object group to obtain the second game data corresponding to the second object group, the method further includes: Based on a preset period, obtaining a plurality of first mean value sequences of the first object grouping for a target performance indicator and a plurality of second mean value sequences of the second object grouping for a target performance indicator; If the difference between the multiple first mean sequences of the first object group and the multiple second mean sequences of the second object group is significant, then determining whether the difference between the first mean sequence and the second mean sequence satisfies a parallel trend; If the parallel trend is not satisfied, the experimental objects in the first object grouping and the second object grouping are re-divided to obtain an updated first object grouping and an updated second object grouping, until the difference between the multiple first mean sequences corresponding to the updated first object grouping and the multiple second mean sequences corresponding to the updated second object grouping is significant and satisfies the parallel trend.
7. The game data processing method according to claim 5, characterized in that: The performing a significance test on the first game data corresponding to the first object group and the second game data corresponding to the second object group comprises: Based on a preset period, obtaining a plurality of first mean value sequences of the first object grouping for a target performance indicator and a plurality of second mean value sequences of the second object grouping for a target performance indicator; Based on the preset period, obtaining a plurality of third mean value sequences for the target performance indicator in the first game data and a plurality of fourth mean value sequences for the target performance indicator in the second game database; If the difference between the first mean sequence and the second mean sequence is not significant, then determining whether the difference between the third mean sequence and the fourth mean sequence is significant; If the difference between the third mean sequence and the fourth mean sequence is not significant, the result of the significance test is not significant; Alternatively, if the difference between the third mean sequence and the fourth mean sequence is significant, the result of the significance test is significant.
8. The game data processing method according to claim 7, characterized in that: The method further comprises: If the difference between the first mean sequence and the second mean sequence is significant and the first mean sequence and the second mean sequence satisfy a parallel trend, determining an average mean difference based on a first target difference sequence formed by the mean difference between the first mean sequence and the second mean sequence; Determine a second target difference sequence according to a mean difference between the third mean sequence and the fourth mean sequence; If the difference between the second target difference sequence and the average mean difference is significantly different from a preset value, the result of the significance test is significant; or, If the difference between the second target difference sequence and the average mean difference is not significantly different from the preset value, the result of the significance test is not significant.
9. The game data processing method according to claim 5, characterized in that: The target performance indicator includes at least one of user retention rate, payment rate, user activity rate, and user online time.
10. A game data processing device, characterized in that: The device comprises: A traffic layer determination module, used to determine a target traffic layer matching the game content to be checked from a plurality of pre-configured traffic layers according to the game content to be checked corresponding to the game product; A sample division module, used for dividing the experimental object sample set into a first object group and a second object group based on the diversion configuration parameters corresponding to the target traffic layer; a game running module, configured to run the game product containing the to-be-tested game content in the experimental objects of the first object group to obtain first game data corresponding to the first object group, and to run the game product not containing the to-be-tested game content in the experimental objects of the second object group to obtain second game data corresponding to the second object group; The experimental analysis module is used to compare the first game data and the second game data based on the analysis configuration parameters corresponding to the target traffic layer to obtain the game value data corresponding to the game content to be tested in the game product.
11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the game data processing method according to any one of claims 1 to 9 is implemented.
12. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; Wherein, the processor is configured to execute the game data processing method described in any one of claims 1 to 9 by executing the executable instructions.
Citation Information
Cited By
Penetration testing method for Android game
CN120381675A
A penetration testing method for Android games
CN120381675B
