A method, apparatus, and equipment for determining the available flow rate in an AB test experiment.

Abstract

This application provides a method, apparatus, and device for determining available traffic for an A / B test experiment. The method includes: in response to a task to determine available traffic for a first test experiment to be started among a set of preset test experiments, determining the total user traffic to be allocated to the first test experiment; filtering candidate user traffic that meets the filtering rules pre-configured for the first test experiment from the total user traffic; determining whether there is a mutual exclusion relationship between the first test experiment and any test experiment already started among the preset test experiments; and when there is a mutual exclusion relationship between the first test experiment and a second test experiment already started, filtering user traffic that has not participated in the second test experiment from the candidate user traffic as the available traffic for the first test experiment. The method, apparatus, and device provided in this application can accurately calculate available traffic.

Description

Technical Field

[0001] This application relates to the field of testing technology, and in particular to a method, apparatus, and device for determining the available flow rate in an AB test experiment. Background Technology

[0002] In recent years, with the development of internet technology, product functions have become increasingly rich. Currently, in order to ensure that product functions meet user needs, A / B testing is often conducted on relevant functions before their official launch.

[0003] During A / B testing, the bandwidth required for each test experiment varies due to different needs. Therefore, accurately determining the available bandwidth for a specific test experiment provides the relevant personnel with precise information, facilitating their planning of bandwidth usage and enabling better control over the test.

[0004] Currently, the available traffic in A / B testing experiments is mostly estimated based on filtering rules, which results in inaccurate estimates of available traffic. Summary of the Invention

[0005] In view of this, this application provides a method, apparatus and equipment for determining the available flow rate in AB test experiments, in order to solve the problem of inaccurate estimation of available flow rate in the prior art.

[0006] Specifically, this application is implemented through the following technical solution:

[0007] A first aspect of this application provides a method for determining the available flow rate in an AB test experiment, the method comprising:

[0008] In response to the available traffic determination task for the first test experiment to be started among a set of pre-defined test experiments, determine the total user traffic to be allocated to the first test experiment;

[0009] Based on the filtering rules pre-configured for the first test experiment, candidate user traffic that meets the filtering rules is filtered out from the total user traffic;

[0010] Determine whether there is a mutual exclusion relationship between the first test experiment and any of the multiple test experiments that have been started;

[0011] When there is a mutual exclusion relationship between the first test experiment and the initiated second test experiment, user traffic that has not participated in the second test experiment is selected from the candidate user traffic as the available traffic for the first test experiment.

[0012] A second aspect of this application provides an apparatus for determining the available flow rate in an AB test experiment. The apparatus includes a determining module, a filtering module, a judging module, and a processing module, wherein...

[0013] The determining module is used to respond to the available traffic determination task for the first test experiment to be started among a preset plurality of test experiments, and to determine the total user traffic to be allocated to the first test experiment.

[0014] The filtering module is used to filter out candidate user traffic that meets the filtering rules from the total user traffic based on the filtering rules pre-configured for the first test experiment;

[0015] The judgment module is used to determine whether there is a mutual exclusion relationship between the first test experiment and any of the multiple test experiments that have been started.

[0016] The processing module is used to filter user traffic that has not participated in the second test experiment from the candidate user traffic as the available traffic for the first test experiment when the judgment module determines that the first test experiment and the started second test experiment are mutually exclusive.

[0017] A third aspect of this application provides an apparatus for determining the available flow rate for an AB test experiment, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of any of the methods provided in the first aspect of this application.

[0018] A fourth aspect of this application provides a storage medium having a program stored thereon, which, when executed by a processor, implements the steps of any of the methods provided in the first aspect of this application.

[0019] The method, apparatus, and device provided in this application for determining available traffic for an A / B test experiment are responsive to a task of determining available traffic for a first test experiment to be started among a plurality of preset test experiments. By determining the total user traffic to be allocated to the first test experiment and filtering candidate user traffic that meets the filtering rules pre-configured for the first test experiment, the method further determines whether there is a mutual exclusion relationship between the first test experiment and any already started test experiment among the plurality of test experiments. If there is a mutual exclusion relationship between the first test experiment and an already started second test experiment, user traffic that has not participated in the second test experiment is filtered from the candidate user traffic as available traffic for the first test experiment. Thus, when calculating available traffic, candidate user traffic is first determined based on the filtering rules, and then user traffic from already started mutually exclusive experiments is filtered out from the candidate user traffic based on the mutual exclusion relationships between the test experiments, resulting in the final available traffic. Because the mutual exclusion relationships between the test experiments are considered, the available traffic can be accurately determined. Attached Figure Description

[0020] Figure 1 A flowchart of an embodiment of the method for determining the available flow rate for AB test experiments provided in this application;

[0021] Figure 2 A flowchart of Embodiment 2 of the method for determining the available flow rate for AB test experiments provided in this application;

[0022] Figure 3 A hardware structure diagram of the equipment for determining the available flow device in the AB test experiment provided in this application;

[0023] Figure 4 This is a schematic diagram of the structure of an embodiment of the device for determining the available flow rate for AB test experiments provided in this application. Detailed Implementation

[0024] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0025] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0026] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0027] This application provides a method, apparatus, and device for determining the available flow rate for AB testing experiments, so as to improve the accuracy of the determined available flow rate for AB testing experiments.

[0028] The method, apparatus, and device provided in this application for determining available traffic for an A / B test experiment are responsive to a task of determining available traffic for a first test experiment to be started among a plurality of preset test experiments. By determining the total user traffic to be allocated to the first test experiment and filtering candidate user traffic that meets the filtering rules pre-configured for the first test experiment, the method further determines whether there is a mutual exclusion relationship between the first test experiment and any of the started test experiments among the plurality of test experiments. If there is a mutual exclusion relationship between the first test experiment and a started second test experiment, user traffic that has not participated in the second test experiment is filtered from the candidate user traffic as available traffic for the first test experiment. Thus, in calculating available traffic, candidate user traffic is first determined based on the filtering rules, and then user traffic from started mutually exclusive experiments is filtered out from the candidate user traffic based on the mutual exclusion relationships between test experiments, resulting in the final available traffic. Because the mutual exclusion relationships between test experiments are considered, the available traffic can be accurately determined.

[0029] The following specific embodiments are given to illustrate the technical solution of this application in detail.

[0030] Figure 1 This is a flowchart of an embodiment of the method for determining the available flow rate for AB test experiments provided in this application. Please refer to... Figure 1 The method provided in this embodiment may include:

[0031] S101, in response to the available traffic determination task for the first test experiment to be started among a plurality of preset test experiments, determine the total user traffic to be allocated to the first test experiment.

[0032] It should be noted that the method provided in this embodiment can be applied to devices that determine the available traffic for A / B testing experiments. These devices can be A / B testing equipment or database servers.

[0033] Available traffic determination tasks are created by testers using a task creation platform on the A / B testing equipment (e.g., a client-side platform). For example, the task creation platform provides a create button; when a tester clicks this button, a creation interface pops up, allowing the tester to create an available traffic determination task for the first test experiment.

[0034] Furthermore, when the first test experiment is about to start, an available traffic determination task can be created for the first test experiment.

[0035] It should be noted that when the executing entity is the database server, after the AB test device creates the available traffic determination task, it can send the available traffic test task to the database server, instructing the database server to execute the available traffic determination task.

[0036] In practice, the total user traffic to be allocated to the first test experiment can be determined according to a preset determination strategy. This preset determination strategy is set according to actual needs, and in this embodiment, the specific content of the preset determination strategy is not limited. For example, in one embodiment, the preset determination strategy can be: determining the set of users who have performed a specified operation on the product under test corresponding to the first test experiment within a specified time period as the total user traffic.

[0037] It should be noted that the specified time period is set according to actual needs, and is not limited in this embodiment. For example, the specified time period can be the day before the current time, or it can be the two days before the current time. Furthermore, the specified operation is set according to actual needs, and is not limited in this embodiment. For example, the specified operation can include at least one of the following operations: login operation, browsing operation, and commenting operation.

[0038] Based on the above description, for example, in one embodiment, the preset determination strategy is to take the set of users who performed login operations on the product under test corresponding to the first test experiment the day before the current time as the total user traffic.

[0039] S102. Based on the filtering rules pre-configured for the first test experiment, candidate user traffic that meets the filtering rules is filtered out from the total user traffic.

[0040] Specifically, the screening rules pre-configured for the first test experiment are set by the testers according to the actual needs of the test experiment. In this embodiment, the specific content of the screening rules is not limited.

[0041] For example, in one possible implementation, the filtering rules may include at least one of the following rules: the geographic location of the user traffic is a specified location, the user traffic is a user of a specified gender, and the user traffic identification information is specified identification information.

[0042] It should be noted that the specified location, specified gender, and specified identification information are set according to actual needs. In this embodiment, the specific content of the specified location, specified gender, and specified identification information is not limited.

[0043] For example, in one embodiment, the filtering rule could be that the geographical location of the user traffic is Beijing. In this case, in this step, user traffic with a geographical location belonging to Beijing is filtered out from the total user traffic as candidate user traffic.

[0044] For example, in another embodiment, the filtering rule could be that the user traffic consists of male users. In this case, this step filters out male user traffic from the total user traffic as candidate user traffic.

[0045] In practice, the total user traffic records the attribute information of each user's traffic. This attribute information includes at least one of the following: identifier, gender, and geographic location. In this step, candidate user traffic that meets the filtering rules can be filtered from the total user traffic based on the user traffic attribute information.

[0046] Based on the above examples, for instance, in one embodiment, the total user traffic includes 10 users. For ease of distinction, these 10 users are respectively denoted as User 1, User 2, ..., User 10. Furthermore, Table 1 is an attribute information table of each user traffic in the total user traffic shown in an exemplary embodiment of this application.

[0047] Table 1 shows the attribute information of each user's traffic in the total user traffic.

[0048] user Identification information gender Geographical location User 1 1 male Beijing User 2 2 female Shanghai User 3 3 female Hangzhou User 4 4 male Chongqing User 5 5 female Suzhou User 6 6 male Tianjin User 7 7 female Hangzhou User 8 8 male Beijing User 9 9 female Nanjing User 10 10 female Heilongjiang

[0049] Using the example above, for instance, if the filtering rule is that the geographical location of the user traffic is Beijing, then according to Table 1, the filtered candidate user traffic includes user 1 and user 8.

[0050] For example, if the filtering rule is that the user traffic is male, then according to Table 1, the filtered candidate user traffic includes user 1, user 4, user 6 and user 8.

[0051] It should be noted that when the executing entity is a database server, in one embodiment, the database server can determine the task based on available traffic and obtain experimental configuration information pre-configured for the first test experiment from the test device. This experimental configuration information includes filtering rules, and then filters candidate user traffic based on the filtering rules. Alternatively, after generating the available traffic determination task for the first test experiment, the test device sends the available traffic test task and the experimental configuration information of the first test experiment to the database server. In this way, the database server can execute the available traffic test task based on the experimental configuration information.

[0052] It should be noted that after the test equipment generates the available traffic determination task for the first test experiment, when it sends the available traffic test task and the experiment configuration information of the first test experiment to the database server, it can generate the corresponding SQL statement based on the experiment configuration information, and then send the available traffic test task and the SQL statement to the database server together.

[0053] S103. Determine whether there is a mutual exclusion relationship between the first test experiment and any of the multiple test experiments that have been started.

[0054] Specifically, "started test experiment" refers to a test experiment that has already begun testing. For example, in a practical implementation, the test equipment configures status information for the test experiments. The initial value of the status information is "not started." When a test experiment starts, its status information is updated from "not started" to "started." In this step, the currently started test experiment can be found from multiple test experiments by using the current status information of each test experiment, and then it can be determined whether there is a mutual exclusion relationship between the first test experiment and the currently started test experiment.

[0055] For example, in one embodiment, the preset multiple test experiments include 5 test experiments, and the current status information of these 5 test experiments are as follows: Experiment 1: Started, Experiment 2: Not Started, Experiment 3: Not Started, Experiment 4: Not Started, Experiment 5: Not Started. At this time, for example, the first test experiment to be started is Experiment 3. In this step, firstly, based on the current status information of these 5 test experiments, the currently started test experiment is found to be Test Experiment 1. Further, it is determined whether there is a mutual exclusion relationship between the first test experiment (for example, in one embodiment, the first test experiment is Test Experiment 3) and Test Experiment 1.

[0056] For example, in another embodiment, the preset multiple test experiments include 10 test experiments, all of which are currently inactive. In this case, it indicates that the first test experiment to be started is the first test experiment to be started. Furthermore, since it is the first test experiment to be started, it is not mutually exclusive with any other test experiment. In this case, the candidate user traffic is directly determined as available traffic.

[0057] It should be noted that mutual exclusion between two test experiments means that there is overlap in user traffic between them. Furthermore, the mutual exclusion relationship between two experiments can be determined based on their configuration information. The specific implementation principle of this determination will be described in detail in the following embodiments, and will not be repeated here.

[0058] S104. When there is a mutual exclusion relationship between the first test experiment and the started second test experiment, select the user traffic that has not participated in the second test experiment from the candidate user traffic as the available traffic for the first test experiment.

[0059] For example, in one embodiment, the total user traffic is 500. After filtering by the filtering rules, the candidate user traffic is determined to be 400. Furthermore, among these 400 user traffic, 100 user traffic have participated in the second test experiment. At this time, the available traffic is determined to be the remaining 300 user traffic.

[0060] It should be noted that for each initiated test experiment, a corresponding test record will be generated, which will record the user traffic that has participated in the test experiment. In this step, user traffic that has not participated in the second test experiment can be filtered from the candidate user traffic based on the test record of the second test experiment.

[0061] Optionally, if no initiated test experiment exists in the multiple tests, or if there is no mutual exclusion relationship between the first test experiment and any initiated test experiment, the method further includes:

[0062] The candidate user traffic is used as the available traffic for the first test experiment.

[0063] The method for determining available traffic for A / B testing experiments provided in this embodiment responds to a task of determining available traffic for a first test experiment to be started among a set of preset test experiments. It determines the total user traffic to be allocated to the first test experiment and, based on pre-configured filtering rules for the first test experiment, filters out candidate user traffic that meets the filtering rules from the total user traffic. Then, it determines whether there is a mutual exclusion relationship between the first test experiment and any of the already started test experiments among the preset test experiments. If there is a mutual exclusion relationship between the first test experiment and an already started second test experiment, user traffic that has not participated in the second test experiment is filtered out from the candidate user traffic as available traffic for the first test experiment. Thus, in calculating available traffic, candidate user traffic is first determined based on filtering rules, and then user traffic from already started mutually exclusive experiments is filtered out from the candidate user traffic based on the mutual exclusion relationships between test experiments, resulting in the final available traffic. Because the mutual exclusion relationships between test experiments are considered, the available traffic can be accurately determined.

[0064] Figure 2 The flowchart for Embodiment 2 of the method for determining the available flow rate for AB test experiments provided in this application is shown below. Please refer to... Figure 2 The method provided in this embodiment may include:

[0065] S201. In response to the task of determining the available traffic for the first test experiment to be started among a plurality of preset test experiments, the set of users who have performed a specified operation on the product under test corresponding to the first test experiment within a specified time period is determined as the total user traffic.

[0066] S202. Based on the filtering rules pre-configured for the first test experiment, candidate user traffic that meets the filtering rules is filtered out from the total user traffic.

[0067] The specific implementation principles and processes of steps S201 and S202 can be found in the description of the above embodiments, and will not be repeated here.

[0068] S203. Based on the mutual exclusion information of the first test experiment, determine whether at least one mutually exclusive experiment recorded in the mutual exclusion information is the same as any test experiment that has been started; wherein, the mutual exclusion information records the identification information of each mutually exclusive experiment that has a mutual exclusion relationship with the first test experiment.

[0069] It should be noted that, in one possible implementation, the experimental configuration information configured by the tester for the test experiment includes mutual exclusion information. The mutual exclusion information records the identification information of each mutually exclusive experiment that has a mutual exclusion relationship with the first test experiment. For example, referring to the example above, multiple test experiments include 5 test experiments. For the first test experiment (e.g., the first test experiment is test experiment 3), the mutual exclusion information of the first test experiment can be that this test experiment is mutually exclusive with test experiment 1 and test experiment 5.

[0070] In this step, it is determined whether test experiment 1 and test experiment 5 are the same as the already started test experiments. For example, in one embodiment, the already started test experiment is test experiment 2. In this case, this step determines whether at least one mutually exclusive experiment recorded in the mutual exclusion information is different from the already started test experiment 2. As another example, if the already started test experiment is test experiment 1, this step determines whether at least one mutually exclusive experiment recorded in the mutual exclusion information is the same as the already started test experiment 1.

[0071] S204. When the mutually exclusive experiment recorded in the mutual exclusion information is the same as the second test experiment that has been started, it is determined that the first test experiment and the second test experiment that has been started are mutually exclusive, and user traffic that has not participated in the second test experiment is selected from the candidate user traffic as the available traffic for the first test experiment.

[0072] Combining the examples above, for instance, in the second example, if it is determined that at least one mutually exclusive experiment (mutually exclusive experiments include test experiment 1 and test experiment 5) recorded in the mutual exclusion information is the same as the already started test experiment 1, then user traffic that has not participated in test experiment 1 is selected from the candidate user traffic as the available traffic for the first test experiment (test experiment 3).

[0073] S205. When the mutual exclusion experiment recorded in the mutual exclusion information is different from the second test experiment that has been started, it is determined that there is no mutual exclusion relationship between the first test experiment and any of the test experiments that have been started, and the candidate user traffic is taken as the available traffic of the first test experiment.

[0074] Combining the examples above, for instance, in the first example, if it is determined that at least one mutually exclusive experiment (mutually exclusive experiments include test experiment 1 and test experiment 5) recorded in the mutual exclusion information is different from the started test experiment (test experiment 2), then the candidate user traffic is determined as the available traffic of the first test experiment (test experiment 3).

[0075] S206. Show the available traffic to the testers.

[0076] In practice, the display can be done locally or on other display devices; this embodiment does not limit this.

[0077] Specifically, when the executing entity is a database server, the database server can send available traffic to the test device so that the test device can display the available traffic to the testers.

[0078] The method for determining available traffic in A / B testing provided in this embodiment first determines candidate user traffic based on filtering rules, and then filters out user traffic of mutually exclusive experiments that have been started from the candidate user traffic by considering the mutual exclusion relationship between the test experiments, so as to obtain the final available traffic. Because the mutual exclusion relationship between the test experiments is taken into account, the available traffic can be accurately determined.

[0079] Corresponding to the aforementioned embodiment of a method for determining the available flow rate for an AB test experiment, this application also provides an embodiment of an apparatus for determining the available flow rate for an AB test experiment.

[0080] An embodiment of the device for determining the available flow rate in an AB test experiment provided in this application can be applied to devices for determining the available flow rate in an AB test experiment. The device embodiment can be implemented through software, hardware, or a combination of both. Taking software implementation as an example, as a logical device, it is formed by the processor of the device where it is located reading the corresponding computer program instructions from non-volatile memory into memory and executing them. From a hardware perspective, such as... Figure 3 The diagram shown is a hardware structure diagram of the device for determining the available flow rate in the AB test experiment, as described in this application. (Except for...) Figure 3 In addition to the processor, memory, network interface, and non-volatile memory shown, the device in the embodiment that determines the available traffic for the AB test experiment may also include other hardware depending on the actual function of the device that determines the available traffic for the AB test experiment, which will not be described in detail here.

[0081] Figure 4 For a schematic diagram of the device for determining the usable flow rate in the AB test experiment provided in this application, please refer to Embodiment 1. Figure 4The apparatus provided in this embodiment includes a determining module 410, a filtering module 420, a judging module 430, and a processing module 440, wherein...

[0082] The determining module 410 is used to determine the total user traffic to be allocated to the first test experiment in response to the available traffic determination task for the first test experiment to be started among a plurality of preset test experiments.

[0083] The filtering module 420 is used to filter out candidate user traffic that meets the filtering rules from the total user traffic based on the filtering rules pre-configured for the first test experiment;

[0084] The judgment module 430 is used to determine whether there is a mutual exclusion relationship between the first test experiment and any of the multiple test experiments that have been started.

[0085] The processing module 440 is used to filter user traffic that has not participated in the second test experiment from the candidate user traffic as the available traffic for the first test experiment when the judgment module determines that the first test experiment and the started second test experiment are mutually exclusive.

[0086] The apparatus provided in this embodiment can be used to perform... Figure 1 The steps, implementation principles, and implementation processes of the method embodiments shown can be found in the descriptions of the previous embodiments, and will not be repeated here.

[0087] Optionally, the determining module 410 is specifically used to determine the set of users who have performed a specified operation on the product under test corresponding to the first test experiment within a specified time period as the total user traffic.

[0088] Optionally, the determining module 410 is further configured to use the candidate user traffic as the available traffic of the first test experiment when the judging module 430 determines that there are no started test experiments among the multiple test experiments, or when the judging module 430 determines that there is no mutual exclusion relationship between the first test experiment and any started test experiment.

[0089] Optionally, the judgment module 430 is specifically used to determine, based on the mutual exclusion information of the first test experiment, whether at least one mutually exclusive experiment recorded in the mutual exclusion information is the same as any test experiment that has been started; wherein, the mutual exclusion information records the identification information of each mutually exclusive experiment that has a mutual exclusion relationship with the first test experiment;

[0090] The judgment module 430 is specifically used to determine that the first test experiment and the second test experiment that has been started have a mutual exclusion relationship when the mutual exclusion experiment recorded in the mutual exclusion information is the same as the second test experiment that has been started.

[0091] The judgment module 330 is specifically used to determine that the first test experiment and any of the started test experiments do not have a mutual exclusion relationship when the mutual exclusion experiment recorded in the mutual exclusion information is different from the second test experiment that has been started.

[0092] Optionally, the processing module 340 is further configured to display the available traffic to the testers after determining the available traffic.

[0093] The specific implementation process of the functions and roles of each unit in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0094] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this application according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0095] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for determining the available flow rate in an AB test experiment, characterized in that, The method comprises: determining total user traffic to be allocated to a first test experiment to be started among preset multiple test experiments in response to a task of determining available traffic for the first test experiment; wherein the task of determining available traffic is created before the first test experiment is started; screening candidate user traffic meeting a screening rule from the total user traffic based on the screening rule configured in advance for the first test experiment; judging whether the first test experiment and any test experiment started among the multiple test experiments have a mutual exclusion relationship; wherein the mutual exclusion relationship is determined based on mutual exclusion information in experiment configuration information of the first test experiment, and the mutual exclusion information records identification information of test experiments having the mutual exclusion relationship with the first test experiment; when the first test experiment and a second test experiment started have the mutual exclusion relationship, screening user traffic not participating in the second test experiment from the candidate user traffic as available traffic of the first test experiment; wherein the screening comprises excluding user traffic having participated in the second test experiment from the candidate user traffic based on test records of the second test experiment.

2. The method of claim 1, wherein, The determination of the total user traffic to be allocated to the first test experiment comprises: determining a user set performing a specified operation on a product to be tested corresponding to the first test experiment within a specified time period as the total user traffic.

3. The method of claim 1, wherein, When there is no test experiment started among the multiple test experiments, or when the first test experiment and any test experiment started do not have the mutual exclusion relationship, the method further comprises: taking the candidate user traffic as the available traffic of the first test experiment.

4. The method of claim 1, wherein, The judgment of whether the first test experiment and any test experiment started among the multiple test experiments have the mutual exclusion relationship comprises: determining whether at least one mutual exclusion experiment recorded in the mutual exclusion information is the same as any test experiment started according to the mutual exclusion information of the first test experiment; when the mutual exclusion experiment recorded in the mutual exclusion information is the same as a second test experiment started, determining that the first test experiment and the second test experiment started have the mutual exclusion relationship; when the mutual exclusion experiment recorded in the mutual exclusion information is not the same as the second test experiment started, determining that the first test experiment and any test experiment started do not have the mutual exclusion relationship.

5. The method according to claim 1 or 3, characterized in that, After determining the available traffic, the method further comprises: displaying the available traffic to a test personnel.

6. An apparatus for determining the usable flow rate in an AB test experiment, characterized in that, The device comprises a determination module, a screening module, a judgment module and a processing module, wherein: the determination module is configured to determine total user traffic to be allocated to a first test experiment to be started among preset multiple test experiments in response to a task of determining available traffic for the first test experiment; wherein the task of determining available traffic is created before the first test experiment is started; the screening module is configured to screen candidate user traffic meeting a screening rule from the total user traffic based on the screening rule configured in advance for the first test experiment; the judgment module is configured to judge whether the first test experiment and any test experiment started among the multiple test experiments have a mutual exclusion relationship; wherein the mutual exclusion relationship is determined based on mutual exclusion information in experiment configuration information of the first test experiment, and the mutual exclusion information records identification information of test experiments having the mutual exclusion relationship with the first test experiment; The judging module is configured to judge whether the first test experiment and any test experiment that has been started among the plurality of test experiments have a mutual exclusion relationship; wherein the mutual exclusion relationship is determined based on mutual exclusion information in experiment configuration information of the first test experiment, and the mutual exclusion information records identification information of test experiments that have the mutual exclusion relationship with the first test experiment; The processing module is configured to, when the judging module judges that the first test experiment and a second test experiment that has been started have a mutual exclusion relationship, filter user traffic that has not participated in the second test experiment from the candidate user traffic as available traffic of the first test experiment; wherein the filtering includes excluding user traffic that has participated in the second test experiment from the candidate user traffic based on test records of the second test experiment.

7. The apparatus of claim 6, wherein, The judging module is specifically configured to determine, according to mutual exclusion information of the first test experiment, whether at least one mutual exclusion experiment recorded in the mutual exclusion information is the same as any test experiment that has been started; wherein the mutual exclusion information records identification information of each mutual exclusion experiment that has the mutual exclusion relationship with the first test experiment. The judging module is specifically configured to, when a mutual exclusion experiment recorded in the mutual exclusion information is the same as a second test experiment that has been started, determine that the first test experiment and the second test experiment that has been started have a mutual exclusion relationship. The judging module is specifically configured to, when none of the mutual exclusion experiments recorded in the mutual exclusion information is the same as a second test experiment that has been started, determine that the first test experiment and any test experiment that has been started do not have a mutual exclusion relationship.

8. The apparatus of claim 6, wherein, The processing module is further configured to, after determining the available traffic, display the available traffic to a test personnel.

9. A device for determining the usable flow rate in an AB test experiment, characterized in that, A computer program product, comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the steps of the method of any one of claims 1-5 when executing the program.

10. A storage medium having stored thereon a program, characterized by The program is executed by the processor to implement the steps of the method of any one of claims 1-5.

Images