Blank traffic shunting method and device based on AB experiment, storage medium and electronic equipment

Abstract

The invention discloses a blank traffic shunting method and device based on an AB experiment, a storage medium and electronic equipment, and the method comprises the following steps: creating a scene in an AB experiment platform, and setting the proportion of blank traffic in the scene and an experiment traffic distribution rule; setting experiment configuration to create an experiment, and associating the scene, so that the flow distribution engine marks the flow with the corresponding proportion in the total flow as the blank flow according to the proportion of the blank flow during the experiment, and grouping the experiment flow according to an experiment flow distribution rule; starting an experiment, and collecting data of blank flow and experimental flow; using the data of the blank flow as a reference, comparing the data with the data of the experimental flow, and calibrating to obtain an experimental conclusion; by adopting the technology, the problem that the original blank traffic cannot be isolated in the existing AB experiment platform technology is solved, so that the business is assisted to carry out data quantitative analysis, and the experiment conclusion is more persuasive and reliable.

Description

Technical Field

[0001] The present invention relates to the technical field of AB testing, and in particular to a blank flow diversion method, device, storage medium and electronic equipment based on AB experiment. Background Art

[0002] AB testing, also known as controlled experiment, is a testing method that compares two or more (A / B version or A / B… / n version) different strategies, designs or versions. AB testing can be applied to various fields such as network applications, advertising, and marketing activities. It mainly divides users with the same (or similar) components into multiple user groups at random in the same time dimension, and allows the user groups to randomly access different versions of strategies for testing. The experience data and business data (such as conversion rate, click volume, retention rate and other indicators) of each user group are collected for comparison, and the differences between different versions of strategies are analyzed. The optimal version is analyzed and evaluated for formal adoption.

[0003] Refer to the client-side orthogonal mutual exclusion diversion method based on AB experiment with patent publication number CN202410485699.9, which includes the following steps: obtaining multiple test users, configuring multiple experiments through the AB platform, configuring multiple experiments through the AB platform includes configuring metadata, dividing multiple test users into multiple test experiment groups according to a preset user access traffic ratio; obtaining multiple test schemes, and constructing a knowledge graph with the test feedback data and multiple test experiment groups corresponding to the test schemes; performing statistical analysis on the test schemes according to the knowledge graph to generate an analysis report. It configures metadata for multiple experiments through the platform, and then sends the metadata configuration of the experiment to the client in real time through a long connection, calculates the bucket ID of the target device under each experiment on the client, and reports the user's diversion log, so as to greatly reduce the request interaction with the server in a high concurrency scenario, thereby reducing the pressure on the server-side storage.

[0004] However, the above technical solution directly distributes traffic to different versions for testing and comparison. The importance of blank traffic (blank traffic is the original traffic that has not undergone any experimental processing) is ignored. Without using blank traffic as a benchmark reference, it is difficult to infer what may happen in the absence of any intervention, making it impossible to accurately assess whether the difference between the control group (Group A) and the experimental group (Group B) is caused by experimental variables or other uncontrolled factors. Therefore, it is impossible to clearly distinguish the impact of experimental variables from the impact of environmental background factors. This will affect the judgment of the validity and reliability of the experimental results. This leads to deviations in the interpretation of the experimental results, making the decision-making basis inaccurate and scientific, which in turn limits the promotion and application of experimental effects in a wider range of situations. Summary of the invention

[0005] To this end, it is necessary to provide a blank traffic diversion method based on AB experiments, isolate a part of the blank traffic at the scenario level, and ensure that the diversion of all experiments in this scenario will not hit the blank traffic, so as to solve the problem that the original blank traffic cannot be isolated in the existing AB experiment platform technology, thereby assisting the business in conducting data quantitative analysis.

[0006] To achieve the above object, the present invention provides a blank flow diversion method based on AB experiment, which comprises the following steps:

[0007] Create a scenario on the AB experiment platform, set the ratio of blank traffic in the scenario and the experimental traffic allocation rules;

[0008] Set the experiment configuration to create an experiment and associate the scenario so that during the experiment, the traffic distribution engine will mark the corresponding proportion of the total traffic as blank traffic according to the proportion of blank traffic, and group the experimental traffic according to the experimental traffic distribution rules;

[0009] Start the experiment and collect data on blank flow and experimental flow;

[0010] The blank flow data is used as a benchmark and compared and calibrated with the experimental flow data to obtain the experimental conclusion.

[0011] Furthermore, before creating a scenario on the AB experiment platform and setting the proportion of blank traffic in the scenario, a blank traffic rule is set in the diversion rule of the traffic diversion engine.

[0012] Furthermore, in the step of setting the experimental configuration to create an experiment and associating the scenario so that during the experiment, the traffic diversion engine marks the corresponding proportion of the total traffic as blank traffic according to the proportion of blank traffic, and groups the experimental traffic according to the experimental traffic allocation rules, the experimental configuration includes the experiment name, experimental mode, experimental type, test version, test version number, and experiment duration.

[0013] Furthermore, the experiment configuration also includes an experiment description, an experiment expiration notification, and a notifier.

[0014] Further, in setting the experimental configuration to create an experiment and associating the scenario so that during the experiment, the traffic diversion engine marks a corresponding proportion of the total traffic as blank traffic according to the proportion of blank traffic, and in the step of grouping the experimental traffic according to the experimental traffic allocation rule, the experimental traffic is grouped according to the traffic allocation rule, including the following steps:

[0015] Determine whether the created experiment has a mutually exclusive relationship with the previously created experiment;

[0016] If it exists, the remaining traffic will be allocated to the created experiment and the previously created experiment as their respective experimental traffic according to the traffic allocation rules; otherwise, the remaining traffic will be used as the experimental traffic of the created experiment.

[0017] Furthermore, the experimental conclusions obtained are presented in the form of graphs and / or tables.

[0018] Furthermore, the experimental conclusions obtained include traffic overview, focus indicators and grouping details.

[0019] A device for blank flow diversion based on AB experiment, comprising

[0020] Traffic setting module: receives instructions for setting the ratio of blank traffic in the scene and traffic distribution rule instructions, and transmits them to the traffic diversion engine;

[0021] Traffic diversion engine module: receives the instruction of setting the ratio of blank traffic transmitted by the blank traffic setting module, marks the corresponding ratio of traffic as blank traffic according to the ratio of blank traffic; and groups the experimental traffic according to the traffic allocation rule instruction;

[0022] Data collection and analysis module: Data are collected for blank flow and experimental flow respectively. When analyzing the experimental results, the data of blank flow is used as a benchmark to compare and calibrate with the data of experimental flow to obtain the experimental conclusions.

[0023] A storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of the blank flow diversion method based on the AB experiment.

[0024] An electronic device comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the steps of the blank flow diversion method based on the AB experiment are implemented.

[0025] Different from the existing technology, the above technical solution mainly isolates a part of blank traffic at the scenario level to ensure that the diversion of all experiments in the scenario will not hit the blank traffic, that is, a part of the traffic is marked as blank traffic and no experimental processing is performed to solve the problem that the original blank traffic cannot be isolated in the existing AB experimental platform technology, thereby assisting the business in data quantitative analysis; by introducing blank traffic as a benchmark, a comparable original state is provided for the experimental results, other interference factors are eliminated, and more realistic user behavior data is obtained, thereby improving the accuracy of the experimental results and making the experimental conclusions more convincing and reliable; at the same time, it allows the proportion and allocation rules of blank traffic to be customized, and the proportion and allocation rules of blank traffic can be freely adjusted according to the experimental objectives and experimental plans to meet different experimental needs and improve the operability and flexibility of the experiment. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is a schematic flow chart of the blank flow diversion method based on AB experiment of the present invention;

[0027] Figure 2 Schematic diagram of the module of the device for blank flow diversion based on AB experiment.

[0028] Description of reference numerals:

[0029] 10. Flow setting module;

[0030] 20. Traffic diversion engine module;

[0031] 30. Data collection and analysis module. DETAILED DESCRIPTION

[0032] In order to explain the technical content, structural features, achieved objectives and effects of the technical solution in detail, the following is a detailed description in conjunction with specific embodiments and accompanying drawings.

[0033] Reference to "embodiment" herein means that the specific features, structures or characteristics described in conjunction with the embodiment may be included in at least one embodiment of the present application. The term "embodiment" appearing in various places in the specification does not necessarily refer to the same embodiment, nor does it particularly limit its independence or association with other embodiments. In principle, in the present application, as long as there is no technical contradiction or conflict, the various technical features mentioned in the embodiments can be combined in any way to form a corresponding implementable technical solution.

[0034] Unless otherwise defined, the technical terms used in this document have the same meanings as those generally understood by those skilled in the art to which this application belongs; the use of relevant terms in this document is only for describing specific embodiments and is not intended to limit this application.

[0035] In the description of this application, the term "and / or" is an expression used to describe the logical relationship between objects, indicating that three relationships may exist, for example, A and / or B, which means: A exists, B exists, and A and B exist at the same time. In addition, the character " / " in this article generally indicates that the objects before and after are in an "or" logical relationship.

[0036] In the present application, terms such as “first” and “second” are merely used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship of quantity, priority or sequence between these entities or operations.

[0037] Without further limitations, in this application, the words "include", "comprises", "has" or other similar expressions used in the sentences are intended to cover non-exclusive inclusion. These expressions do not exclude the presence of additional elements in the process, method or product including the elements, so that the process, method or product including a series of elements may include not only those limited elements, but also other elements not explicitly listed, or also include elements inherent to such process, method or product.

[0038] Similar to the understanding in the Patent Examination Guidelines, in this application, expressions such as "greater than", "less than", "exceed" and the like are understood to exclude the number itself; expressions such as "above", "below", "within" and the like are understood to include the number itself. In addition, in the description of the embodiments of this application, "multiple" means more than two (including two), and similar expressions related to "multiple" are also understood in this way, such as "multiple groups", "multiple times", etc., unless otherwise clearly and specifically limited.

[0039] In the description of the embodiments of the present application, space-related expressions used, such as "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "vertical", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", "axial", "radial", "circumferential", etc., indicate the orientation or position relationship based on the orientation or position relationship shown in the specific embodiments or drawings, and are only for the convenience of describing the specific embodiments of the present application or facilitating the reader's understanding, and do not indicate or imply that the referred device or component must have a specific position, a specific orientation, or be constructed or operated in a specific orientation, and therefore cannot be understood as a limitation on the embodiments of the present application.

[0040] Unless otherwise expressly specified or limited, in the description of the embodiments of the present application, the terms such as "install", "connect", "connect", "fix", "set", etc. used should be understood in a broad sense. For example, the "connection" can be a fixed connection, a detachable connection, or an integrated setting; it can be a mechanical connection, an electrical connection, or a communication connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium; it can be the internal connection of two elements or the interaction relationship between two elements. For technicians in the technical field to which the present application belongs, the specific meanings of the above terms in the embodiments of the present application can be understood according to the specific circumstances.

[0041] join Figure 1As shown, the present invention provides a blank traffic diversion method based on AB experiments, which mainly isolates a part of blank traffic at the scenario level to ensure that the diversion of all experiments in the scenario will not hit the blank traffic, that is, a part of the traffic is marked as blank traffic, and no experimental processing is performed to solve the problem that the original blank traffic cannot be isolated in the existing AB experimental platform technology, thereby assisting the business in data quantitative analysis; by introducing blank traffic as a benchmark, a comparable original state is provided for the experimental results, other interference factors are eliminated, and more realistic user behavior data is obtained, thereby improving the accuracy of the experimental results and making the experimental conclusions more convincing and reliable; at the same time, the proportion and allocation rules of blank traffic can be customized, and the proportion and allocation rules of blank traffic can be freely adjusted according to the experimental objectives and experimental plans to meet different experimental needs, improve the operability and flexibility of the experiment, and help product managers and developers better understand the natural state of user behavior, thereby optimizing the design and parameter settings of the experiment.

[0042] The following further describes a blank flow diversion method based on AB experiment of the present invention, which comprises the following steps:

[0043] S1. Create a scenario on the AB experiment platform, set the ratio of blank traffic in the scenario and the experimental traffic allocation rules;

[0044] S2. Set the experiment configuration to create an experiment and associate the scenario so that during the experiment, the traffic diversion engine marks a corresponding proportion of the total traffic as blank traffic according to the proportion of blank traffic, and groups the experimental traffic according to the experimental traffic allocation rule;

[0045] S3, start the experiment and collect data of blank flow and experimental flow;

[0046] S4. Use the blank flow data as a benchmark, compare and calibrate it with the experimental flow data, and obtain the experimental conclusion.

[0047] The above scenario is a specific scenario under the entire application system, such as promotional activities, recommendation algorithms, etc. The above experiment is a specific experimental project, testing different versions, including experimental groups and control groups, etc. The above setting sets the proportion of blank traffic. After setting, this part of blank traffic will not be allocated to any experiment in this scenario. Grouping the experimental traffic according to the experimental traffic allocation rules means grouping the experimental traffic according to the experimental traffic allocation rules so that the experimental traffic can randomly access the experimental group and the control group by group during the experiment. The experimental traffic can be grouped according to the experimental traffic allocation rules according to the proportional allocation, directional allocation, random allocation, average allocation and other allocation rules.

[0048] The above-mentioned traffic diversion engine is implemented by modifying the existing traffic allocation engine according to the blank traffic rule and adding a blank traffic mark. That is, a blank traffic rule is set in the diversion rule of the traffic diversion engine. Specifically, a blank traffic mark can be added to the traffic allocation engine, and according to the blank traffic rule, part of the total traffic (according to the set blank traffic ratio, blank traffic = blank traffic ratio * total traffic) is marked as blank traffic. This part of blank traffic will not be allocated to any experiment in this scenario. At the same time, part of the traffic will be allocated to the experimental group as experimental traffic, and the experiment will be run within the set experimental duration.

[0049] In the above step S2, the experimental configuration is set to create an experiment, and the scenario is associated so that during the experiment, the traffic diversion engine marks the corresponding proportion of the total traffic as blank traffic according to the proportion of blank traffic, and groups the experimental traffic according to the experimental traffic allocation rules. The experimental configuration may include but is not limited to the experiment name, experimental mode, experimental type, test version, test version number, and experiment duration.

[0050] The above experimental modes may include programming experiments, business system experiments, etc.; the above experimental types include client and server; the above test versions include all application and Web application ends. 、 App application side, etc.; the above test version number can be selected from the version library, which can include development version files, debug version files, and released version files, etc.

[0051] By setting the experimental mode, experimental type, test version, and test version number, you can control and manage the experiment more precisely, thereby reducing the impact of external factors on the experimental results and strengthening the control and effect of the experiment. At the same time, by setting the experimental duration, you can ensure the stability and continuity of the experiment and avoid deviations and errors in the experimental results due to too short or too long an experimental time.

[0052] In practical applications, the experiment configuration may also include an experiment description, an experiment expiration notification, a notifier, etc. By setting the experiment description, the purpose, process, and expected effect of the experiment can be more clearly understood, thereby improving the comprehensibility and participation of the experiment. By setting the experiment expiration notification and the notifier, the participants of the experiment can be reminded of the expiration time of the experiment so that they can understand the experimental conclusion in a timely manner.

[0053] In the above step S2, the experiment configuration is set to create an experiment, and the scenario is associated so that during the experiment, the traffic diversion engine marks the corresponding proportion of the total traffic as blank traffic according to the proportion of blank traffic, and the experimental traffic is grouped according to the experimental traffic allocation rule. The experimental traffic is grouped according to the traffic allocation rule, and it also includes determining whether there is a mutually exclusive relationship between the experiments, which specifically includes the following steps:

[0054] Determine whether the created experiment has a mutually exclusive relationship with the previously created experiment;

[0055] If it exists, the remaining traffic will be allocated to the created experiment and the previously created experiment as their respective experimental traffic according to the traffic allocation rules; otherwise, the remaining traffic will be used as the experimental traffic of the created experiment.

[0056] The above remaining traffic = total traffic - ratio of blank traffic * total traffic. The above allocating the remaining traffic to the created experiment and the previously created experiment as their respective experimental traffic according to the traffic allocation rules means allocating the remaining traffic according to the experimental traffic allocation rules, so that the created experiment and the previously created experiment are tested according to the allocated experimental traffic, and the traffic in the remaining traffic that does not participate in the previously created experiment is used as the experimental traffic of the created experiment. Allocating the remaining traffic according to the experimental traffic allocation rules can be grouped according to allocation rules such as proportional allocation, directional allocation, random allocation, and average allocation.

[0057] The above-mentioned AB experiment platform creates a scene, sets the ratio of blank traffic in the scene and the experimental traffic allocation rules and / or sets the experimental configuration to create an experiment, and associates the scene. This can be done directly through instructions, or through visual clicking, dragging, and manual input. For example, the visual interface of the AB experiment platform provides a friendly interactive interface, which can be used to create a scene, set the ratio of blank traffic in the scene and the experimental traffic allocation rules and / or set the experimental configuration to create an experiment, and associate the scene by clicking, dragging, and manual input. The experimental conclusions obtained above can be displayed in the form of graphs and / or tables, and can also be displayed through the visual interface of the AB experiment platform.

[0058] The experimental conclusions obtained in the above step S4 may include traffic overview, focus indicators and group details, etc. The traffic overview can help understand the distribution of experimental traffic, evaluate the quality and effect of experimental traffic, optimize the distribution and control of experimental traffic, so as to achieve the accuracy and reliability of experimental results. Focus indicators can estimate the effectiveness of the experiment, understand the impact of the experiment on business goals, determine the success criteria and thresholds of the experiment, and optimize the strategies and measures of the experiment, so as to achieve the optimization of the experiment and the realization of business goals. Group details can help experimenters determine the best experimental grouping, understand the experimental effects of different groups, optimize the distribution and control of experimental traffic, and improve the credibility and accuracy of the experiment, so as to achieve the optimization of experimental effects and the scientific nature of decision-making.

[0059] See also Figure 2 As shown, the present invention also provides a device for blank flow diversion based on AB experiment, which comprises

[0060] Traffic setting module 10: receiving instructions for setting the ratio of blank traffic in the scene and traffic distribution rule instructions, and transmitting them to the traffic diversion engine;

[0061] Traffic diversion engine module 20: receiving the instruction for setting the ratio of blank traffic transmitted by the blank traffic setting module, marking the corresponding ratio of traffic as blank traffic according to the ratio of blank traffic; and grouping the experimental traffic according to the traffic allocation rule instruction;

[0062] Data collection and analysis module 30: collects data for blank flow and experimental flow respectively, and when analyzing the experimental results, uses the data of blank flow as a benchmark, compares and calibrates with the data of experimental flow, and obtains experimental conclusions.

[0063] The present invention also provides a storage medium, which stores a computer program, and when the computer program is executed by a processor, the steps of the blank flow diversion method based on the AB experiment are implemented. The computer program involved in the embodiment can be stored in a computer device readable storage medium, and the computer device readable storage medium includes but is not limited to a disk, a tape, a magnetic card, a floppy disk, a flash memory, an optical disk, an optical card, a read-only memory (ROM), a random access memory (RAM), an erasable programmable ROM (EPROM) and an electrically erasable programmable ROM (EEPROM), etc., and also includes other biological, physical or chemical structures that can realize the same or equivalent functions as the storage medium listed above, such as DNA, RNA, protein and other units with information storage capabilities. In a specific embodiment, the storage medium involved can be one of the above-mentioned media types, or a combination of the above-mentioned media types. In different embodiments, the computer program involved in the embodiment can be stored in a single medium in a centralized manner, or it can be stored in multiple media in a distributed manner. The memory containing a computer device readable storage medium can be a non-volatile memory or a random access memory. These computer device readable storage media may be built into the device, or may be connected to the device involved in the embodiment as an external device or part of an external device. In some embodiments, a memory with a computer device readable storage medium is deployed locally; in other embodiments, a solution of deploying the memory away from the processor may also be adopted, such as a network attached memory accessed via an RF circuit or an external port and a communication network, wherein the communication network may be the Internet, one or more intranets, a local area network (LAN), a wide area network (WLAN), a storage area network (SAN), etc., or a suitable combination thereof, as long as the computer device can access the memory. In addition, the computer program involved in the embodiment may be stored in plaintext / ciphertext form, or may be designed as training data, which may be integrated and reorganized through model training and implicitly stored in the parameter state of a deep neural network or other machine learning model.

[0064] The present invention also provides an electronic device, characterized in that it includes a memory and a processor, wherein a computer program is stored on the memory, and when the computer program is executed by the processor, the steps of the blank flow diversion method based on the AB experiment are implemented. The processor described in the embodiment of the present application can be implemented by hardware, firmware, software or a combination thereof, and can use a circuit, a single or multiple application-specific integrated circuits (Application Specific Integrated Circuit, ASIC), a digital signal processor (Digital Signal Processor, DSP), a digital signal processing device (Digital Signal Processing Device, DSPD), a programmable logic device (Programmable Logic Device, PLD), a field programmable gate array (Field Programmable Gate Array, FPGA), a central processing unit (Central Processing Unit, CPU), a controller, a microcontroller, at least one of a microprocessor, and also includes other physical, biological or chemical structures that can realize functions similar to or equivalent to the above-listed processors, such as biological neurons, quantum computing units, DNA computing units, etc., so that the processor can execute some or all of the steps in the computer program or method involved in each embodiment of the present application or any combination of the steps mentioned therein.

[0065] It should be noted that, although the above embodiments have been described in this article, the patent protection scope of the present invention is not limited thereby. Therefore, based on the innovative concept of the present invention, changes and modifications made to the embodiments described herein, or equivalent structures or equivalent process changes made using the contents of the present invention specification and drawings, directly or indirectly applying the above technical solutions to other related technical fields, are all included in the patent protection scope of the present invention.

Claims

1. A blank flow diversion method based on AB experiment, characterized in that: The following steps are involved: Create a scenario on the AB experiment platform, set the ratio of blank traffic in the scenario and the experimental traffic allocation rules; Set the experiment configuration to create an experiment and associate the scenario so that during the experiment, the traffic distribution engine will mark the corresponding proportion of the total traffic as blank traffic according to the proportion of blank traffic, and group the experimental traffic according to the experimental traffic distribution rules; Start the experiment and collect data on blank flow and experimental flow; The blank flow data is used as a benchmark and compared and calibrated with the experimental flow data to obtain the experimental conclusion.

2. The blank flow diversion method based on AB experiment according to claim 1 is characterized in that: Before creating a scenario on the AB experiment platform and setting the proportion of blank traffic in the scenario, a blank traffic rule is set in the diversion rule of the traffic diversion engine.

3. The blank flow diversion method based on AB experiment according to claim 1 is characterized in that: In the step of setting the experimental configuration to create an experiment and associating the scenario so that during the experiment, the traffic diversion engine marks the corresponding proportion of the total traffic as blank traffic according to the proportion of blank traffic, and groups the experimental traffic according to the experimental traffic allocation rules, the experimental configuration includes the experiment name, experimental mode, experimental type, test version, test version number, and experiment duration.

4. The blank flow diversion method based on AB experiment according to claim 1 is characterized in that: The experiment configuration also includes an experiment description, an experiment expiration notification, and a notifier.

5. The blank flow diversion method based on AB experiment according to claim 1 is characterized in that: In the step of setting the experiment configuration to create an experiment and associating the scenario so that the traffic diversion engine marks a corresponding proportion of the total traffic as blank traffic according to the proportion of blank traffic during the experiment, and grouping the experimental traffic according to the experimental traffic allocation rule, the following steps are included: Determine whether the created experiment has a mutually exclusive relationship with the previously created experiment; If it exists, the remaining traffic will be allocated to the created experiment and the previously created experiment as their respective experimental traffic according to the traffic allocation rules; otherwise, the remaining traffic will be used as the experimental traffic of the created experiment.

6. The blank flow diversion method based on AB experiment according to claim 1 is characterized in that: The experimental conclusions obtained are presented in the form of graphs and / or tables.

7. The blank flow diversion method based on AB experiment according to claim 1 is characterized in that: The experimental conclusions obtained include traffic overview, focus indicators and group details.

8. A device for blank flow diversion based on AB experiment, characterized in that: include Traffic setting module: receives instructions for setting the ratio of blank traffic in the scene and traffic distribution rule instructions, and transmits them to the traffic diversion engine; Traffic diversion engine module: receives the instruction of setting the ratio of blank traffic transmitted by the blank traffic setting module, marks the corresponding ratio of traffic as blank traffic according to the ratio of blank traffic; and groups the experimental traffic according to the traffic allocation rule instruction; Data collection and analysis module: Data are collected for blank flow and experimental flow respectively. When analyzing the experimental results, the data of blank flow is used as a benchmark to compare and calibrate with the data of experimental flow to obtain the experimental conclusions.

9. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the blank flow diversion method based on the AB experiment as described in any one of claims 1 to 7 are implemented.

10. An electronic device, characterized in that: It comprises a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the steps of the blank flow diversion method based on the AB experiment as described in any one of claims 1 to 7 are implemented.

Images