A / B experiment diversion method, device, storage medium and computer equipment

By using the Bloom filter and bucketing algorithm in the A/B experiment on the server, the problem of difficult to ensure packet consistency on the server is solved, and fast and reliable sample traffic determination and efficient shunt processing are achieved, which meets high concurrency , low latency, high availability requirements.

CN116016361BActive Publication Date: 2024-11-15SHENZHEN YISHIHUOLALA TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202211590707.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-12-12
Publication Date
2024-11-15
Estimated Expiration
2042-12-12

AI Technical Summary

Technical Problem

In the A/B experiments in the Internet field, it is difficult for the server to ensure the consistency of packets under high concurrency, low latency and high availability conditions, especially in front-end style-related experiments, the user's group jump phenomenon will affect the user experience.

Method used

Using the Bloom filter and bucket algorithm, by initializing a Bloom filter and a bit bucket for each packet, multiple hashing and bucket algorithms are used to determine the packet of sample traffic, and the distribution is performed when the shunt ratio is adjusted. The union of components is processed to ensure accurate shunt of sample flow.

Benefits of technology

It realizes fast and reliable sample traffic determination, meets the performance requirements of the A/B experimental SDK on the server side, ensures packet consistency, improves processing efficiency and supports high concurrency, low latency, and high availability requirements.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116016361B_ABST
    Figure CN116016361B_ABST
Patent Text Reader

Abstract

The present invention provides an A / B experiment diversion method, which is applied to a server and includes the following steps: obtaining sample traffic to be diverted; determining whether the diversion ID of the sample traffic exists in the Bloom filter of the grouping in the A / B experiment; if so, determining that the sample traffic is divided into the group corresponding to the Bloom filter whose diversion ID exists; if not, determining the grouping of the sample traffic based on a bucketing algorithm, and adding the diversion ID to the Bloom filter of the determined grouping for training. This method sets a Bloom filter and a bucket for each grouping in the A / B experiment, and each Bloom filter can quickly and reliably complete the determination of the sample traffic that has entered the grouping, thereby meeting the performance requirements of the A / B experiment SDK on the server side. And because the Bloom filter occupies little memory, the processing efficiency is very high, providing strong technical support to meet the requirements of ensuring that the server faces high concurrency, low latency, and high availability, and enabling the A / B experiment SDK on the server side to ensure that the grouping always remains consistent.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to an A / B experiment diversion method, apparatus, computer-readable storage medium and computer equipment. Background Art

[0002] Since the A / B experiment was proposed in the Internet field, as the current Internet traffic dividend gradually expires, A / B experiment has become a very popular technology for selecting product iteration directions. A / B experiments can be carried out first in areas such as APP style selection, short video push strategy, iterative optimization of machine learning algorithms, etc. The data collected from A / B experiments can be scientifically analyzed, and the final iteration strategy of the product can be selected through significance verification.

[0003] In A / B experiments, it is usually necessary to ensure that "users who have entered a group will not leave the group". For example, if user 1 enters an A / B experiment for the first time and is assigned to group A, then no matter how many times he enters the experiment, he must hit group A. This feature is particularly important in experiments related to front-end styles. If a user enters an experiment multiple times and jumps to another group, the style he sees will be different every time he opens the app, which will inevitably greatly affect the user experience.

[0004] If the A / B experiment SDK is integrated in the client (Android, IOS, etc.), "users who have joined a group will not leave the group" is relatively easy to implement, and it only needs to be cached in the local file of the client. However, if it is a server-side A / B experiment SDK, there is no known good implementation method in the industry to ensure that the server meets the requirements of high concurrency, low latency, and high availability. It is difficult for the server-side A / B experiment SDK to ensure that the grouping remains consistent. Summary of the invention

[0005] In order to solve at least one of the above-mentioned technical defects, the present invention provides an A / B experiment diversion method and corresponding device, computer-readable storage medium and computer equipment according to the following technical solution.

[0006] According to one aspect, an embodiment of the present invention provides an A / B experiment splitting method, comprising the following steps:

[0007] Obtaining sample flow to be diverted;

[0008] Determine whether the diversion ID of the sample traffic exists in the Bloom filter grouped in the A / B experiment;

[0009] If so, determine that the sample traffic is divided into a group corresponding to the Bloom filter whose diversion ID exists;

[0010] If not, the sample traffic is grouped based on a bucketing algorithm, and the diversion ID is added to the Bloom filter of the determined group for training.

[0011] In some feasible embodiments, the A / B experiment diversion method further includes:

[0012] Divide the A / B experiment groups in advance and determine the traffic distribution ratio between each group;

[0013] A Bloom filter and a bit bucket are initialized for each group of the A / B experiment; wherein all positions of the initialized Bloom filter are empty, and the assignment position of the bit bucket of each group is determined according to the diversion ratio.

[0014] In some feasible embodiments, the A / B experiment diversion method further includes:

[0015] When the diversion ratio is adjusted, the Bloom filters of each group in the A / B experiment of different nodes are respectively combined through the distributed component.

[0016] In some feasible embodiments, determining whether the diversion ID of the sample traffic exists in the Bloom filter grouped in the A / B experiment includes:

[0017] The diversion ID is hashed multiple times to obtain a hash result;

[0018] The hash results are mapped to different positions on the Bloom filters grouped in the A / B experiment, and it is determined whether all the positions are assigned values.

[0019] In some feasible embodiments, determining that the sample traffic is divided into a group corresponding to a Bloom filter whose diversion ID exists includes:

[0020] When the ID of the sample flow exists in Bloom filters of two or more groups in the A / B experiment, it is determined that the sample flow is divided into a group corresponding to any Bloom filter whose diversion ID exists.

[0021] In some feasible embodiments, the determining the grouping of the sample traffic based on a bucketing algorithm includes:

[0022] After hashing the diversion ID, a modulo operation is performed according to the initialization size of the bit bucket grouped in the A / B experiment to obtain a modulo operation result;

[0023] A bit bucket having a value assigned at a position corresponding to the modulo operation result is determined, and a group corresponding to the bit bucket is determined as the group of the sample traffic.

[0024] In some feasible embodiments, adding the diversion ID to a Bloom filter for determining a group for training includes:

[0025] The diversion ID is hashed multiple times to obtain a hash result;

[0026] The hash result is mapped to different positions on the Bloom filter that determines the grouping, and values ​​are assigned to the positions.

[0027] In addition, according to another aspect, an embodiment of the present invention provides an A / B experiment splitting device, including:

[0028] A flow acquisition module, used to acquire sample flow to be diverted;

[0029] A judgment module, used to judge whether the diversion ID of the sample flow exists in the Bloom filter grouped in the A / B experiment;

[0030] A flow splitting module, configured to determine, when the flow splitting ID of the sample flow exists in the Bloom filter grouped in the A / B experiment, to split the sample flow into a group corresponding to the Bloom filter whose flow splitting ID exists;

[0031] The diversion module is also used to determine the grouping of the sample flow based on a bucketing algorithm when the diversion ID of the sample flow does not exist in the Bloom filter grouped in the A / B experiment, and add the diversion ID to the Bloom filter that determines the grouping for training.

[0032] According to another aspect, an embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the above-mentioned A / B experiment diversion method is implemented.

[0033] According to another aspect, an embodiment of the present invention provides a computer device, wherein the computer includes one or more processors; a memory; and one or more computer programs, wherein the one or more computer programs are stored in the memory and configured to be executed by the one or more processors, and the one or more computer programs are configured to: execute the above-mentioned A / B experiment diversion method.

[0034] Compared with the prior art, the present invention has the following beneficial effects:

[0035] The A / B experiment diversion method, device, computer-readable storage medium and computer equipment provided by the present invention can quickly and reliably complete the determination of sample traffic that has entered the group through each Bloom filter by setting a Bloom filter and a bucket for each group in the A / B experiment, thereby meeting the performance requirements of the server-side A / B experiment SDK. And because the Bloom filter occupies a small amount of memory, the processing efficiency is very high, providing strong technical support to meet the requirements of ensuring that the server faces high concurrency, low latency and high availability, and enabling the server-side A / B experiment SDK to ensure that the grouping always remains consistent.

[0036] Additional aspects and advantages of the present invention will be given in part in the following description, which will become obvious from the following description, or may be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0038] Figure 1 A method flow chart of the A / B experiment diversion method provided by an embodiment of the present invention;

[0039] Figure 2 A schematic diagram of an initialization bit bucket provided in an embodiment of the present invention;

[0040] Figure 3 A schematic diagram of merging Bloom filters provided in an embodiment of the present invention;

[0041] Figure 4 A schematic diagram of the structure of an A / B experiment splitting device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0042] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be interpreted as limiting the present invention.

[0043] Those skilled in the art will appreciate that, unless otherwise stated, the singular forms "a", "an", "said" and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present invention refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. The term "and / or" used herein includes all or any unit and all combinations of one or more associated listed items.

[0044] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as generally understood by those skilled in the art in the art to which the present invention belongs. It should also be understood that terms such as those defined in general dictionaries should be understood to have meanings consistent with the meanings in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless specifically defined as herein.

[0045] The embodiment of the present invention provides an A / B experiment diversion method, which is applied to a server, such as Figure 1 As shown, the method includes:

[0046] Step S110: Obtain the sample flow to be diverted.

[0047] For this embodiment, the sample traffic to be distributed is specifically the sample traffic to be distributed into any group in the current A / B experiment. Each sample traffic is identified by its distribution ID (Identity Document, unique code).

[0048] Step S120: Determine whether the diversion ID of the sample flow exists in the Bloom filter grouped in the A / B experiment.

[0049] In this embodiment, a Bloom filter is set for each group of the A / B experiment, and the Bloom filter can be used to determine whether the current sample traffic has been divided into its corresponding group.

[0050] Bloom filter was proposed by Bloom in 1970. It is actually composed of a very long binary vector and a series of arbitrary mapping functions. It is a probability-based data structure, mainly used to determine whether an element is in a set. It has the advantages of fast running speed and small memory usage. It can quickly determine that an element is definitely not in the set or may be in the set.

[0051] In one possible implementation, the judgment process is specifically as follows: hashing the diversion ID multiple times to obtain a hash result; mapping the hash result to different positions on the Bloom filter grouped in the A / B experiment, and judging whether all the positions are assigned values.

[0052] Among them, hash is also called hash, hash, or transliterated as hash, which is to transform an input of any length into an output of fixed length through a hash algorithm, and the output is the hash value. The diversion ID is transformed by multiple hash functions respectively to obtain hash results corresponding to each hash function, and the hash results corresponding to each hash function are respectively mapped to different positions on the Bloom filter of the group in the A / B experiment to determine whether these positions are all assigned values. By mapping the hash results to the Bloom filter positions of all groups in the A / B experiment, it is determined whether the diversion ID exists in the Bloom filter of any at least one group in the A / B experiment.

[0053] Since Bloom filters take up little memory and the judgment logic uses hashed position comparison, the processing efficiency is very high, providing strong technical support for meeting the requirements of high concurrency, low latency, and high availability for the server.

[0054] In addition, by training a Bloom filter for each group, each Bloom filter can quickly and reliably determine the sample traffic that has entered the group, meeting the performance requirements of the server-side A / B experiment SDK.

[0055] Step S130: If yes, determine that the sample traffic is divided into a group corresponding to the Bloom filter whose diversion ID exists.

[0056] For this embodiment, the hash results corresponding to each hash function are mapped to different positions on the Bloom filter of a group in the A / B experiment. If these positions are all assigned values, it can be determined that the diversion ID of the sample traffic exists in the Bloom filter of the group.

[0057] Among them, the diversion ID of a sample flow may exist in the Bloom filters of multiple groups.

[0058] In a possible implementation, when the ID of the sample traffic exists in Bloom filters of two or more groups in the A / B experiment, it is determined that the sample traffic is divided into a group corresponding to any Bloom filter whose diversion ID exists.

[0059] In a possible implementation, when the ID of the sample traffic exists in Bloom filters of two or more groups in the A / B experiment, it is determined that the sample traffic is divided into the group corresponding to the first Bloom filter whose diversion ID exists.

[0060] In addition, the expected error rate of the Bloom filter can be estimated during the initialization phase, and multiple Bloom filters are used for verification when determining whether the diversion ID of the sample traffic exists in the Bloom filter grouped in the A / B experiment, thereby ensuring that the error rate will be lower than the expected error rate.

[0061] Step S140: If not, determine the grouping of the sample traffic based on a bucketing algorithm, and add the diversion ID to the Bloom filter of the determined grouping for training.

[0062] For this embodiment, the hash results corresponding to each hash function are mapped to different positions on the Bloom filter of a group in the A / B experiment. If there is at least one unassigned position among these positions, it can be determined that the diversion ID of the sample traffic does not exist in the Bloom filter of the group. When there is at least one unassigned position on the Bloom filters of all groups, it can be determined that the diversion ID of the sample traffic does not exist in the Bloom filter of the group in the A / B experiment.

[0063] For this embodiment, when the sample traffic is not divided into any group of the A / B experiment, the sample traffic is randomly diverted based on the bucketing algorithm to determine the group of the sample traffic in the A / B experiment. After the group of the sample traffic is determined, the diversion ID of the sample traffic is added to the Bloom filter of the group for training, so that the Bloom filter can be used to determine whether the sample traffic has been divided into the group.

[0064] The A / B experiment diversion method provided by the embodiment of the present invention sets a Bloom filter and a bucket for each group in the A / B experiment, and each Bloom filter can quickly and reliably complete the determination of the sample flow that has entered the group, meeting the performance requirements of the server-side A / B experiment SDK. And because the Bloom filter occupies a small amount of memory, the processing efficiency is very high, providing strong technical support to meet the requirements of ensuring that the server faces high concurrency, low latency, and high availability, and enabling the server-side A / B experiment SDK to ensure that the grouping always remains consistent.

[0065] In some feasible embodiments, the A / B experiment diversion method also includes: pre-dividing the A / B experiment into groups and determining the diversion ratio between each group; initializing a Bloom filter and a bit bucket for each group of the A / B experiment; wherein all positions of the initialized Bloom filter are empty, and the assignment position of the bit bucket of each group is determined according to the diversion ratio.

[0066] Specifically, before the A / B experiment diversion is officially started, each A / B experiment is pre-divided into groups. For example, the current A / B experiment is pre-divided into two groups, Group A and Group B, and the sample flow to be diverted is divided into Group A or Group B. The number of the groups can be set according to actual application requirements, and the embodiment of the present invention does not limit this.

[0067] For this embodiment, a Bloom filter is initialized for each group of the A / B experiment. The Bloom filter is composed of a very long binary vector and a series of arbitrary mapping functions. All positions of the initialized Bloom filter are empty. When a sample flow is divided into a group, the flow ID of the sample flow will be added to the Bloom filter of the group for training, and the corresponding position on the Bloom filter will be assigned a value so that the Bloom filter can be used to determine that the sample flow has been divided into the group.

[0068] The size of the Bloom filter, that is, the number of assignment positions, is determined by the estimated sample traffic number n and the expected error rate P given by the user, and the formula is:

[0069]

[0070] The number of hash functions used to transform the diversion ID is determined by the size m of the Bloom filter and the expected error rate P, and the formula is:

[0071]

[0072] By estimating the expected error rate in the initialization phase and using multiple Bloom filters to verify whether the diversion ID of the sample traffic exists in the Bloom filter grouped in the A / B experiment, it can be ensured that the error rate will be lower than the expected error rate.

[0073] In this embodiment, a bit bucket is initialized for each group of the A / B experiment, and a split ratio is preset between each group. The assignment position of the bit bucket of each group in the initialized state is determined according to the split ratio. Figure 2 As shown, it is a schematic diagram of initializing the bit bucket. The current A / B experiment is pre-divided into two groups, namely Group A and Group B. Among them, the diversion ratio of Group A to Group B is 40% to 60%. Then, the first 40% positions of the bit bucket of Group A are assigned values, and the last 60% positions of the bit bucket of Group A are empty. The last 60% positions of the bit bucket of Group B are assigned values, and the first 40% positions of the bit bucket of Group B are empty.

[0074] The size of the initialization bit bucket needs to be set, that is, the number of assignable positions. For example, each group initializes a bit bucket of size 100.

[0075] In some feasible embodiments, the grouping of the sample traffic based on the bucketing algorithm in step S140 includes: performing a hash operation on the diversion ID and then performing a modulo operation on the initialization size of the bit bucket of the grouping in the A / B experiment to obtain a modulo operation result; determining a bit bucket with an assigned value at a position corresponding to the modulo operation result, and determining the group corresponding to the bit bucket as the grouping of the sample traffic.

[0076] For example, for a bit bucket with an initialization size of 100, the sample traffic is randomly diverted based on the bucketing algorithm. Specifically, the diversion ID is hashed and modulo 100 to obtain a modulo operation result, where the modulo operation result is an integer from 0 to 99. The modulo operation result is judged in the bit buckets of each group in the A / B experiment, and the group corresponding to the bit bucket with an assigned value at the position corresponding to the modulo operation result is determined as the group after the sample traffic is randomly diverted.

[0077] In some feasible embodiments, in step S140, the diversion ID is added to the Bloom filter for determining the group for training, including: hashing the diversion ID multiple times to obtain a hash result; mapping the hash result to different positions on the Bloom filter for determining the group, and assigning values ​​to the positions.

[0078] For this embodiment, after determining the grouping of the sample traffic, the diversion ID of the sample traffic is added to the Bloom filter of the group for training. Specifically, the diversion ID is transformed by multiple hash functions to obtain hash results corresponding to each hash function. The hash results corresponding to each hash function are mapped to different positions on the Bloom filter of the group, and values ​​are assigned to these positions.

[0079] In some feasible embodiments, the A / B experiment diversion method further includes: when the diversion ratio is adjusted, the Bloom filters of each group in the A / B experiment of different nodes are respectively combined by distributed components.

[0080] For this embodiment, if the diversion ratio between each group in the A / B experiment is adjusted, the Bloom filter is merged through distributed components such as Redis (Remote Dictionary Server, i.e., remote dictionary service, is an open source log-type, Key-Value database written in ANSI C language, supporting the network, memory-based and persistent, and providing APIs in multiple languages.), such as Figure 3 As shown, the Bloom filters of the same group of different nodes are merged, the same positions of the two Bloom filters are processed by union, and finally the new Bloom filter obtained after the union processing is sent to each node to replace the original Bloom filters of each group.

[0081] In this embodiment, the initialization method of the bit bucket can ensure the consistency of distributed nodes. By adopting the parallel method of the Bloom filter and hash bucket strategy, the Bloom filter needs to be synchronized with distributed components such as Redis when the group traffic ratio is adjusted, avoiding the performance loss caused by requesting the central node for each request. On each node, the Bloom filters of the same group are unioned and sent to each node to replace the original Bloom filter. Repeating the above steps can realize the function of the A / B experiment SDK (Software Development Kit) on the server side that "users who enter the group no longer leave the group", and realize the A / B experiment diversion method applied to the server.

[0082] In addition, an embodiment of the present invention provides an A / B experiment splitting device, such as Figure 4 As shown, the device includes: a flow acquisition module 10, a judgment module 20 and a flow distribution module 30; wherein,

[0083] The flow acquisition module 10 is used to acquire the sample flow to be diverted;

[0084] The judgment module 20 is used to judge whether the diversion ID of the sample flow exists in the Bloom filter grouped in the A / B experiment;

[0085] The flow splitting module 30 is used to determine that the sample flow is split into a group corresponding to the Bloom filter whose flow splitting ID exists when the flow splitting ID of the sample flow exists in the Bloom filter grouped in the A / B experiment;

[0086] The diversion module 30 is also used to determine the grouping of the sample flow based on a bucketing algorithm when the diversion ID of the sample flow does not exist in the Bloom filter grouped in the A / B experiment, and add the diversion ID to the Bloom filter that determines the grouping for training.

[0087] In some feasible embodiments, the A / B experiment splitting device further includes: an initialization module; the initialization module is used to:

[0088] Divide the A / B experiment groups in advance and determine the traffic distribution ratio between each group;

[0089] A Bloom filter and a bit bucket are initialized for each group of the A / B experiment; wherein all positions of the initialized Bloom filter are empty, and the assignment position of the bit bucket of each group is determined according to the diversion ratio.

[0090] In some feasible embodiments, the A / B experiment splitting device further includes: a synchronization module; the synchronization module is used to:

[0091] When the diversion ratio is adjusted, the Bloom filters of each group in the A / B experiment of different nodes are respectively combined through the distributed component.

[0092] In some feasible embodiments, the determination module 20 is specifically used to:

[0093] The diversion ID is hashed multiple times to obtain a hash result;

[0094] The hash results are mapped to different positions on the Bloom filters grouped in the A / B experiment, and it is determined whether all the positions are assigned values.

[0095] In some feasible embodiments, when the flow distribution module 30 is used to determine that the sample flow is distributed into a group corresponding to the Bloom filter whose flow distribution ID exists, it is specifically used to:

[0096] When the ID of the sample flow exists in Bloom filters of two or more groups in the A / B experiment, it is determined that the sample flow is divided into a group corresponding to any Bloom filter whose diversion ID exists.

[0097] In some feasible embodiments, when the flow distribution module 30 is used to determine the grouping of the sample flow based on the bucketing algorithm, it is specifically used to:

[0098] After hashing the diversion ID, a modulo operation is performed according to the initialization size of the bit bucket grouped in the A / B experiment to obtain a modulo operation result;

[0099] A bit bucket having a value assigned at a position corresponding to the modulo operation result is determined, and a group corresponding to the bit bucket is determined as the group of the sample traffic.

[0100] In some feasible embodiments, when the traffic distribution module 30 is used to add the traffic distribution ID to the Bloom filter of the determined group for training, it is specifically used to:

[0101] The diversion ID is hashed multiple times to obtain a hash result;

[0102] The hash result is mapped to different positions on the Bloom filter that determines the grouping, and values ​​are assigned to the positions.

[0103] The contents of the method embodiments of the present invention are all applicable to the embodiments of the present device. The functions specifically implemented by the embodiments of the present device are the same as those of the above-mentioned method embodiments. Please refer to the description in the method embodiments for details, which will not be repeated here. The embodiments of the present device can achieve: by setting a Bloom filter and a bucket for each group in the A / B experiment, each Bloom filter can quickly and reliably complete the determination of the sample flow that has entered the group, thereby meeting the performance requirements of the A / B experiment SDK on the server side. And because the Bloom filter occupies little memory, the processing efficiency is very high, which provides strong technical support to meet the needs of ensuring that the server faces high concurrency, low latency, and high availability, and enables the A / B experiment SDK on the server side to ensure that the groups always remain consistent.

[0104] In addition, an embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the A / B experimental diversion method described in any of the above embodiments is implemented. Wherein, the computer-readable storage medium includes but is not limited to any type of disk (including floppy disk, hard disk, optical disk, CD-ROM, and magneto-optical disk), ROM (Read-Only Memory), RAM (Random Access Memory), EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), flash memory, magnetic card or optical card. That is, the storage device includes any medium that can store or transmit information in a readable form by a device (for example, a computer, a mobile phone), which can be a read-only memory, a disk or an optical disk, etc.

[0105] The contents of the method embodiments of the present invention are all applicable to the present storage medium embodiments. The functions specifically implemented by the present storage medium embodiments are the same as those of the above-mentioned method embodiments. Please refer to the description in the method embodiments for details, which will not be repeated here. The present storage medium embodiment can achieve: by setting a Bloom filter and a bucket for each group in the A / B experiment, each Bloom filter can quickly and reliably complete the determination of the sample flow that has entered the group, thereby meeting the performance requirements of the server-side A / B experiment SDK. And because the Bloom filter occupies little memory, the processing efficiency is very high, which provides strong technical support to meet the needs of ensuring that the server faces high concurrency, low latency, and high availability, and enables the server-side A / B experiment SDK to ensure that the groupings always remain consistent.

[0106] In addition, an embodiment of the present invention further provides a computer device, and the computer device described in this embodiment may be a server, a personal computer, a network device, and other devices. Specifically, the computer device is the server. The computer device includes: one or more processors, a memory, and one or more computer programs, wherein the one or more computer programs are stored in the memory and configured to be executed by the one or more processors, and the one or more computer programs are configured to execute the A / B experiment diversion method described in any of the above embodiments.

[0107] The contents of the method embodiments of the present invention are all applicable to the computer device embodiments. The functions specifically implemented by the computer device embodiments are the same as those of the above-mentioned method embodiments. Please refer to the description in the method embodiments for details, which will not be repeated here. The computer device embodiment can achieve: by setting a Bloom filter and a bucket for each group in the A / B experiment, each Bloom filter can quickly and reliably complete the determination of the sample traffic that has entered the group, meeting the performance requirements of the server-side A / B experiment SDK. And because the Bloom filter occupies little memory, the processing efficiency is very high, which provides strong technical support to meet the needs of ensuring that the server faces high concurrency, low latency, and high availability, and enables the server-side A / B experiment SDK to ensure that the groupings always remain consistent.

[0108] In addition, each functional unit in each embodiment of the present invention may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0109] The above descriptions are only partial embodiments of the present invention. It should be pointed out that, for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. An A / B experiment diversion method, characterized in that: Applied to the server, it includes the following steps: Obtaining sample flow to be diverted; Determine whether the diversion ID of the sample traffic exists in the Bloom filter grouped in the A / B experiment; If so, determine that the sample traffic is divided into a group corresponding to the Bloom filter whose diversion ID exists; If not, determine the grouping of the sample traffic based on a bucketing algorithm, and add the diversion ID to the Bloom filter of the determined grouping for training; The determining whether the diversion ID of the sample traffic exists in the Bloom filter grouped in the A / B experiment includes: The diversion ID is hashed multiple times to obtain a hash result; Mapping the hash results to different positions on the Bloom filters grouped in the A / B experiment, and determining whether all the positions are assigned values; The step of determining that the sample flow is divided into a group corresponding to the Bloom filter whose diversion ID exists includes: When the ID of the sample flow exists in Bloom filters of two or more groups in the A / B experiment, it is determined that the sample flow is divided into a group corresponding to any Bloom filter whose diversion ID exists.

2. The A / B experiment diversion method according to claim 1, characterized in that: Also includes: Divide the A / B experiment groups in advance and determine the traffic distribution ratio between each group; A Bloom filter and a bit bucket are initialized for each group of the A / B experiment; wherein all positions of the initialized Bloom filter are empty, and the assignment position of the bit bucket of each group is determined according to the diversion ratio.

3. The A / B experiment diversion method according to claim 2, characterized in that: Also includes: When the diversion ratio is adjusted, the Bloom filters of each group in the A / B experiment of different nodes are respectively combined through the distributed component.

4. The A / B experiment diversion method according to claim 1, characterized in that: The determining the grouping of the sample traffic based on a bucketing algorithm includes: After hashing the diversion ID, a modulo operation is performed according to the initialization size of the bit bucket grouped in the A / B experiment to obtain a modulo operation result; A bit bucket having a value assigned at a position corresponding to the modulo operation result is determined, and a group corresponding to the bit bucket is determined as the group of the sample traffic.

5. The A / B experiment diversion method according to claim 1, characterized in that: The adding the diversion ID to the Bloom filter of the determined group for training includes: The diversion ID is hashed multiple times to obtain a hash result; The hash result is mapped to different positions on the Bloom filter that determines the grouping, and values ​​are assigned to the positions.

6. An A / B experiment splitting device, characterized in that: include: A flow acquisition module, used to acquire sample flow to be diverted; A judgment module is used to judge whether the diversion ID of the sample flow exists in the Bloom filter grouped in the A / B experiment, specifically comprising: hashing the diversion ID multiple times to obtain a hash result; mapping the hash result to different positions on the Bloom filter grouped in the A / B experiment, and judging whether the positions are all assigned values; The flow splitting module is used to determine that when the flow splitting ID of the sample flow exists in the Bloom filter grouped in the A / B experiment, the sample flow is split into a group corresponding to the Bloom filter whose flow splitting ID exists, specifically including: when the ID of the sample flow exists in two or more Bloom filters grouped in the A / B experiment, determine that the sample flow is split into a group corresponding to any Bloom filter whose flow splitting ID exists; The diversion module is also used to determine the grouping of the sample flow based on a bucketing algorithm when the diversion ID of the sample flow does not exist in the Bloom filter grouped in the A / B experiment, and add the diversion ID to the Bloom filter that determines the grouping for training.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the A / B experiment diversion method according to any one of claims 1 to 5 is implemented.

8. A computer device, characterized in that: It includes: one or more processors; Memory; One or more computer programs, wherein the one or more computer programs are stored in the memory and configured to be executed by the one or more processors, and the one or more computer programs are configured to: execute the A / B experiment diversion method according to any one of claims 1 to 5.