Method and device for distributing small flow in shunting experiment and electronic equipment

By using multi-layer orthogonal shunt method and hash algorithm to generate hash values ​​in the shunt group, combined with the whitelist mechanism, the problem of uneven shunt of small flows is solved, the reliability and scientificity of experimental results are improved, and the Long-term effective experimental results verification.

CN119052178APending Publication Date: 2024-11-29QIAN JIN NETWORK INFORMATION TECH SHANGHAI LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411084270.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-08
Publication Date
2024-11-29

AI Technical Summary

Technical Problem

Traditional hashing algorithms and random allocation methods cannot ensure uniform distribution of traffic in small traffic scenarios, resulting in uneven shunt flow, affecting the accuracy and reliability of experimental results, and lacking complex shunt management and traffic distribution functions, making it difficult to achieve. Verification of multi-layer orthogonal shunt and long-term effective experimental effects.

Method used

The multi-layer orthogonal shunt method is adopted to determine the experimental group in which the traffic flow is entered by using the shunt group through the series shunt group, the shunt ratio, the shunt identification and the access time identification, and the hash value is generated using the hash algorithm and the sorting algorithm. Ensure uniform distribution of flows across each shunt and experimental groups, and optimize traffic distribution through a whitelisting mechanism.

Benefits of technology

The flow rate is uniformly distributed in each shunt and experimental group, which reduces the phenomenon of shunt uneven shunt, improves the reliability and scientificity of experimental results, avoids mutual interference between different shunts and experiments, and supports long-term effective Experimental effect verification.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119052178A_ABST
    Figure CN119052178A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of big data, in particular to the technical field of internet data service, and particularly relates to a method for distributing small flow in a flow distribution experiment, which comprises the following steps: distributing the flow into a flow distribution layer group or a blank group according to access information of the flow and the flow distribution proportion of the flow distribution layer group and the blank group; according to the access information of the flow distributed to the flow distribution layer group, the first flow distribution layer identifier, the access time identifier and the flow distribution proportion of each experiment group in the first flow distribution layer, determining the experiment group into which the flow is about to enter; and according to the access information of the flow passing through the first shunting layer, the second shunting layer identifier, the access time identifier and the shunting proportion of each experimental group in the second shunting layer, determining the experimental group into which the flow is about to enter. According to the method, the entered experimental group is determined by jointly referring to the shunting proportion, the shunting layer identifier and the access time identifier, and uniform distribution of the flow is ensured. Especially in a small-flow scene, the phenomenon of non-uniform shunting is reduced, and the reliability of an experimental result is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of big data technology, specifically to the field of Internet data service technology, and in particular to a method, device, electronic device and storage medium for allocating small traffic in a diversion experiment. Background Art

[0002] A traffic diversion experiment is a test of the effects of different solutions by distributing user traffic to different experimental and control groups during product or service development. Traffic diversion experiments are usually implemented through a diversion layer, each of which is responsible for performing specific diversion tasks to ensure that user traffic is distributed according to a predetermined strategy. The reasonable design and management of the diversion layer is crucial to achieving the scientificity and accuracy of the experiment.

[0003] When using traditional methods to distribute small traffic, it is easy to cause uneven distribution between the distribution layer / experiments, especially when the traffic is small in the early stage of the experiment. This is because traditional hash algorithms and random distribution methods cannot guarantee uniform distribution of traffic in small traffic scenarios. According to the law of large numbers, random distribution can only be uniform when the sample size is large enough, and in the case of small traffic, random distribution often leads to uneven grouping. This uneven distribution will bring uncertainty to the interpretation of experimental results and affect the accuracy and reliability of experimental results.

[0004] In addition, traditional methods lack complex diversion layer management and traffic distribution functions, and cannot effectively implement multi-layer orthogonal diversion, which easily leads to mutual interference between different diversion layers and experiments. Personalized traffic control in business needs usually requires the caller to develop separately, resulting in repeated development and waste of resources. At the same time, due to the lack of cross-layer management capabilities, experimental results can often only be compared within the experiment, making it difficult to achieve unified comparison and verify the long-term effectiveness of experimental results. Summary of the invention

[0005] In response to the technical problems existing in the prior art, the present application proposes a method for distributing small flows in a diversion experiment, wherein the diversion experiment includes at least: a diversion layer group and a blank group, wherein the diversion layer group includes at least a first diversion layer and a second diversion layer connected in series, and the method includes: distributing the flow into a diversion layer group or a blank group according to the access information of the flow and the diversion ratio of the diversion layer group and the blank group, wherein the diversion layer group and the blank group are mutually exclusive; determining the experimental group that the flow is to enter according to the access information of the flow allocated to the diversion layer group, the first diversion layer identifier, the access time identifier, and the diversion ratio of each experimental group in the first diversion layer; determining the experimental group that the flow is to enter according to the access information of the flow passing through the first diversion layer, the second diversion layer identifier, the access time identifier, and the diversion ratio of each experimental group in the second diversion layer.

[0006] The allocation method as described above further includes: generating a first hash value through a hash algorithm based on the access information allocated to the diversion layer group traffic, the first diversion layer identifier and the access time identifier; sorting multiple first hash values ​​according to preset rules; and according to the diversion ratio of each experimental group in the first diversion layer, the multiple first hash values ​​are sequentially entered into the buckets of the corresponding experimental groups.

[0007] The allocation method as described above further includes: sorting the multiple first hash values ​​using a sorting algorithm, the sorting algorithm including: selecting a target numerical value from the first hash values; obtaining a first random hash value and a second random hash value by comparing the numerical value in the first hash value with the target numerical value, the numerical values ​​in the first random hash value are all less than or equal to the target numerical value, and the numerical values ​​in the second random hash value are all greater than the target numerical value; merging the first random hash value and the second random hash value into a random hash value; and sorting the random hash values ​​according to the numerical value.

[0008] According to the allocation method described above, the access time identifier includes at least: a current timestamp and an update period; if the current timestamp is within a preset update period, the time identifier is not updated, and a first hash value is generated using the time identifier; if the current timestamp is not within the preset update period, the time identifier is updated, and a first hash value is generated using the updated time identifier.

[0009] According to the allocation method described above, the update period is 3-10 days.

[0010] The allocation method as described above, before determining whether the traffic enters the diversion layer group or the blank group based on the access information of the traffic and the diversion ratio of the diversion layer group and the blank group, further includes: judging whether the traffic is in the whitelist cache based on the access information of the traffic; if the traffic is in the whitelist cache, allocating the traffic to the whitelist group; if the traffic is not in the whitelist, judging whether the traffic hits the whitelist; if the traffic hits the whitelist, allocating the traffic to the whitelist group; if the traffic does not hit the whitelist, determining whether the traffic enters the diversion layer group or the blank group based on the access information of the traffic and the diversion ratio of the diversion layer group and the blank group.

[0011] The allocation method as described above further includes: generating a third hash value according to the access information allocated to the blank group traffic and the blank group identifier; wherein the validity period of the third hash value is 1-3 months.

[0012] According to another aspect of the present application, a distribution device for small flows in a diversion experiment is proposed, comprising: a first distribution module, used to distribute the flow into a diversion layer group or a blank group according to the access information of the flow and the diversion ratio of the diversion layer group and the blank group, wherein the diversion layer group and the blank group are mutually exclusive; a second distribution module, used to determine the experimental group that the flow is to enter according to the access information of the flow allocated to the diversion layer group, the first diversion layer identifier, the access time identifier and the diversion ratio of each experimental group in the first diversion layer; a third distribution module, used to determine the experimental group that the flow is to enter according to the access information of the flow passing through the first diversion layer, the second diversion layer identifier, the access time identifier and the diversion ratio of each experimental group in the second diversion layer.

[0013] According to another aspect of the present application, an electronic device is provided, comprising: a processor and a memory storing computer program instructions; when the electronic device executes the computer program instructions, the method described above is implemented.

[0014] According to another aspect of the present application, a computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the method described above is implemented.

[0015] This application adopts a multi-layer orthogonal diversion method, that is, when a diversion layer group includes multiple diversion layers, before each layer of diversion, the diversion ratio, diversion layer identification and access time identification are jointly referenced to determine the experimental group to enter, ensuring that the traffic is evenly distributed in each diversion layer and experimental group, which helps to accurately analyze the effect of each experimental group and avoid mutual interference between different diversion layers and experiments. Especially in small traffic scenarios, it reduces the phenomenon of uneven diversion and improves the reliability of experimental results. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The preferred embodiments of the present invention will be further described in detail below with reference to the accompanying drawings, wherein:

[0017] Figure 1 is a flow chart of a method for allocating small flows in a diversion experiment according to an embodiment of the present application;

[0018] Figure 2 yes Figure 1 Flowchart of the method for determining that the flow will enter the experimental group in step S120;

[0019] Figure 3 is a flow chart of a method for determining whether traffic is in a whitelist according to an embodiment of the present application;

[0020] Figure 4 is a schematic diagram of the structure of a diversion experiment service module according to an embodiment of the present application;

[0021] Figure 5 is a schematic structural diagram of a shunt layer according to an embodiment of the present application;

[0022] Figure 6 is a schematic structural diagram of a flow dividing layer group according to an embodiment of the present application;

[0023] Figure 7 is a schematic diagram of an orthogonal bucketing process according to an embodiment of the present application;

[0024] Figure 8 This is a schematic diagram of the structure of the scattering strategy of the diversion layer according to an embodiment of the present application;

[0025] Fig. 9 is a schematic diagram of the structure of a distribution device for small flow in a diversion experiment according to an embodiment of the present application; and

[0026] Fig.10 A schematic diagram showing the hardware structure of an electronic device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0028] In the following detailed description, reference may be made to the various specification drawings that are part of the present application and are used to illustrate specific embodiments of the present application. In the accompanying drawings, similar reference numerals describe substantially similar components in different figures. The various specific embodiments of the present application are described below in sufficient detail so that a person of ordinary skill in the art with relevant knowledge and skills in the art can implement the technical solutions of the present application. It should be understood that other embodiments may also be used or structural, logical or electrical changes may be made to the embodiments of the present application.

[0029] Figure 1 FIG. 1 is a flow chart of a method for distributing small flows in a diversion experiment according to an embodiment of the present application. Figure 1 The diversion experiment includes: a diversion layer group and a blank group, the diversion layer group includes at least a first diversion layer and a second diversion layer connected in series, and the method includes:

[0030] Step S110, allocating the traffic into the shunt layer group or the blank group according to the traffic access information and the shunt ratio of the shunt layer group and the blank group, wherein the shunt layer group and the blank group are mutually exclusive;

[0031] Step S120, determining the experimental group that the traffic will enter according to the access information of the traffic allocated to the diversion layer group, the first diversion layer identifier, the access time identifier, and the diversion ratio of each experimental group in the first diversion layer;

[0032] Step S130, determining the experimental group that the traffic will enter according to the access information of the traffic passing through the first diversion layer, the second diversion layer identifier, the access time identifier, and the diversion ratio of each experimental group in the second diversion layer.

[0033] The shunting experiment in the present application includes: one or more of a shunting layer, a shunting layer group and a blank group. The shunting layer group includes multiple shunting sub-layers (such as the first shunting layer and the second shunting layer above). The shunting layer group and the blank group are mutually exclusive, that is, a flow either enters the shunting layer group or the blank group, and does not enter both at the same time, thereby ensuring the reasonable distribution of the flow between the experiment and the non-experiment. As a control group without any experimental intervention, the blank group can provide a benchmark to help researchers evaluate the differences between the experimental group and the control group. By comparing the effects of the blank group and the experimental group, the true impact of the experiment can be determined more accurately. The blank group can be used as a long-term control group to help verify the sustainability of the experimental effect. By comparing the performance of the blank group and the experimental group in different time periods, it can be evaluated whether the experimental intervention has long-term effectiveness.

[0034] According to an embodiment of the present application, a third hash value is generated based on the access information and blank group identifier assigned to the blank group traffic; wherein the validity period of the third hash value is 1-3 months. Compared with the traffic in the diversion layer, the traffic in the blank group remains unchanged for a long time, so that each experiment in each diversion layer can be compared with the blank group; and when a new round of comparison is performed, it is not necessary to switch to a new experiment or add a new blank group comparison, which can also improve the efficiency of the experimental implementation.

[0035] This application adopts a multi-layer orthogonal diversion method, that is, when a diversion layer group includes multiple diversion sub-layers, before each layer of diversion, the diversion ratio, diversion layer identifier and access time identifier are jointly referenced to determine the experimental group to enter, ensuring that the traffic is evenly distributed in each diversion layer and experimental group, which helps to accurately analyze the effect of each experimental group and avoid mutual interference between different diversion layers and experiments. Especially in small traffic scenarios, it reduces the phenomenon of uneven diversion and improves the reliability of experimental results.

[0036] Figure 2 yes Figure 1 Flow chart of the method for determining that the flow will enter the experimental group in step S120. Figure 2 As shown, the method includes:

[0037] Step S121, generating a first hash value by a hash algorithm according to the access information allocated to the flow of the shunt layer group, the first shunt layer identifier and the access time identifier;

[0038] Step S122, sorting the multiple first hash values ​​according to a preset rule; and

[0039] Step S123, according to the diversion ratio of each experimental group in the first diversion layer, the multiple first hash values ​​are sequentially entered into the buckets of the corresponding experimental groups.

[0040] The access information and identification information are converted into a hash value using a hash algorithm. The hash value is randomly generated by the hash algorithm and will be used for subsequent traffic distribution. The hash values ​​are sorted according to the preset rules, and then the hash values ​​are distributed to different buckets. The more buckets there are, the higher the accuracy (for example, if the number of buckets is configured as 100, the accuracy of the diversion ratio control is 1%). Each bucket has a pre-set mapping relationship with the experimental group. According to the mapping relationship, the bucket results are mapped to the experimental group. The choice of hash algorithm is crucial to ensure that the generated hash value has good uniform distribution characteristics.

[0041] The preset rules of this application can be a simple sorting by numerical value, or a sorting algorithm. The sorting algorithm includes:

[0042] Select a target value from the first hash value;

[0043] A first random hash value and a second random hash value are obtained by comparing the value in the first hash value with the target value, wherein the values ​​in the first random hash value are all less than or equal to the target value, and the values ​​in the second random hash value are all greater than the target value;

[0044] Combining the first random hash value and the second random hash value into a random hash value;

[0045] Sort the random hash values ​​by numerical size.

[0046] Sorting with the above algorithm can further improve the randomness of the hash value, making the distribution result more uniform. By sorting, the distribution of the hash value can be guaranteed to be more uniform, thereby improving the uniformity of the experimental group distribution.

[0047] The use of hash algorithms and preset rules for diversion ensures that the diversion process has good repeatability and consistency, making the experimental results more reliable. Through precise diversion ratio control and hash value sorting, it ensures that the traffic distribution of each experimental group meets expectations, improving the scientificity and accuracy of the experimental results.

[0048] According to one embodiment of the present application, the hash algorithm includes: one or more of mur3_128, mur3_128(100) and sha_256. MurMurHash3 is a high-performance, non-encrypted hash function, widely used in scenarios such as hash tables and distributed systems that require fast hash calculations. mur3_128 is its 128-bit version with higher hash uniformity and speed. mur3_128(100) is a version of mur3_128 that increases its randomness and security by applying the hash function multiple times (such as iterating 100 times) or other adjustments. SHA-256 (Secure Hash Algorithm 256-bit) is a cryptographic hash function that is widely used in scenarios with high data integrity verification and security requirements.

[0049] In another embodiment, in order to further improve the uniformity of hash values ​​and the fairness of distribution, the above-mentioned multiple hash algorithms can be used in combination. Specifically,

[0050] Generate a first sub-hash value by a first hash algorithm according to the access information assigned to the flow group of the shunt layer, the first shunt layer identifier and the access time identifier;

[0051] generating a second sub-hash value by a second hash algorithm according to the access information assigned to the traffic of the shunt layer group, the first shunt layer identifier, and the access time identifier;

[0052] The first sub-hash value and the second sub-hash value are combined into a first hash value.

[0053] According to an embodiment of the present application, the access time identifier includes at least: a current timestamp and an update period;

[0054] If the current timestamp is within the preset update period, the time identifier is not updated, and the first hash value is generated using the time identifier;

[0055] If the current timestamp is not within the preset update period, the time identifier is updated, and the first hash value is generated using the updated time identifier, wherein the update period is 3-10 days.

[0056] By setting the access time mark and the preset update cycle, the hash value is dynamically updated over time, which improves the uniformity and fairness of traffic distribution. Regularly updating the time mark can prevent the distribution lag effect caused by the long-term unchanged time mark, ensuring the real-time and accuracy of the experimental data. The preset update cycle can be flexibly adjusted according to specific needs (3-10 days), adapting to different scenarios and needs, and improving the flexibility of the system.

[0057] Those skilled in the art should understand that Figure 2The traffic distribution method and the method for generating hash values ​​in can both be applied in step S130 and are not limited here.

[0058] Figure 3 1 is a flow chart of a method for determining whether traffic is in a whitelist according to an embodiment of the present application. Figure 3 As shown, the method includes:

[0059] Step S310, judging whether the traffic is in the whitelist cache according to the access information of the traffic;

[0060] Step S320: if the traffic is in the whitelist cache, the traffic is assigned to the whitelist group; then step S360 is executed to refresh the cache;

[0061] Step S330: if the traffic is not in the whitelist, determine whether the traffic hits the whitelist;

[0062] Step S340: if the traffic hits the whitelist, the traffic is assigned to the whitelist group; then step S370 is executed to write the traffic into the cache.

[0063] Step S350: If the traffic does not hit the whitelist, execute Figure 1 In step S110, the flow is determined to enter the diversion layer group or the blank group according to the access information of the flow and the diversion ratio of the diversion layer group and the blank group.

[0064] A unique identifier is generated based on the traffic access information such as source, user ID, IP address, etc., to determine whether the traffic is in the whitelist cache. This application configures the whitelist with the highest priority. By giving priority to whitelist traffic, system resource allocation can be optimized, ensuring that critical traffic is given priority and improving overall system performance. The rules and conditions of the whitelist can be flexibly configured and adjusted to suit different business needs and scenarios, enhancing the flexibility and adaptability of the system.

[0065] In one embodiment, once the traffic is assigned to the whitelist group, it remains unchanged for a month or a quarter and does not enter any experiment, meeting the need for some algorithms to compare the stability of experimental results. The rule for assigning to the whitelist can be to identify whether it is malicious traffic, use the filtering and verification mechanism of the whitelist to detect and block malicious traffic, and ensure the security and stability of the system.

[0066] The above describes the implementation methods and advantages of the embodiments of the present application through multiple embodiments. The following describes in detail the distribution process of small flow in the flow diversion experiment of the embodiments of the present application in combination with specific examples.

[0067] Figure 4FIG. 1 is a schematic diagram of the structure of a diversion experiment service module according to an embodiment of the present application. Figure 4 As shown, the diversion experiment service module 400 includes: a diversion layer group structure management module 410, a diversion layer series configuration module 420, a blank group and diversion layer diversion ratio configuration module 430, a whitelist configuration module 440, a diversion experiment orthogonal bucket diversion module 450, a blank group module 460 and an experimental grouping and blank group periodic dispersion module 470.

[0068] Among them, the diversion layer group structure management module 410 is responsible for managing and configuring the organizational structure of the diversion layer, defining the diversion levels of the experiment and their relationships. The diversion layer group structure management module 410 can be configured to manage the number of diversion layers and the series relationship. The diversion layer may include a diversion layer group and a blank group. The diversion layer group includes multiple diversion sub-layers and / or multiple diversion sub-layer groups. The diversion layer structure management module 410 can flexibly adjust the architecture of the diversion layer to meet different experimental requirements. The present application provides a unified and standard diversion layer management tool to avoid repeated development by each caller. In addition, the present application will provide a complete configuration log report. By parsing the log, it will be easier to troubleshoot diversion problems.

[0069] The flow splitting layer series configuration module 420 is used to set the mapping relationship between the experimental flow splitting layers to ensure the correct transmission and association of the experimental flow. The flow splitting ratio configuration module 430 of the blank group and the flow splitting layer configures the flow distribution ratio of the blank group and the flow splitting layer in the experiment to ensure the scientificity and rationality of each group of data. Multiple experimental groups and control groups can be set in the flow splitting layer.

[0070] Figure 5 FIG. 1 is a schematic diagram of the structure of a flow separation layer according to an embodiment of the present application. Figure 5 As shown, according to the experimental needs, a structure in which the shunt layer 1, the shunt layer group 1, the shunt layer group 2 and the shunt layer 2 are connected in series is designed, wherein the shunt layer group 1 includes the shunt sub-layer 1 and the shunt sub-layer 2; the shunt layer group 2 includes: the shunt sub-layer 3, the shunt sub-layer 4 and the shunt sub-layer 5. Figure 5 It can be seen that the present application can flexibly adjust the series relationship between multiple diversion layers and the diversion ratio of each diversion layer, and can adapt to the diversion requirements of various experiments.

[0071] The whitelist configuration module 440 is used to manage and configure the whitelist to ensure special treatment of specific users or groups in the experiment. For example, once a certain flow is determined to be in the whitelist configuration, it will no longer participate in the diversion work of other diversion layers to avoid interference with the experimental data.

[0072] The orthogonal bucket splitting module 450 of the flow splitting experiment performs orthogonal bucket splitting on the flow entering the flow splitting layer, and then allocates the flow into one or more experimental groups of the flow splitting layer according to the splitting ratio. Figure 6Schematic diagram of the structure of a flow dividing layer group according to an embodiment of the present application. Figure 6 As shown, the diversion layer group includes diversion layer A, diversion layer B and diversion layer C connected in series. Experiment a is configured in diversion layer A, and experiment a is configured with experimental group 1, experimental group 2 and control group, and the diversion ratio of the three is 11%. Experiment b is configured in diversion layer B, and experiment b is configured with experimental group 1, experimental group 2 and control group, and the diversion ratio of the three is 11%. Experiment c1, experiment c2 and experiment c3 are configured in diversion layer C, and each experiment is configured with experimental group 1, experimental group 2 and control group, and the diversion ratio of the three is 4%. When the flow flows from diversion layer A to diversion layer B and from diversion layer B to diversion layer C, orthogonal bucketing operation is required to evenly and randomly divert each experimental group or control group according to the diversion ratio (for example, 1:1:1).

[0073] Figure 7 FIG. 1 is a schematic diagram of an orthogonal bucketing process according to an embodiment of the present application. Figure 7 As shown, taking the flow from split layer A to split layer B as an example, the steps of orthogonal bucketing are explained as follows:

[0074] (1) According to the identification of the shunt layer B (such as Figure 6 "Layer_id: B" in ), traffic access information (such as Figure 7 The number "0" in the example) generates a hash value using a hash algorithm;

[0075] (2) sorting the multiple hash values ​​according to a preset rule (e.g., sorting by numerical value);

[0076] (3) According to the diversion ratio of experimental group 1: experimental group 2: control group in diversion layer B (for example, the diversion ratio is 1:1:1), the sorted hash values ​​are sequentially placed into the buckets of the corresponding experimental groups. This allows the traffic in diversion layer A to be randomly and evenly distributed to the buckets of diversion layer B.

[0077] The shunt layer calls:

[0078] curl -X POST\

[0079] http: / / 192.168.4.131:31663 / opt / layerShunt\

[0080] -H'Content-Type:application / json'\

[0081] -H'Postman-Token:600fc81a-353e-4470-a8b3-3190d0c52cae'\

[0082] -H'cache-control:no-cache'\

[0083] -d'{

[0084] "layerId":"layerId_s0730_0003",

[0085] "shuntId":"20240725"

[0086] }'

[0087] Response result:

[0088] {

[0089] "code":"0",

[0090] "msg":"Operation successful",

[0091] "data":{

[0092] "experimentId":"testId_0730s_02_0003",

[0093] "groupId":"base",

[0094] "shuntId":"20240725",

[0095] "value":"{base}"

[0096] }

[0097] }

[0098] Referring to the above code, "layerId_s0730_0003" is the traffic layer ID, and "20240725" is the user ID, i.e., the access information of the traffic. From the above code, it can be seen that through this API call, the user ID 20240725 is assigned to the traffic layer layerId_s0730_0003, and the experiment hit by the user and the experimental grouping information are returned.

[0099] For example, the search and push algorithm experiment is generally divided into three nodes: coarse sorting, fine sorting, and recall, and is serial. The corresponding experiments are diversion layer Layer_id: A, diversion layer Layer_id: B, and diversion layer Layer_id: C. Each experiment can also set sub-experiments according to the modules it acts on. For example, three sub-experiments named: c1, name: c2, and name: c3 are designed in the diversion layer Layer_id: C. Through the orthogonal bucketing processing of this application, it is ensured that the diversion effect of the coarse sorting layer is uniform and randomly distributed to the fine sorting layer, and the fine sorting layer is distributed to the recall layer by analogy. The experiments of the three diversion layers can be observed separately to objectively evaluate the experimental effect of each layer, and the search and push server calls the AB service to return the diversion results of the three layers at a time. Generally, the experimental group is the new version of the algorithm, and the control group is the old stable version of the algorithm. The grouping / algorithm strategy version of each layer is output in turn, and all users pass through three layers, which are orthogonal and independent of each other when evaluating the experiments of each layer.

[0100] Hash algorithms include: one or more of mur3_128, mur3_128(100) and sha_256. In order to verify which algorithm is suitable for this solution, this application designed an effect verification test, using mur3_128, mur3_128(100) and sha_256 hash algorithms to calculate hash values, and statistically analyze the time consumption and discrete rate. The specific results are shown in Table 1 below:

[0101] Table 1

[0102]

[0103] As shown in Table 1 above, three hash algorithms (sha_256, mur3_128, mur3_128(100)) are compared under different data volumes (1w, 10w, 100w). As the amount of data increases, the time consumption of all algorithms increases. The mur3_128(100) algorithm has the shortest time consumption under each data volume, indicating that it has the best performance. The time consumption of the sha_256 algorithm is the longest under all data volumes, indicating that it has the worst performance. Under 1w data volume (100 buckets), the discrete rates of mur3_128 and mur3_128(100) algorithms are slightly higher than sha_256; under 10w and 100w data volumes (1000 and 10000 buckets), the discrete rates of all algorithms are very low and perform well. The repetition of all algorithms under all data volumes is 0, indicating that no hash collision occurs in all algorithms in these tests. Therefore, the mur3_128(100) algorithm performs best in terms of time consumption, while the sha_256 algorithm takes the longest time at all data levels. In terms of discreteness and repetition, the performance of the three algorithms is not much different.

[0104] The blank group module 460 is used to form a blank group for comparative experiments with the experimental group in the same flow distribution layer. Before allocating the traffic to the blank group, it is also necessary to generate a hash value based on the access information of the traffic and the blank group identifier. The hash value is valid for up to 3 months, which keeps the traffic in the blank group unchanged for a long time, and forms a comparison with the experimental results in the flow distribution.

[0105] The module 470 for periodically disrupting the experimental groups and the blank groups is used to periodically disrupt the order of the experimental groups and the blank groups to eliminate deviations and interference factors in the experiment.

[0106] Figure 8 FIG. 1 is a schematic diagram of a sparse strategy structure of a flow splitting layer according to an embodiment of the present application. Figure 8 As shown, the flow splitting layer includes: flow splitting layer layer1, flow splitting layer layer2 and blank group, wherein the flow splitting ratio of the blank group is 5%, and the flow splitting ratios of flow splitting layer layer1 and flow splitting layer layer2 are both 95%. The scattering periods of flow splitting layer layer1, flow splitting layer layer2 and blank group can be configured separately, such as Figure 8 As shown, the dispersion period of the blank group is M days, the dispersion period of the diversion layer layer1 is 7 days, and the dispersion period of the diversion layer layer2 is 3 days and 4 days. The dispersion period of the blank group is generally greater than 7 days, which can be 30-90 days. The flow in the blank group is kept stable, so that each experiment of each diversion layer can be compared with the blank group, and each dispersion can be a new round of comparison, so that there is no need to switch to a new experiment, which not only increases the comparison of the blank group but also improves the efficiency of the experimental implementation.

[0107] The method of configuring the scattering period may be to add a time stamp when calculating the hash value using the hash algorithm. If the current timestamp is within the update period, i.e., the scattering period, the time stamp is not updated, and the hash value is generated using the time stamp; otherwise, the time stamp is updated to change the hash value, so that the traffic is regrouped.

[0108] This scheme is uniformly orthogonalized

[0109] Two experiments were set up each time through two validations. The experiment IDs were 12 and 14, and 24 and 25.

[0110] First verification:

[0111] Table 2

[0112]

[0113] From Table 2 above, we can see that 50.01% and 49.97% of the traffic of the control group in the experiment id 12 is distributed in the experimental group and the control group in the experiment id 14 respectively;

[0114] 49.90% and 50.08% of the traffic of the experimental group with experiment id 12 is distributed in the experimental group with experiment id 14 and the control group respectively.

[0115] Second verification:

[0116] Table 3

[0117]

[0118]

[0119] From Table 3 above, we can see that 50.16% and 49.47% of the traffic of the control group in the experiment id 24 is distributed in the experimental group and the control group in the experiment id 25 respectively;

[0120] 49.63% and 50.02% of the traffic of the experimental group with experiment id 24 is distributed in the experimental group with experiment id 25 and the control group respectively.

[0121] The orthogonality rate error of the first verification and the second verification is within 1% (100 buckets correspond to 1% accuracy), which meets the design requirements.

[0122] Corresponding to the method embodiment of the present application, the present application also provides a distribution device for small flow in a diversion experiment, such as Fig. 9 As shown, the distribution device 100 for small flow in the diversion experiment includes: a first distribution module 110, which is used to distribute the flow into the diversion layer group or the blank group according to the access information of the flow and the diversion ratio of the diversion layer group and the blank group, and the diversion layer group and the blank group are mutually exclusive;

[0123] The second allocation module 120 is used to determine the experimental group that the traffic will enter according to the access information of the traffic allocated to the diversion layer group, the first diversion layer identifier, the access time identifier, and the diversion ratio of each experimental group in the first diversion layer;

[0124] The third allocation module 130 is used to determine the experimental group that the traffic will enter according to the access information of the traffic passing through the first diversion layer, the second diversion layer identifier, the access time identifier and the diversion ratio of each experimental group in the second diversion layer.

[0125] The electronic device in the embodiment of the present application may be a user terminal device, a server, other computing devices, or a cloud server. Fig.10 A schematic diagram of the hardware structure of an electronic device according to an embodiment of the present application is shown. The electronic device may include a processor 601 and a memory 602 storing computer program instructions. When the processor 601 executes the computer program instructions, the process or function of any of the above-mentioned embodiments of the method is implemented.

[0126] Specifically, the processor 601 may include a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application. The memory 602 may include a large-capacity memory for data or instructions. For example, the memory 602 may be at least one of the following: a hard disk drive (HDD), a read-only memory (ROM), a random access memory (RAM), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a tape, a universal serial bus (USB) drive or other physical / tangible memory storage device. For another example, the memory 602 may include a removable or non-removable (or fixed) medium. For another example, the memory 602 may be inside or outside the integrated gateway disaster recovery device. The memory 602 may be a non-volatile solid-state memory. In other words, the memory 602 usually includes a tangible (non-transitory) computer-readable storage medium (such as a memory device) encoded with computer-executable instructions, and when the software is executed (such as executed by one or more processors), the operation described in the method of the embodiment of the present application can be performed. The processor 601 implements the process or function of any method in the above embodiments by reading and executing the computer program instructions stored in the memory 602.

[0127] In one example, Fig.10 The electronic device shown may also include a communication interface 603 and a bus 610. Among them, the processor 601, the memory 602, and the communication interface 603 are connected through the bus 610 and complete the communication between each other. The communication interface 603 is mainly used to realize the communication between each module, device, unit and / or device in the embodiment of the present application. The bus 610 includes hardware, software or both, and can couple the components of the online data traffic billing device to each other. For example, the bus may include at least one of the following: an accelerated graphics port (AGP) or other graphics bus, an enhanced industrial standard architecture (EISA) bus, a front-end bus (FSB), a hypertransport (HT) interconnect, an industrial standard architecture (ISA) bus, an infinite bandwidth interconnect, a low pin count (LPC) bus, a memory bus, a micro channel architecture (MCA) bus, a peripheral component interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a serial advanced technology attachment (SATA) bus, a video electronics standard association local (VLB) bus or other suitable buses. The bus 610 may include one or more buses. Although the embodiments of the present application describe or illustrate a specific bus, the embodiments of the present application may consider any suitable bus or interconnection method.

[0128] In combination with the method in the above embodiments, an embodiment of the present application also provides a computer-readable storage medium, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the process or function of any method in the above embodiments is implemented.

[0129] In addition, an embodiment of the present application further provides a computer program product, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the process or function of any one of the methods in the above embodiments is implemented.

[0130] The above embodiments are only used to illustrate the present invention, but not to limit the present invention. Ordinary technicians in the relevant technical field can make various changes and modifications without departing from the scope of the present invention. Therefore, all equivalent technical solutions should also fall within the scope of the present invention.

Claims

1. A method for distributing small flows in a diversion experiment, characterized in that: The diversion experiment includes at least a diversion layer group and a blank group, and the diversion layer group includes at least a first diversion layer and a second diversion layer connected in series. The method includes: Allocating the traffic into the shunt layer group or the blank group according to the traffic access information and the shunt ratio of the shunt layer group and the blank group, wherein the shunt layer group and the blank group are mutually exclusive; Determine the experimental group that the traffic will enter according to the access information of the traffic allocated to the diversion layer group, the first diversion layer identifier, the access time identifier, and the diversion ratio of each experimental group in the first diversion layer; The experimental group that the traffic will enter is determined according to the access information of the traffic passing through the first diversion layer, the second diversion layer identifier, the access time identifier, and the diversion ratio of each experimental group in the second diversion layer.

2. The distribution method according to claim 1, characterized in that: Further including: Generate a first hash value by a hash algorithm according to the access information assigned to the flow group of the shunt layer, the first shunt layer identifier and the access time identifier; Sorting the multiple first hash values ​​according to a preset rule; as well as According to the diversion ratio of each experimental group in the first diversion layer, the multiple first hash values ​​are sequentially entered into the buckets of the corresponding experimental groups; The hash algorithm includes one or more of mur3_128, mur3_128(100) and sha_256.

3. The distribution method according to claim 2, characterized in that: Further including: Sorting the plurality of first hash values ​​using a sorting algorithm, the sorting algorithm comprising: Select a target value from the first hash value; A first random hash value and a second random hash value are obtained by comparing the values ​​in the first random hash value with the target value, wherein the values ​​in the first random hash value are all less than or equal to the target value, and the values ​​in the second random hash value are all greater than the target value; Combining the first random hash value and the second random hash value into a random hash value; Sort the random hash values ​​by numerical size.

4. The distribution method according to claim 2, characterized in that: The access time identifier includes at least: the current date and update period; If the current date is within the preset update period, the time identifier is not updated, and the first hash value is generated using the time identifier; If the current date is not within the preset update period, the time identifier is updated, and the first hash value is generated using the updated time identifier.

5. The distribution method according to claim 4, characterized in that: The update cycle is 3-10 days.

6. The distribution method according to claim 1, characterized in that: Before determining whether the traffic enters the shunt layer group or the blank group according to the access information of the traffic and the shunt ratio of the shunt layer group and the blank group, the method further includes: Determine whether the traffic is in the whitelist cache based on the traffic access information; If the traffic is in the whitelist cache, the traffic is assigned to the whitelist group; If the traffic is not in the whitelist, determine whether the traffic hits the whitelist; If the traffic hits the whitelist, the traffic is assigned to the whitelist group; If the traffic does not hit the whitelist, the traffic is determined to enter the diversion layer group or the blank group according to the access information of the traffic and the diversion ratio of the diversion layer group and the blank group.

7. The distribution method according to claim 1, characterized in that: Further including: generating a third hash value according to the access information allocated to the blank group traffic and the blank group identifier; The validity period of the third hash value is 1-3 months.

8. A distribution device for small flow in a diversion experiment, characterized in that: include: A first allocation module is used to allocate the traffic into the shunt layer group or the blank group according to the access information of the traffic and the shunt ratio of the shunt layer group and the blank group, wherein the shunt layer group and the blank group are mutually exclusive; A second allocation module is used to determine the experimental group that the traffic will enter according to the access information of the traffic allocated to the diversion layer group, the first diversion layer identifier, the access time identifier, and the diversion ratio of each experimental group in the first diversion layer; The third allocation module is used to determine the experimental group that the traffic will enter according to the access information of the traffic passing through the first diversion layer, the second diversion layer identifier, the access time identifier and the diversion ratio of each experimental group in the second diversion layer.

9. An electronic device, characterized in that: The electronic device comprises: a processor and a memory storing computer program instructions; when the electronic device executes the computer program instructions, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, and when the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.