A / B experiment method and device based on flow shunting, electronic equipment and medium

Abstract

The invention provides an A / B experiment method and device based on flow shunting, electronic equipment and a medium. The method comprises the following steps: acquiring a preset shunt tree model and request traffic of a target experiment; the preset shunting tree model is used for guiding distribution of request traffic; determining expected path information corresponding to the request traffic in a preset shunting tree model; according to the expected path information and a preset shunt tree model, determining a grouping result of the request traffic in the target experiment, the grouping result corresponding to the request traffic being used for indicating whether the request traffic is traffic in a control group in the target experiment or traffic in an experiment group; and performing an A / B experiment according to the grouping result of the request traffic. According to the technical scheme provided by the embodiment of the invention, the reliability of the experiment result of the A / B experiment can be improved.

Description

Technical Field

[0001] The present disclosure belongs to the field of data processing technology, and in particular relates to an A / B experiment method, device, electronic equipment and medium based on traffic diversion. Background Art

[0002] A / B experiment refers to taking part of the online traffic and completely randomly distributing it to strategy A and strategy B, and then combining it with certain statistical methods to accurately estimate the relative effects of the two strategies, so as to determine which strategy is better. In related technologies, although A / B experiments have achieved hierarchical configuration, as the number and complexity of experiments increase, the complexity of managing and coordinating different levels and experiments will also increase, which may lead to unreasonable distribution of the two groups of traffic, and thus make the reliability of the experimental results of A / B experiments low. Summary of the invention

[0003] The embodiments of the present disclosure provide an implementation scheme different from the related art to solve the technical problem in the related art that the distribution of traffic is unreasonable, thereby resulting in low reliability of the experimental results of the A / B experiment.

[0004] In a first aspect, the present disclosure provides an A / B experiment method based on traffic diversion, comprising:

[0005] Obtaining a preset traffic flow tree model and a request flow of a target experiment; the preset traffic flow tree model is used to guide the distribution of the request flow;

[0006] Determine the expected path information corresponding to the request traffic in the preset branching tree model;

[0007] Determine, according to the expected path information and the preset branching tree model, a grouping result of the request traffic in the target experiment, wherein the grouping result corresponding to the request traffic is used to indicate whether the request traffic is traffic in a control group or traffic in an experimental group in the target experiment;

[0008] An A / B experiment is performed based on the grouping result of the request traffic.

[0009] In a second aspect, the present disclosure provides an A / B experimental device based on traffic diversion, comprising:

[0010] An acquisition unit, used to acquire a preset branching tree model and a request flow of a target experiment; the preset branching tree model is used to guide the distribution of the request flow;

[0011] A determination unit, configured to determine expected path information corresponding to the request traffic in the preset branching tree model;

[0012] The determining unit is further used to determine a grouping result of the request traffic in the target experiment according to the expected path information and the preset branching tree model, wherein the grouping result corresponding to the request traffic is used to indicate whether the request traffic is a traffic in a control group or a traffic in an experimental group in the target experiment;

[0013] The experiment unit is used to perform an A / B experiment according to the grouping result of the request traffic.

[0014] In a third aspect, the present disclosure provides an electronic device, including:

[0015] Processor; and

[0016] A memory, configured to store executable instructions of the processor;

[0017] The processor is configured to execute any method in the first aspect or any possible implementation manner of the first aspect by executing the executable instructions.

[0018] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the computer program implements the first aspect and any one of the methods in the possible implementation modes of the first aspect.

[0019] The present disclosure provides a method for obtaining a preset diversion tree model and request traffic of a target experiment; the preset diversion tree model is used to guide the distribution of the request traffic; the expected path information corresponding to the request traffic in the preset diversion tree model is determined; based on the expected path information and the preset diversion tree model, the grouping result of the request traffic in the target experiment is determined, and the grouping result corresponding to the request traffic is used to indicate whether the request traffic is the traffic in the control group or the traffic in the experimental group in the target experiment; a scheme for conducting an A / B experiment based on the grouping result of the request traffic can guide the distribution of the request traffic of the target experiment through the preset diversion tree model, and determine the grouping result of the request traffic in the target experiment by using the expected path information corresponding to the request traffic in the preset diversion tree model and the preset diversion tree model, thereby improving the rationality of the traffic distribution and further improving the reliability of the experimental results of the A / B experiment. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0021] Figure 1 A schematic diagram of a flow chart of an A / B experiment method based on traffic diversion provided by an exemplary embodiment of the present disclosure;

[0022] Figure 2 A schematic diagram of the structure of an A / B experiment system provided by an exemplary embodiment of the present disclosure;

[0023] Figure 3 A schematic diagram of a preset branching tree model provided for an exemplary embodiment of the present disclosure;

[0024] Figure 4 A schematic diagram of an experimental design architecture provided for an exemplary embodiment of the present disclosure;

[0025] Figure 5 A schematic diagram of the structure of an A / B experimental device based on flow diversion provided by an exemplary embodiment of the present disclosure;

[0026] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0027] Embodiments of the present disclosure are described in detail below, and examples of the embodiments are shown in the accompanying drawings. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present disclosure, but should not be understood as limiting the present disclosure.

[0028] The terms "first" and "second" and the like in the specification, claims, and drawings of the disclosed embodiments are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the disclosed embodiments described herein can, for example, be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0029] First, some terms in the embodiments of the present disclosure are explained below to facilitate understanding by those skilled in the art.

[0030] Hash is the transliteration of Hash, which transforms an input of arbitrary length (also called pre-image) into an output of fixed length through a hash algorithm. The output is the hash value.

[0031] A database (DB) is a warehouse that organizes, stores, and manages data according to data structures.

[0032] A / B experiment refers to taking out part of the online traffic, completely randomly distributing it to strategy A and strategy B, and then combining it with certain statistical methods to obtain an accurate estimate of the relative effects of the two strategies, so as to determine which strategy is better. In the related art, although the A / B experiment has achieved hierarchical configuration, as the number and complexity of experiments increase, the complexity of managing and coordinating different levels and experiments will also increase, which may lead to unreasonable distribution of the two groups of traffic, and thus make the reliability of the experimental results of the A / B experiment low. Therefore, the present disclosure provides an A / B experiment method, device, electronic device and medium based on traffic diversion to solve the technical problem of unreasonable distribution of traffic in the related art, which leads to low reliability of the experimental results of the A / B experiment.

[0033] The following specific embodiments are used to describe in detail the technical solution of the present invention and how the technical solution of the present invention solves the above-mentioned technical problems. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present invention will be described below in conjunction with the accompanying drawings.

[0034] Figure 1 A flow chart of an A / B experiment method based on traffic diversion provided by an exemplary embodiment of the present disclosure is provided. The method can be applied to electronic devices such as personal computers, tablet computers, and smart phones. The method at least includes the following steps S1-S4:

[0035] S1. Obtaining a preset traffic flow tree model and a request flow of a target experiment; the preset traffic flow tree model is used to guide the distribution of the request flow;

[0036] Among them, the request traffic of the target experiment refers to the request traffic of the target experiment to be performed; specifically, the request traffic carries a request identifier, the request identifier includes an experiment identifier and a traffic identifier, the experiment identifier is used to uniquely identify the target experiment corresponding to the request traffic, and the traffic identifier is used to uniquely identify the request traffic.

[0037] In some embodiments, the preset diversion tree model includes multiple nodes of the following types: domain nodes, hierarchical nodes and experimental nodes, wherein the domain nodes include default domain nodes, the root nodes of the preset diversion tree model include default domain nodes, and the leaf nodes of the preset diversion tree model include experimental nodes of different experiments.

[0038] Among them, the diversion tree model can be used to distribute the request traffic to the control group or experimental group of the corresponding experiment. The diversion tree model includes three types of nodes, namely domain nodes, layer nodes and experimental nodes. Among them, an experimental node in the diversion tree model corresponds to an experiment; the domain node in the diversion tree model is used to represent a domain in the A / B experiment. The request traffic in the A / B experiment can be divided into several domains, each domain only includes a part of the traffic, and the domain node includes a default domain node, which is used to represent the default domain in the A / B experiment (that is, all request traffic); the layer node in the diversion tree model can represent the layer in the A / B experiment, and the layer node may include a domain layer node and an experimental layer node. Specifically, the son node of the domain node includes a layer node, the son node of the domain layer node includes a domain node, and the son node of the experimental layer node includes an experimental node. The root node of the preset diversion tree model is generally the default domain node, and the leaf nodes of the preset diversion tree model include experimental nodes of different experiments; further, the leaf nodes of the preset diversion tree model may also include domain nodes and layer nodes, that is, the domain nodes and layer nodes exist suspended on the leaf nodes.

[0039] In some embodiments, relevant personnel can use the relevant A / B experiment system to construct the above-mentioned preset branching tree model. Figure 2 A schematic diagram of the structure of an A / B experiment system provided by an exemplary embodiment of the present disclosure is shown in FIG. Figure 2 As shown, the A / B experiment system includes management services, diversion services, experiment configuration DB and experiment grouping DB. Among them, the experiment configuration DB can be used to store the preset diversion tree model and other experiment-related information, and the experiment grouping DB can be used to store information such as grouping results. The experiment configuration DB and the experiment grouping DB can be specifically MySQL databases.

[0040] In some embodiments, the above management service specifically includes a domain management module, a hierarchical management module, and an experimental management module. In actual applications, relevant personnel can use the domain management module to create a domain node and bind it to a certain domain hierarchical node, so that the domain node is used as a child node of the bound domain hierarchical node; they can also use the hierarchical management module to create a domain hierarchical node and / or an experimental hierarchical node, under which these two hierarchical nodes can be bound to domain nodes and experimental nodes respectively; they can also use the experimental management module to create an experimental node and bind the experimental node to a certain experimental hierarchical node, so that the experimental node is used as a child node of the bound experimental hierarchical node, and the experimental node can also be configured with a grouping of experiments corresponding to the experimental node, for example, including a control group and multiple experimental groups.

[0041] In some embodiments, the above-mentioned diversion service may specifically include a domain diversion module, a layered diversion module, and an experimental diversion module. Among them, the domain diversion module can be used to use a preset hash function to perform domain diversion on the experimental traffic and determine which layered node under the domain node it enters; the layered diversion module is used to use a preset hash function to perform layer diversion on the traffic corresponding to the experimental node and determine which domain node (or experimental node, depending on whether the layered node is an experimental layered node or a domain layered node) under the layered node it enters.

[0042] Figure 3 A schematic diagram of a preset branching tree model provided by an exemplary embodiment of the present disclosure is shown as follows: Figure 3 As shown, the nodes included in the preset branching tree model can be expressed as: D_11, D_31, D_32, D_33, D_51, D_52, DL_21, DL_22, DL_41, DL_42, EL_23, EL_43, EL_61, EL_62, EL_63, E_34, E_53, E_71, E_72 and E_73; among them, D_11, D_31, D_32, D_33, D_51 and D_52 are domain nodes, DL_21, DL_22, DL_41 and DL_42 are domain hierarchical nodes, EL_23, EL_43, EL_61, EL_62 and EL_63 are experimental hierarchical nodes, and E_34, E_53, E_71, E_72 and E_73 are experimental nodes.

[0043] Figure 3 D_11 in the preset shunt tree model is the default domain node, that is, D_11 is the root node of the preset shunt tree model. The child nodes of D_11 include: DL_21, DL_22 and EL_23; the child nodes of DL_21 include: D_31 and D_32; the child nodes of DL_22 include D_33; the child nodes of EL_23 include: E_34; the child nodes of D_31 include: DL_41 and DL_42; the child nodes of D_33 include: EL_43; the child nodes of EL_43 include E_53; the child nodes of DL_41 include: D_51 and D_52; the child nodes of D_51 include EL_61 and EL_62; the child nodes of D_52 include EL_63; the child nodes of EL_61 include E_71 and E_72; the child nodes of EL_62 include E_73.

[0044] Figure 3The parent node and the child node in the preset traffic tree model in are connected by dotted lines or solid lines. For a group of child nodes and parent nodes connected by dotted lines, the traffic corresponding to the child node and other child nodes of the parent node are orthogonal to each other. For example, for the child node DL_21 and the parent node D_11 connected by dotted lines, the traffic corresponding to the child node DL_21 and other child nodes DL_22 and EL_23 of the parent node D_11 are orthogonal to each other, which means that all the traffic corresponding to the parent node D_11 can directly enter DL_21, or directly enter DL_22 or EL_23. For a group of child nodes and parent nodes connected by solid lines, the traffic corresponding to the child node and other child nodes of the parent node is mutually exclusive. For example, for the child node D_31 and the parent node DL_21 connected by solid lines, the traffic corresponding to the child node D_31 and other child node D_32 of the parent node DL_21 is mutually exclusive, which means that the traffic corresponding to the parent node DL_21 can only enter one of the child nodes D_31 and D_32. In addition, when the hierarchical node is used as the parent node and the traffic is diverted to the child nodes of the parent node, the traffic can be diverted according to the weights pre-configured for each child node. Figure 3 Where w0-w9 represents the weight value.

[0045] S2. Determine the expected path information corresponding to the requested traffic in the preset traffic distribution tree model;

[0046] Among them, the expected path information may refer to the expected path of the request traffic for the target experiment to be conducted in the preset diversion tree model, that is, according to the design of the preset diversion tree model, the path information of the request traffic from the default domain node (i.e., the root node) to the experimental node corresponding to the target experiment (i.e., the leaf node). The expected path information may include the domain nodes, hierarchical nodes, and experimental nodes that the request traffic passes through, so as to correctly distribute the request traffic to the corresponding experimental group or control group. After obtaining the request traffic of the target experiment, the path of the request traffic from the default domain node to the experimental node corresponding to the target experiment can be found in the preset diversion tree model to obtain the expected path information of the request traffic.

[0047] In some embodiments, in S2, determining the expected path information corresponding to the request traffic in the preset traffic tree model includes the following steps S21-S22:

[0048] S21, determining a target experimental node corresponding to the request flow in a leaf node in the preset diversion tree model;

[0049] In some embodiments, the target experimental node corresponding to the request flow can be determined according to the experimental identifier in the request identifier corresponding to the request flow. Specifically, the leaf node in the preset diversion tree model whose corresponding experimental node identifier (such as experiment_id) is the same as the request identifier corresponding to the request flow is determined as the target experimental node corresponding to the request flow.

[0050] S22. Based on the target experimental node, access the preset diversion tree model level by level until the default domain node of the preset diversion tree model is accessed, and the expected path information corresponding to the request traffic in the preset diversion tree model is determined according to the accessed node.

[0051] That is to say, taking the target experimental node as the starting point, tracing back from bottom to top in the preset diversion tree model until the root node (i.e., the default domain node) of the preset diversion tree model is accessed, the expected path information corresponding to the request traffic in the preset diversion tree model is determined based on all the nodes that have been visited.

[0052] In some embodiments, in S22, based on the target experimental node, the preset shunt tree model is accessed level by level until the default domain node of the preset shunt tree model is accessed, and the expected path information corresponding to the request traffic in the preset shunt tree model is determined according to the accessed node, including the following steps S221-S225:

[0053] S221, taking the target experimental node as the first current node;

[0054] S222: Based on the first current node, access the parent node of the first current node in the preset branching tree model to obtain a hierarchical node corresponding to the first current node;

[0055] In the above-mentioned preset branching tree model, when tracing back from bottom to top, the parent node of the first current node is unique. Therefore, based on the first current node, the parent node of the first current node (that is, the hierarchical node corresponding to the first current node) can be directly determined and accessed in the preset branching tree model.

[0056] S223: Access the parent node of the hierarchical node corresponding to the first current node in the preset branching tree model to obtain the domain node corresponding to the first current node;

[0057] In some embodiments, in the preset branching tree model, the parent node of the hierarchical node corresponding to the first current node is used as the domain node corresponding to the first current node.

[0058] S224: Determine whether the domain node corresponding to the first current node is the default domain node;

[0059] In some embodiments, if it is determined that the domain node corresponding to the first current node is the default domain node, it means that the root node of the preset diversion tree model has been accessed; if it is determined that the domain node corresponding to the first current node is not the default domain node, it means that the root node of the preset diversion tree model has not been accessed.

[0060] S225. If the domain node corresponding to the first current node is not the default domain node, the domain node corresponding to the first current node is used as the new first current node, and the process returns to the step of accessing the parent node of the first current node in the preset diversion tree model to obtain the hierarchical node corresponding to the first current node, until it is determined that the domain node corresponding to the first current node is the default domain node, and the expected path information corresponding to the request traffic in the preset diversion tree model is generated based on the target experimental node and the domain nodes or hierarchical nodes accessed at each level.

[0061] In some embodiments, in S225, generating the expected path information corresponding to the request traffic in the preset diversion tree model based on the target experimental node and the domain nodes or layer nodes accessed at each level includes the following steps:

[0062] The target experimental node and the domain nodes or hierarchical nodes accessed at each level, as well as the access order of each node, are determined as the expected path information corresponding to the request traffic in the preset diversion tree model.

[0063] By accessing nodes layer by layer, the expected path information of the request traffic in the preset diversion tree model can be determined, making the traffic diversion process clearer and more predictable, so as to effectively guide the diversion process of the request traffic in the preset diversion tree model and improve the performance and efficiency of the diversion.

[0064] by Figure 3Taking the preset diversion tree model shown as an example, the experimental identifier of the request traffic is E_53. At this time, it can be determined that the target experimental node corresponding to the request traffic is node E_53. With node E_53 as the first current node, the node of the previous level corresponding to node E_53 is accessed in the preset diversion tree model, that is, the parent node of node E_53, and the hierarchical node corresponding to node E_53 is obtained as node EL_43; then, the node of the previous level corresponding to node EL_43 is continued to be accessed in the preset diversion tree model, that is, the parent node of node EL_43, and the domain node corresponding to node E_53 is obtained as node D_33. Since the domain node corresponding to node E_53 is node D_33, which is not the default domain node, that is, node D_33 is not the root node of the preset diversion tree model, node D_33 is used as the new first current node, and the node of the previous level corresponding to node D_33 in the preset diversion tree model is continued to be accessed, that is, the parent node of node D_33, and the hierarchical node corresponding to node D_33 is obtained as node DL_22, and the node of the previous level corresponding to node DL_22 is accessed in the preset diversion tree model, that is, the parent node of node DL_22, and the domain node corresponding to node D_33 is obtained as node D_11. Since node D_11 is the default domain node, that is, node D_11 is the root node of the preset diversion tree model, the expected path information corresponding to the request traffic in the preset diversion tree model can be generated based on the target experimental node E_53 and the domain nodes or hierarchical nodes accessed at each level: node D_11, node DL_22, node D_33, node EL_43, node E_53.

[0065] S3. Determine, according to the expected path information and the preset branching tree model, a grouping result of the request traffic in the target experiment, wherein the grouping result corresponding to the request traffic is used to indicate whether the request traffic is a traffic in a control group or an experimental group in the target experiment;

[0066] Specifically, starting from the default domain node (i.e., the root node) of the preset diversion tree model, diversion judgment can be performed from top to bottom based on the preset diversion tree model to obtain the path information that the request traffic currently actually accesses in the preset diversion tree model. If the path information actually accessed by the request traffic is inconsistent with the expected path information, the request traffic can be determined as the traffic of the control group of the target experiment. If the path information actually accessed by the request traffic is consistent with the expected path information, the request traffic can be determined as the traffic of the experimental group of the target experiment, and the experimental grouping of the request traffic in the target experiment can be determined.

[0067] In some embodiments, in S3, according to the expected path information and the preset branching tree model, determining the grouping result of the request traffic in the target experiment includes the following steps S31-S34:

[0068] S31, taking the default domain node in the preset shunt tree model as the starting point, according to the hierarchical order of the preset shunt tree model, and according to the second current node where the request traffic is located in the current hierarchical order, performing shunt judgment on the request traffic, obtaining at least one actual node to which the request traffic is shunted in the next hierarchical order, and comparing the actual node of the next hierarchical order with the expected node corresponding to the next hierarchical order in the expected path information;

[0069] Among them, the hierarchical order may refer to the level at which each node is located in the preset diversion tree model. For example, starting from the root node, the root node is at the first level, its direct child node is at the second level, the child node of the child node is at the third level, and so on. The hierarchical order increases with each downward level. The actual node may refer to the node that the request traffic actually enters in each hierarchical order in the preset diversion tree model, and the expected node may refer to the node that the request traffic enters in each hierarchical order in the preset diversion tree model as recorded in the expected path information. Specifically, for any hierarchical order of the preset diversion tree model, the request traffic can be diverted according to the second current node where the request traffic is located in the previous hierarchical order, and at least one actual node to which the request traffic is diverted in the next hierarchical order is obtained.

[0070] For example, when the current hierarchical order is the first hierarchical order, the request traffic can be diverted to the next hierarchical order (i.e., the second hierarchical order) according to the second current node into which the request traffic flows at the first hierarchical order, so as to obtain the actual node to which the request traffic is diverted at the second hierarchical order, and then the actual node to which the request traffic is diverted at the second hierarchical order can be compared with the expected node of the second hierarchical order recorded in the expected path information; when the current hierarchical order is the second hierarchical order, the request traffic can be diverted to the next hierarchical order (i.e., the third hierarchical order) according to the second current node into which the request traffic flows at the second hierarchical order, so as to obtain the actual node to which the request traffic is diverted at the third hierarchical order, and then the actual node to which the request traffic is diverted at the third hierarchical order can be compared with the expected node of the third hierarchical order recorded in the expected path information; and so on and so forth will not be repeated.

[0071] Continue with Figure 3 For example, Figure 3In the preset diversion tree model shown, the first level includes node D_11, the second level includes node DL_21, node DL_22 and node EL_23, and the third level includes node D_31, node D_32, node DL_33 and node E_34; when the current level order is the first level, the request traffic can be diverted to the next level order (i.e., the second level) according to the node D_11 into which the request traffic flows at the first level, so as to obtain the actual node to which the request traffic is diverted at the second level; assuming that the actual node to which the request traffic is diverted at the second level is node DL_21, the request traffic can be diverted to the next level order (i.e., the third level) according to the node DL_21 into which the request traffic flows at the second level, so as to determine whether the actual node to which the request traffic is diverted at the third level is node D_31 or node D_32.

[0072] Specifically, when making a judgment on the next level of diversion, the actual node to which the request traffic is diverted in the next level of order can be determined among the subnodes under the second current node according to the pre-configured weights of the subnodes under the second current node in the preset diversion tree model. Figure 3 As shown, the weight w0 of node D_31 and the weight w1 of node D_32, when the request traffic flows into the node DL_21 at the second level, the request traffic accounting for w0 can be diverted to the node D_31 at the third level, and the request traffic accounting for w1 can be diverted to the node D_32 at the third level.

[0073] Furthermore, the actual node to which the request traffic is diverted in the next hierarchical order can be determined in the preset diversion tree model and in the child nodes of the second current node based on the node identifier of the second current node into which the current hierarchical order flows and the request identifier of the request traffic.

[0074] Furthermore, when making a diversion judgment for the next hierarchical order, the node type corresponding to the second current node can be obtained according to the node identifier of the second current node into which the current hierarchical order flows, and then different diversion judgments can be made according to the node type of the second current node; in some embodiments, in S31, a diversion judgment is made on the request traffic according to the second current node where the request traffic is located in the current hierarchical order, and at least one actual node to which the request traffic is diverted in the next hierarchical order is obtained, including the following steps S311-S312:

[0075] S311, if the second current node where the request traffic is located in the current hierarchical order is a hierarchical node, according to the node identifier of the hierarchical node and the request identifier of the request traffic, determine the actual node to which the request traffic is diverted in the next hierarchical order in the child node corresponding to the hierarchical node in the preset diversion tree model;

[0076] The actual node may refer to the node that the request traffic actually enters in the preset diversion tree model. Specifically, the node type corresponding to the second current node may be obtained based on the node identifier of the second current node into which the current hierarchical order flows. If the second current node is a hierarchical node, the actual node to which the request traffic is diverted may be determined in the child node corresponding to the hierarchical node in the preset diversion tree model based on the node identifier of the hierarchical node and the request identifier of the request traffic.

[0077] In some embodiments, in S311, the determining, in the child node corresponding to the hierarchical node in the preset shunt tree model, the actual node to which the request traffic is diverted in the next hierarchical order according to the node identifier of the hierarchical node and the request identifier of the request traffic, includes the following steps S3111-S3113:

[0078] S3111, generating a target value based on the node identifier of the hierarchical node and the request identifier of the request flow;

[0079] Optionally, the node identifier of the hierarchical node and the request identifier of the request traffic may be concatenated to obtain a target value.

[0080] S3112, using a preset hash function to process the target value to obtain a processing result;

[0081] In some embodiments, the preset hash function may be a Hash algorithm, that is, taking a target value as input data, and using the Hash algorithm to convert the target data into an output value of a fixed length as a processing result.

[0082] S3113. Determine, in accordance with the processing result, in the child nodes corresponding to the hierarchical nodes in the preset traffic diversion tree model, the actual nodes to which the request traffic is diverted in the next hierarchical order.

[0083] In some embodiments, among the sub-nodes corresponding to the hierarchical nodes in the preset diversion tree model, the sub-nodes whose corresponding node identifiers are the same as the processing results can be determined as the actual nodes to which the request traffic is diverted in the next level order.

[0084] Specifically, the node identifier of the hierarchical node and the request identifier of the request traffic can be concatenated to obtain a target value, and the target value can be mapped to a target hash value using a hash algorithm, and then the target hash value is modulo-operated with the first quantity, and finally the actual node to which the request traffic is diverted is determined according to the result of the modulo-operation. The first quantity can be the number of child nodes corresponding to the hierarchical node, or the number of child nodes corresponding to the hierarchical node plus 1, that is, a null value group is set, and the actual node to which the request traffic is diverted in the next level order can be a null value.

[0085] For example, the actual node to which the request traffic is diverted can be determined by the following formula (1):

[0086] Num=|hash(user id ||layer_id)|%n (1)

[0087] Among them, user id Indicates the request ID of the request flow, layer_id indicates the node ID of the layer node, (user id ||layer_id) represents the node identifier of the concatenated layered node and the request identifier of the request traffic, n represents the first quantity, hash() represents the hash algorithm, Num represents the serial number of the node to which the request traffic is diverted, and the child nodes corresponding to the layered nodes can be sorted in advance to determine the serial numbers of each child node, and then based on the calculation result Num, the actual node to which the request traffic is diverted in the next level order is determined in the child nodes corresponding to the layered nodes.

[0088] Furthermore, considering that each subnode under the hierarchical node in the preset diversion tree model is pre-configured with a diversion weight, the diversion interval of each subnode can be determined according to the pre-configured diversion weight of each subnode under the hierarchical node, and then the second number is determined according to the diversion interval of each subnode. For example, the hierarchical node includes 3 subnodes, each with a pre-configured diversion weight of 10%, 50% and 40%. It can be determined that the diversion intervals of each subnode are [0, 9], [10, 49] and [50, 100] respectively, and the second number is determined to be 100; and then the actual node to which the request traffic is diverted can be determined by the following formula (2):

[0089] Num=|hash(user id ||layer_id)|%m (2)

[0090] Among them, user idRepresents the request identifier of the request traffic, layer_id represents the node identifier of the layered node, m represents the second number, hash() represents the hash algorithm, and Num represents the diversion interval value to which the request traffic is diverted; after obtaining the diversion interval value Num to which the request traffic is diverted, based on the diversion interval value Num where the diversion interval value Num is located, determine the actual node to which the request traffic is diverted in the next level order in the child node corresponding to the layered node.

[0091] S312: If the second current node is a domain node, all child nodes corresponding to the domain node in the preset traffic diversion tree model are determined as actual nodes to which the request traffic is diverted in the next level order.

[0092] Specifically, the node type corresponding to the second current node can be obtained according to the node identifier of the second current node flowing into the current hierarchical order. If the second current node is a domain node, all child nodes corresponding to the domain node in the preset branching tree model can be determined as the actual nodes to which the request traffic is diverted in the next hierarchical order. Figure 3 In the preset diversion tree model shown, if the current hierarchical order is the first level, the node D_11 to which the request traffic flows in the first level is used, and the diversion judgment of the request traffic to the next hierarchical order (i.e., the second level) is performed, the child nodes of node D_11, i.e., nodes DL_21, DL_22, and EL_23, can be used as the actual nodes to which the request traffic is diverted in the second level. In other words, the domain node is not processed and can continue to be accessed downward, that is, the request traffic flowing into each hierarchical node under the same domain node does not need to be diverted, so that the request traffic of multiple layers under the same domain is orthogonal.

[0093] S32: if the actual node does not include the expected node, determine that the grouping result of the request traffic in the target experiment is a control group;

[0094] S33: If the actual node includes the expected node and the actual node is an experimental node, determine that the grouping result of the request traffic in the target experiment is an experimental group;

[0095] S34. If the actual node includes the expected node and the actual node is not an experimental node, use the actual node as the new second current node of the request traffic, continue to execute the steps of making a diversion judgment on the request traffic according to the second current node where the request traffic is located in the current hierarchical order, obtain at least one actual node to which the request traffic is diverted in the next hierarchical order, and compare the actual node of the next hierarchical order with the expected node corresponding to the next hierarchical order in the expected path information.

[0096] That is, in the preset branching tree model, according to the hierarchical order of the preset branching tree model, comparison is performed layer by layer from the root node (default domain node) downward to determine whether the actual node includes the expected node.

[0097] By comparing the actual nodes with the expected nodes, it is possible to determine whether the request traffic meets the expected experimental conditions, thereby performing effective experimental analysis and comparison to obtain a grouping result with higher accuracy.

[0098] Combination Figure 3 , taking the expected path information corresponding to the request traffic in the preset diversion tree model as: node D_11, node DL_22, node D_33, node EL_43, node E_53 as an example, taking the default domain node (i.e., node D_11) in the preset diversion tree model as the starting point, according to the hierarchical order of the preset diversion tree model, the actual nodes entered by the request traffic in different hierarchical orders are judged in sequence. Specifically, when the current hierarchical order is the first level, the node into which the request traffic flows in the first level is node D_11, and the diversion judgment of the next hierarchical order (i.e., the second level) is performed on the request traffic. Since node D_11 is a domain node, the child nodes of node D_11 can be regarded as the actual nodes to which the request traffic is diverted in the second level. At this time, the actual nodes of the second level include node DL_21, node DL_22, and node EL_23.

[0099] By comparing the actual nodes of the second level (node ​​DL_21, node DL_22 and node EL_23) and the expected nodes of the second level recorded in the expected path information (i.e., node DL_22), the actual nodes of the second level now include the expected nodes corresponding to the second level in the expected path information (i.e., node DL_22), and node DL_22 is not an experimental node. Node DL_22 can be used as the new second current node of the request traffic. At this time, the current level order is updated to the second level, and the request traffic continues to be diverted based on the second current node (i.e., node DL_22) where the request traffic is located in the second level, and at least one actual node to which the request traffic is diverted at the third level is obtained.

[0100] Since node DL_22 is a domain hierarchical node, the actual node to which the request traffic is diverted at the third level can be determined based on the node identifier of node DL_22, the request identifier of the request traffic, and the weight configured by node D_33.

[0101] If the actual node to which the request traffic is diverted at the third level is a null value, or the actual node does not include the expected node and does not include the expected node corresponding to the third level in the expected path information, that is, the request traffic cannot enter the domain node D_33 of the next level recorded in the expected path information at the domain hierarchical node DL_22, it can be determined that the grouping result of the request traffic in the target experiment is the control group.

[0102] If the actual node to which the request traffic is diverted at the third level is node D_33, that is, the actual node to which the request traffic is diverted at the third level includes the expected node corresponding to the third level in the expected path information, then the diversion relationship between node DL_22 and node D_33 is recorded, and node D_33 is used as the new second current node for the request traffic. At this time, the current level order is updated to the third level, and the diversion judgment of the request traffic continues to be performed based on the second current node (that is, node D_33) where the request traffic is located at the third level, and at least one actual node to which the request traffic is diverted at the fourth level is obtained.

[0103] Continuing to take the actual node to which the request traffic is diverted at the third level as node D_33 as an example, since node D_33 is a domain node, all the child nodes of node D_33 can be used as the actual nodes to which the request traffic is diverted at the fourth level, and at this time, the actual nodes of the fourth level include node EL_43. By comparing the actual nodes of the fourth level (i.e., node EL_43) and the expected nodes of the fourth level recorded in the expected path information (i.e., node EL_43), at this time, the actual nodes of the fourth level include the expected nodes corresponding to the fourth level in the expected path information (i.e., node EL_43), and node EL_43 is not an experimental node, node EL_43 can be used as the new second current node of the request traffic, and at this time, the current level order is updated to the fourth level, and the request traffic is continued to be diverted according to the second current node (i.e., node EL_43) where the request traffic is located at the fourth level, and at least one actual node to which the request traffic is diverted at the fifth level is obtained.

[0104] Since node EL_43 is an experimental hierarchical node, the actual node to which the request traffic is diverted at the fifth level can be determined based on the node ID of node EL_43, the request ID of the request traffic, and the weight configured for node E_53.

[0105] If the actual node to which the request traffic is diverted at the fifth level is a null value, or the actual node does not include the expected node and does not include the expected node corresponding to the fifth level in the expected path information, that is, the request traffic cannot enter the experimental node E_5 of the next level recorded in the expected path information at the experimental layered node EL_43, it can be determined that the grouping result of the request traffic in the target experiment is the control group.

[0106] If the actual node to which the request traffic is diverted at the fifth layer is node E_53, that is, the actual node to which the request traffic is diverted at the fifth layer includes the expected node corresponding to the fifth layer in the expected path information, and the expected node corresponding to the fifth layer in the expected path information is the experimental node, it can be determined that the grouping result of the request traffic in the target experiment is the experimental group.

[0107] Furthermore, considering that the experimental group of the target experiment may include multiple experimental groups, the experimental group of the request traffic in the experimental group corresponding to the target experiment can be obtained through the experimental node; specifically, the node identifier of the hierarchical node can be spliced ​​with the request identifier of the request traffic to obtain the target value, and the target value can be mapped to a target hash value using a hash algorithm, and then the target hash value is modulo-operated on the second number, and finally the experimental group to which the request traffic is diverted is determined according to the operation structure of the modulo operation. Among them, the second number can be the number of experimental groups of the target experiment.

[0108] S4. Perform an A / B experiment according to the grouping result of the request traffic.

[0109] In some embodiments, in S4, an A / B experiment is performed according to the grouping result of the request traffic, including the following steps S41-S42:

[0110] S41, when the grouping result of the request traffic indicates that the request traffic is the traffic in the control group in the target experiment, processing the request traffic according to the first experimental parameter corresponding to the control group;

[0111] S42: When the grouping result of the request traffic indicates that the request traffic is the traffic in the experimental group in the target experiment, process the request traffic according to a second experimental parameter corresponding to the experimental group.

[0112] In some embodiments, the first experimental parameter and the second experimental parameter may refer to relevant information pre-set for the control group and the experimental group for experimental processing. The first experimental parameter and the second experimental parameter may be used to effectively compare flow processing under different conditions, and perform experimental analysis and comparison.

[0113] Different from the traditional A / B experiment design architecture, this disclosure provides an experimental design architecture that supports domain and layer nesting. Figure 4 A schematic diagram of an experimental design architecture provided by an exemplary embodiment of the present disclosure is shown in FIG. Figure 4The experimental design architecture provided by the present invention can support the configuration of domain nodes and layer nodes, support their infinite levels of nesting, and support custom configuration to configure a preset diversion tree model. For example, multiple experimental domains (corresponding to multiple domain nodes) and a blank domain (corresponding to another domain node) can be set under the same layer (corresponding to a layer node). The setting of multiple experimental domains is for different business teams to conduct comparative experiments, so as to support multiple business teams to conduct long-term blank domain control experiments at the same time, and the request traffic between different teams is mutually exclusive; at the same time, for any experimental domain, multiple layers (corresponding to multiple layer nodes) can also be set under the experimental domain. The setting of multiple layers can be used for different business groups within the business team to conduct different experiments, and the request traffic of different business groups within the business team is orthogonal, that is, the request traffic for experiments within the same business team is the same, and the request traffic for experiments between different business teams is different.

[0114] Furthermore, for any layer under any experimental domain, different experimental subdomains (corresponding domain nodes) can be set so that the same business team can set different experimental groups when conducting experiments, that is, the request traffic corresponding to different experimental groups when the business team conducts experiments is different.

[0115] By nesting domains and layers at unlimited levels, experiments can be run under multiple hierarchical nodes and multiple domain nodes, making the experimental design more flexible and scalable, and able to meet the needs of different experiments. It can effectively improve the processing efficiency of A / B experiments and the reliability of experimental results.

[0116] like Figure 4As shown, the default domain (i.e., the default domain node) includes layer 1 (corresponding to a layer node), layer 2 (corresponding to a layer node), and layer 3 (corresponding to a layer node). The request traffic between layers 1, 2, and 3 is orthogonal, that is, layers 1, 2, and 3 share the same request traffic. For layers 1 and 2, experiments (corresponding to experimental nodes) and different experimental groups and control groups under the experiments can be set. The request traffic of different experimental groups and control groups is mutually exclusive, that is, different experimental groups and control groups under layer 1 (or layer 2) divide the request traffic flowing into the layer. For layer 3, domain 1 (corresponding to a domain node) and domain 2 (corresponding to a domain node) can be set. The request traffic of domain 1 and domain 2 is mutually exclusive, that is, the request traffic flowing into the layer is divided. Specifically, domain 1 can be used as a long-term experimental domain, and domain 2 can be used as a long-term control domain of the department. For domain 1, multiple layers (corresponding to layer nodes) can be set, such as layer 4, layer 5 and layer 6, among which layer 4 can set two domains (corresponding to domain nodes) as the experimental group and the control group under an experiment respectively, and layer 6 can set two domains (corresponding to domain nodes), and one of the domains is nested to set two layers. It can be understood that different experiments can be set under any layer node or domain node under layer 3 based on the requirements of mutual exclusion or orthogonality of request traffic, which can greatly improve the flexibility and scalability of experimental design, meet the requirements of orthogonality or mutual exclusion of request traffic between different experiments, and can effectively improve the processing efficiency of A / B experiments and the reliability of experimental results. Subsequently, a preset diversion tree model can be constructed through the hierarchical relationship between domain nodes, layered nodes and experimental nodes. After obtaining the request traffic of the target experiment, the expected path information corresponding to the request traffic in the preset diversion tree model can be determined. According to the expected path information and the preset diversion tree model, the grouping result of the request traffic in the target experiment is determined. The grouping result corresponding to the request traffic is used to indicate whether the request traffic is the traffic in the control group or the traffic in the experimental group in the target experiment. Finally, an A / B experiment is performed based on the grouping result of the request traffic.

[0117] In some embodiments, the above-mentioned A / B experiment method and experiment design framework based on traffic diversion can be applied to the fields of Internet products, design, search, recommendation system, advertising system, growth hacking, user growth, data analysis, digital operation, intelligent marketing, etc. For example, the following specific scenarios can be included: Scenario 1: In teams that rely on algorithm-driven growth, such as technology finance and e-commerce, there is a relatively high demand, and it is necessary to calculate the incremental effects of multiple experiments on a quarterly or monthly basis. A complete "blank control group" is required, so that some users corresponding to the "blank control group" do not participate in any experiment, so as to evaluate the overall benefits brought by multiple experiments. Such demand can be called "long-term experiment"; Scenario 2: For different groups in the same department, at the same time, different groups expect the experiments within the team to be orthogonal and the experiments outside the team to be mutually exclusive. With a quarter as the node, the experimental effects of the two groups are first evaluated based on the experimental effects within the group, and then the overall evaluation is performed based on the experimental effects of the department.

[0118] The experimental design architecture supporting domain and layer nesting and the A / B experimental method based on traffic diversion provided by the present disclosure, when applied to the above-mentioned scenarios, can compare the effect of long-term iteration (experimental group) with a certain version blank (control group) for a long time, avoid the influence of external factors, thereby improving the efficiency of A / B experiments and the reliability of experimental results.

[0119] The present disclosure provides a method for obtaining a preset diversion tree model and request traffic of a target experiment; the preset diversion tree model is used to guide the distribution of the request traffic; the expected path information corresponding to the request traffic in the preset diversion tree model is determined; based on the expected path information and the preset diversion tree model, the grouping result of the request traffic in the target experiment is determined, and the grouping result corresponding to the request traffic is used to indicate whether the request traffic is the traffic in the control group or the traffic in the experimental group in the target experiment; a scheme for conducting an A / B experiment based on the grouping result of the request traffic can guide the distribution of the request traffic of the target experiment through the preset diversion tree model, and determine the grouping result of the request traffic in the target experiment by using the expected path information corresponding to the request traffic in the preset diversion tree model and the preset diversion tree model, thereby improving the rationality of the traffic distribution and further improving the reliability of the experimental results of the A / B experiment.

[0120] Figure 5 A schematic diagram of the structure of an A / B experimental device based on flow diversion provided by an exemplary embodiment of the present disclosure;

[0121] The device comprises:

[0122] The acquisition unit 51 is used to acquire a preset branching tree model and a request flow of a target experiment; the preset branching tree model is used to guide the distribution of the request flow;

[0123] A determination unit 52, configured to determine expected path information corresponding to the request traffic in the preset branching tree model;

[0124] The determining unit 52 is further configured to determine a grouping result of the request traffic in the target experiment according to the expected path information and the preset branching tree model, wherein the grouping result corresponding to the request traffic is used to indicate whether the request traffic is a traffic in a control group or an experimental group in the target experiment;

[0125] The experiment unit 53 is used to perform an A / B experiment according to the grouping result of the request traffic.

[0126] In some optional embodiments provided by the present disclosure, the preset diversion tree model includes multiple nodes of the following types: domain nodes, hierarchical nodes and experimental nodes, wherein the domain nodes include default domain nodes, the root nodes of the preset diversion tree model include default domain nodes, and the leaf nodes of the preset diversion tree model include experimental nodes of different experiments.

[0127] In some optional embodiments provided by the present disclosure, when the determining unit 52 is used to determine the expected path information corresponding to the request traffic in the preset branching tree model, it is specifically used to:

[0128] Determine a target experimental node corresponding to the request flow in a leaf node in the preset diversion tree model;

[0129] Based on the target experimental node, the preset diversion tree model is accessed level by level until the default domain node of the preset diversion tree model is accessed, and the expected path information corresponding to the request traffic in the preset diversion tree model is determined according to the visited node.

[0130] In some optional embodiments provided by the present disclosure, the determining unit 52 is used to access the preset shunt tree model hierarchically based on the target experimental node until the default domain node of the preset shunt tree model is accessed, and when determining the expected path information corresponding to the request traffic in the preset shunt tree model according to the accessed node, it is specifically used to:

[0131] Taking the target experimental node as the first current node;

[0132] Based on the first current node, access the parent node of the first current node in the preset branching tree model to obtain a hierarchical node corresponding to the first current node;

[0133] Accessing the parent node of the hierarchical node corresponding to the first current node in the preset branching tree model to obtain the domain node corresponding to the first current node;

[0134] Determining whether the domain node corresponding to the first current node is the default domain node;

[0135] If the domain node corresponding to the first current node is not the default domain node, the domain node corresponding to the first current node is used as the new first current node, and the process returns to the step of accessing the parent node of the first current node in the preset diversion tree model to obtain the hierarchical node corresponding to the first current node, until it is determined that the domain node corresponding to the first current node is the default domain node, and based on the target experimental node and the domain nodes or hierarchical nodes accessed at each level, the expected path information corresponding to the request traffic in the preset diversion tree model is generated.

[0136] In some optional embodiments provided by the present disclosure, when the determining unit is used to determine the grouping result of the request traffic in the target experiment according to the expected path information and the preset branching tree model, it is specifically used to:

[0137] Taking the default domain node in the preset shunt tree model as the starting point, according to the hierarchical order of the preset shunt tree model, and according to the second current node where the request traffic is located in the current hierarchical order, the request traffic is shunt judged, at least one actual node to which the request traffic is shunted in the next hierarchical order is obtained, and the actual node of the next hierarchical order is compared with the expected node corresponding to the next hierarchical order in the expected path information;

[0138] If the actual node does not include the expected node, determining that the grouping result of the request traffic in the target experiment is a control group;

[0139] If the actual node includes the expected node and the actual node is the experimental node, determining that the grouping result of the request traffic in the target experiment is an experimental group;

[0140] If the actual node includes the expected node and the actual node is not the experimental node, the actual node is used as the new second current node of the request traffic, and the steps of performing diversion judgment on the request traffic according to the second current node where the request traffic is located in the current hierarchical order are continued, obtaining at least one actual node to which the request traffic is diverted in the next hierarchical order, and comparing the actual node of the next hierarchical order with the expected node corresponding to the next hierarchical order in the expected path information.

[0141] In some optional embodiments provided by the present disclosure, the determination unit 52 is used to perform a diversion judgment on the request traffic according to the current node where the request traffic is located in the current hierarchical order, and obtain at least one actual node to which the request traffic is diverted in the next hierarchical order, specifically for:

[0142] If the second current node where the request traffic is located in the current hierarchical order is the hierarchical node, according to the node identifier of the hierarchical node and the request identifier of the request traffic, determine the actual node to which the request traffic is diverted in the next hierarchical order in the child node corresponding to the hierarchical node in the preset diversion tree model;

[0143] If the second current node is the domain node, all child nodes corresponding to the domain node in the preset branching tree model are determined as actual nodes to which the request traffic is branched in the next level order.

[0144] In some optional embodiments provided by the present disclosure, the determining unit 52 is used to determine, according to the node identifier of the hierarchical node and the request identifier of the request traffic, in the child node corresponding to the hierarchical node in the preset shunt tree model, the actual node to which the request traffic is shunted in the next hierarchical order, specifically for:

[0145] Generate a target value based on the node identifier of the hierarchical node and the request identifier of the request flow;

[0146] Processing the target value using a preset hash function to obtain a processing result;

[0147] According to the processing result, the actual node to which the request traffic is diverted in the next level order is determined in the child node corresponding to the hierarchical node in the preset diversion tree model.

[0148] In some optional embodiments provided by the present disclosure, when the experiment unit 53 is used to perform an A / B experiment according to the grouping result of the request traffic, it is specifically used to:

[0149] When the grouping result of the request traffic indicates that the request traffic is traffic in a control group in the target experiment, processing the request traffic according to a first experimental parameter corresponding to the control group;

[0150] When the grouping result of the request traffic indicates that the request traffic is traffic in the experimental group in the target experiment, the request traffic is processed according to a second experimental parameter corresponding to the experimental group.

[0151] It should be understood that the device embodiment and the method embodiment may correspond to each other, and similar descriptions may refer to the method embodiment. To avoid repetition, no further description is given here. Specifically, the device may perform the above method embodiment, and the above and other operations and / or functions of each module in the device are the corresponding processes in each method in the above method embodiment, respectively, and no further description is given here for the sake of brevity.

[0152] It can be understood that in the specific implementation of the present disclosure, data related to the preset diversion tree model, request traffic, etc. are involved. When the above embodiments of the present disclosure are applied to specific products or technologies, user permission, consent or authorization is required, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0153] The above describes the device of the embodiment of the present disclosure from the perspective of the functional module in conjunction with the accompanying drawings. It should be understood that the functional module can be implemented in hardware form, can be implemented by instructions in software form, and can also be implemented by a combination of hardware and software modules. Specifically, the steps of the method embodiment in the embodiment of the present disclosure can be completed by the hardware integrated logic circuit and / or software form instructions in the processor, and the steps of the method disclosed in conjunction with the embodiment of the present disclosure can be directly embodied as a hardware decoding processor to execute, or a combination of hardware and software modules in the decoding processor to execute. Optionally, the software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps in the above method embodiment in conjunction with its hardware.

[0154] Figure 6 is a schematic block diagram of an electronic device provided by an embodiment of the present disclosure, and the electronic device may include:

[0155] The memory 701 and the processor 702, the memory 701 is used to store the computer program and transmit the program code to the processor 702. In other words, the processor 702 can call and run the computer program from the memory 701 to implement the method in the embodiment of the present disclosure.

[0156] For example, the processor 702 may be configured to execute the above method embodiments according to instructions in the computer program.

[0157] In some embodiments of the present disclosure, the processor 702 may include but is not limited to:

[0158] General-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, etc.

[0159] In some embodiments of the present disclosure, the memory 701 includes but is not limited to:

[0160] Volatile memory and / or non-volatile memory. Among them, the non-volatile memory can be read-only memory (ROM), programmable ROM (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM) or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus random access memory (DR RAM).

[0161] In some embodiments of the present disclosure, the computer program may be divided into one or more modules, which are stored in the memory 701 and executed by the processor 702 to complete the method provided by the present disclosure. The one or more modules may be a series of computer program instruction segments capable of completing specific functions, and the instruction segments are used to describe the execution process of the computer program in the electronic device.

[0162] like Figure 6 As shown, the electronic device may also include:

[0163] The transceiver 703 may be connected to the processor 702 or the memory 701 .

[0164] The processor 702 may control the transceiver 703 to communicate with other devices, specifically, to send information or data to other devices, or to receive information or data sent by other devices. The transceiver 703 may include a transmitter and a receiver. The transceiver 703 may further include an antenna, and the number of antennas may be one or more.

[0165] It should be understood that the various components in the electronic device are connected via a bus system, wherein the bus system includes not only a data bus but also a power bus, a control bus and a status signal bus.

[0166] The present disclosure also provides a computer storage medium on which a computer program is stored, and when the computer program is executed by a computer, the computer can perform the method of the above method embodiment. In other words, the present disclosure embodiment also provides a computer program product containing instructions, and when the instructions are executed by a computer, the computer can perform the method of the above method embodiment.

[0167] When software is used for implementation, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the embodiment of the present disclosure is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a digital video disc (digital video disc, DVD)), or a semiconductor medium (e.g., a solid state disk (solid state disk, SSD)), etc.

[0168] Those of ordinary skill in the art will appreciate that the modules and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this disclosure.

[0169] In the several embodiments provided in the present disclosure, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the module is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0170] The modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. For example, each functional module in each embodiment of the present disclosure may be integrated into a processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.

[0171] The above are only specific embodiments of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present disclosure, which should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims

1. An A / B experiment method based on traffic diversion, characterized in that: include: Get the request traffic of the preset branch tree model and target experiment; The preset traffic distribution tree model is used to guide the distribution of the request traffic; Determine the expected path information corresponding to the request traffic in the preset branching tree model; Determine, according to the expected path information and the preset branching tree model, a grouping result of the request traffic in the target experiment, wherein the grouping result corresponding to the request traffic is used to indicate whether the request traffic is traffic in a control group or traffic in an experimental group in the target experiment; An A / B experiment is performed based on the grouping result of the request traffic.

2. The method according to claim 1, characterized in that The preset diversion tree model includes multiple nodes of the following types: domain nodes, hierarchical nodes and experimental nodes, wherein the domain nodes include default domain nodes, the root nodes of the preset diversion tree model include default domain nodes, and the leaf nodes of the preset diversion tree model include experimental nodes of different experiments.

3. The method according to claim 2, characterized in that The determining the expected path information corresponding to the request traffic in the preset branching tree model includes: Determine a target experimental node corresponding to the request flow in a leaf node in the preset diversion tree model; Based on the target experimental node, the preset diversion tree model is accessed level by level until the default domain node of the preset diversion tree model is accessed, and the expected path information corresponding to the request traffic in the preset diversion tree model is determined according to the visited node.

4. The method according to claim 3, characterized in that: The step of accessing the preset shunt tree model hierarchically based on the target experimental node until the default domain node of the preset shunt tree model is accessed, and determining the expected path information corresponding to the request traffic in the preset shunt tree model according to the accessed node includes: Taking the target experimental node as the first current node; Based on the first current node, access the parent node of the first current node in the preset branching tree model to obtain a hierarchical node corresponding to the first current node; Accessing the parent node of the hierarchical node corresponding to the first current node in the preset branching tree model to obtain the domain node corresponding to the first current node; Determining whether the domain node corresponding to the first current node is the default domain node; If the domain node corresponding to the first current node is not the default domain node, the domain node corresponding to the first current node is used as the new first current node, and the process returns to the step of accessing the parent node of the first current node in the preset diversion tree model to obtain the hierarchical node corresponding to the first current node, until it is determined that the domain node corresponding to the first current node is the default domain node, and based on the target experimental node and the domain nodes or hierarchical nodes accessed at each level, the expected path information corresponding to the request traffic in the preset diversion tree model is generated.

5. The method according to claim 2, characterized in that: The determining, according to the expected path information and the preset branching tree model, a grouping result of the request traffic in the target experiment includes: Taking the default domain node in the preset shunt tree model as the starting point, according to the hierarchical order of the preset shunt tree model, and according to the second current node where the request traffic is located in the current hierarchical order, the request traffic is shunt judged, at least one actual node to which the request traffic is shunted in the next hierarchical order is obtained, and the actual node of the next hierarchical order is compared with the expected node corresponding to the next hierarchical order in the expected path information; If the actual node does not include the expected node, determining that the grouping result of the request traffic in the target experiment is a control group; If the actual node includes the expected node and the actual node is the experimental node, determining that the grouping result of the request traffic in the target experiment is an experimental group; If the actual node includes the expected node and the actual node is not the experimental node, the actual node is used as the new second current node of the request traffic, and the steps of performing diversion judgment on the request traffic according to the second current node where the request traffic is located in the current hierarchical order are continued, obtaining at least one actual node to which the request traffic is diverted in the next hierarchical order, and comparing the actual node of the next hierarchical order with the expected node corresponding to the next hierarchical order in the expected path information.

6. The method according to claim 5, characterized in that According to the current node where the request traffic is located in the current hierarchical order, the request traffic is diverted and judged, and at least one actual node to which the request traffic is diverted in the next hierarchical order is obtained, including: If the second current node where the request traffic is located in the current hierarchical order is the hierarchical node, according to the node identifier of the hierarchical node and the request identifier of the request traffic, determine the actual node to which the request traffic is diverted in the next hierarchical order in the child node corresponding to the hierarchical node in the preset diversion tree model; If the second current node is the domain node, all child nodes corresponding to the domain node in the preset branching tree model are determined as actual nodes to which the request traffic is branched in the next level order.

7. The method according to claim 6, characterized in that The determining, according to the node identifier of the hierarchical node and the request identifier of the request traffic, in the child node corresponding to the hierarchical node in the preset shunt tree model, the actual node to which the request traffic is shunted in the next hierarchical order includes: Generate a target value based on the node identifier of the hierarchical node and the request identifier of the request flow; Processing the target value using a preset hash function to obtain a processing result; According to the processing result, the actual node to which the request traffic is diverted in the next level order is determined in the child node corresponding to the hierarchical node in the preset diversion tree model.

8. The method according to claim 1, characterized in that The performing an A / B experiment according to the grouping result of the request traffic includes: When the grouping result of the request traffic indicates that the request traffic is traffic in a control group in the target experiment, processing the request traffic according to a first experimental parameter corresponding to the control group; When the grouping result of the request traffic indicates that the request traffic is traffic in the experimental group in the target experiment, the request traffic is processed according to a second experimental parameter corresponding to the experimental group.

9. An A / B experimental device based on traffic diversion, characterized in that: include: An acquisition unit, used to acquire the request flow of a preset branching tree model and a target experiment; The preset traffic distribution tree model is used to guide the distribution of the request traffic; A determination unit, configured to determine expected path information corresponding to the request traffic in the preset branching tree model; The determining unit is further used to determine a grouping result of the request traffic in the target experiment according to the expected path information and the preset branching tree model, wherein the grouping result corresponding to the request traffic is used to indicate whether the request traffic is a traffic in a control group or a traffic in an experimental group in the target experiment; The experiment unit is used to perform an A / B experiment according to the grouping result of the request traffic.

10. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; The processor is configured to perform the method of any one of claims 1 to 8 by executing the executable instructions.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images