AB test system shunt verification method and device and computer equipment

Abstract

The application relates to an ABTest system shunting verification method and device and computer equipment. The method comprises the following steps: stratifying an experimental area, obtaining preset parameter default values of a control group and an experimental group of each experimental layer, and the number of the experimental layers is at least one; sending access traffic to the control group or the experimental group of the experimental layer, and returning a first parameter default value; based on the preset parameter default values, judging whether the first parameter default values inconsistent with the preset parameter default values exist, so as to verify whether the ABTest system shunting state is abnormal. By adopting the method, the data regression test workload can be reduced, the test result stability can be improved, and the verification process is simpler and more efficient.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automatic regression testing, in particular to an ABTest system shunt verification method and device and computer equipment. BACKGROUND

[0002] ABTest is a new web optimization method, which can be used to increase conversion rate, registration rate and other web indicators, and has gradually become the mainstream method of web optimization.

[0003] Osp-abt system reconstruction needs to calculate the experimental range of user input parameters according to experimental bucket data, and count the results of hitting or missing the corresponding experimental layers. However, the test verification range of this system reconstruction method is large, and the data regression test workload is large; and the stability of the system reconstruction engineering commonly used in the prior art is poor, and the accuracy of the results is not high.

[0004] Therefore, it is urgent to provide an ABTest system shunt verification method, device and computer equipment which can reduce the data regression test workload, improve the stability of the test results, and is simple and efficient. SUMMARY

[0005] Therefore, it is urgent to provide an ABTest system shunt verification method, device and computer equipment which can reduce the data regression test workload, improve the stability of the test results, and is simple and efficient.

[0006] In one aspect, an ABTest system shunt verification method is provided, which comprises:

[0007] Step A: stratifying the experimental area to obtain the preset parameter default value of the control group and the experimental group of each experimental layer, wherein the number of the experimental layers is at least one;

[0008] Step B: sending access traffic to the control group or the experimental group of the experimental layer, and returning the first parameter default value;

[0009] Step C: based on the preset parameter default value, determining whether there is a first parameter default value inconsistent with the preset parameter default value, to verify whether the ABTest system shunt state is abnormal.

[0010] In one embodiment, verifying whether the ABTest system shunt state is abnormal comprises: if there is no first parameter default value inconsistent with the preset parameter default value, determining that the ABTest system shunt state is normal; and if there is a first parameter default value inconsistent with the preset parameter default value, determining that the ABTest system shunt state is abnormal.

[0011] In one of the embodiments, when there is no first parameter default value inconsistent with the preset parameter default value, the method further comprises: presetting the proportion of the control group and the experimental group of the experimental layer in the traffic; obtaining the proportion of the traffic hitting the control group and the experimental group of the experimental layer; based on the experimental layer, comparing whether the proportion of the traffic hitting the control group of the experimental layer and the preset proportion of the control group in the traffic, and the proportion of the traffic hitting the experimental group of the experimental layer and the preset proportion of the experimental group in the traffic are consistent, respectively, to verify whether the ABTest system shunting state is abnormal.

[0012] In one of the embodiments, when verifying whether the ABTest system shunting state is abnormal, the method comprises: if the proportion of the traffic hitting the control group of the experimental layer and the preset proportion of the control group in the traffic are consistent, and the proportion of the traffic hitting the experimental group of the experimental layer and the preset proportion of the experimental group in the traffic are consistent, it is determined that the ABTest system shunting is normal; otherwise, it is determined that the ABTest system shunting is abnormal.

[0013] In one of the embodiments, when sending access traffic to the experimental group and the control group of the experimental layer, the method comprises: presetting the data type associated with each experimental layer; based on the data type of the access traffic, judging whether the access traffic belongs to the data type associated with the experimental layer layer by layer;

[0014] If the access traffic belongs to the data type associated with the experimental layer, the access traffic is sent to the control group or the experimental group of the experimental layer.

[0015] In one of the embodiments, presetting the proportion of the control group and the experimental group of the experimental layer in the traffic comprises: obtaining the total amount of the first parameter default value to obtain a first total amount; based on the first parameter default value, obtaining the total amount of the parameter default value consistent with the first parameter default value to obtain a second total amount; based on the quotient of the second total amount and the first total amount, obtaining the proportion of the traffic hitting the control group and the experimental group of the experimental layer.

[0016] In one of the embodiments, the data type comprises: new / old customers, regions, ages, genders, order quantities, order attributes, and / or member levels.

[0017] On the other hand, an ABTest system shunting verification device is provided, which comprises:

[0018] An experimental layering unit is configured to layer the experimental area, obtain the preset parameter default value of the control group and the experimental group of each experimental layer, and the number of the experimental layers is at least one;

[0019] A traffic distribution unit, which is in communication connection with the experiment layering unit, to send access traffic to the control group or the experimental group of the experiment layer and return a first parameter default value;

[0020] A checking unit, which is in communication connection with the traffic distribution unit, to judge whether there is a first parameter default value inconsistent with the preset parameter default value based on the first parameter default value, to check whether the ABTest system shunting state is abnormal.

[0021] In one of the embodiments, the traffic distribution unit comprises:

[0022] A judging module to judge whether the access traffic belongs to the data type associated with the experiment layer layer by layer based on the data type of the access traffic;

[0023] A shunting module to send the access traffic to the control group or the experimental group of the experiment layer if the access traffic belongs to the data type associated with the experiment layer.

[0024] In another aspect, a computer device is provided, which comprises a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor implements the following steps when executing the computer program:

[0025] Step A: layering the experiment area to obtain the preset parameter default value of the control group and the experimental group of each experiment layer, and the number of the experiment layers is at least one;

[0026] Step B: sending access traffic to the control group or the experimental group of the experiment layer and returning a first parameter default value;

[0027] Step C: judging whether there is a first parameter default value inconsistent with the preset parameter default value based on the preset parameter default value, to check whether the ABTest system shunting state is abnormal.

[0028] The ABTest system shunting verification method, device and computer equipment, the method comprises: stratifying an experimental area, obtaining preset parameter default values of a control group and an experimental group of each experimental layer, the number of the experimental layers is at least one; sending access traffic to the control group or the experimental group of the experimental layer, and returning a first parameter default value; based on the preset parameter default value, judging whether there is a first parameter default value inconsistent with the preset parameter default value to verify whether the ABTest system shunting state is abnormal. Based on the ABTest system shunting result, the ABTest system shunting process is deduced, the ABTest system or the ABTest shunting algorithm is regarded as a black box, and the shunting state of the ABTest system or the ABTest shunting algorithm is preliminarily judged. On the one hand, the state of the ABTest system and the ABTest shunting algorithm is monitored, and on the other hand, the regression test of the ABTest system shunting is realized. The method can reduce the data regression test workload, improve the stability of the test result, and the process of verifying the ABTest system or the ABTest shunting algorithm is simpler and more efficient. BRIEF DESCRIPTION OF DRAWINGS

[0029] Figure 1 An application environment diagram of the ABTest system shunting verification method in an embodiment;

[0030] Figure 2 A flowchart of the ABTest system shunting verification method in an embodiment;

[0031] Figure 3 A flowchart of the ABTest system shunting verification method in an embodiment;

[0032] Figure 4 A structural block diagram of the ABTest system shunting verification device in an embodiment;

[0033] Figure 5 An internal structure diagram of the computer equipment in an embodiment. DETAILED DESCRIPTION

[0034] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0035] Embodiment one

[0036] The ABTest system shunting verification method provided by the present application can be applied to, for example Figure 1The application environment is shown. Among them, the terminal 102 communicates with the server 104 through the network. Among them, the server can be a VQP platform, when the server monitors the data input from the VQP platform, the ABTest system shunting mechanism is automatically executed. The terminal 102 can be, but is not limited to, various personal computers, notebook computers, smart phones and tablet computers, and the server 104 can be implemented by an independent server or a server cluster composed of multiple servers.

[0037] In one embodiment, as shown in Figure 2 An ABTest system shunting verification method is provided, which is applied to the terminal in Figure 1 for example, including the following steps:

[0038] Step A: stratifying the experimental area, obtaining the preset parameter default value of the control group and the experimental group of each experimental layer, and the number of the experimental layers is at least one;

[0039] Among them, the ABTest system can exist as an independent system, and the ABTest system can perform several different experiments without interfering with each other, and different experiments have different configurations and may be directed to different user access traffic. That is, the ABTest system can include multiple experimental areas, each experimental area corresponds to a different experiment, and each experimental area can include multiple experimental layers, and each experimental layer includes a control group and an experimental group. It should be understood that in actual application scenarios, the experimental area can be stratified vertically or horizontally, and the experimental layers do not affect each other, and the user access traffic can pass through each experimental layer without damage until the user's access traffic hits the experimental layer associated with it. Preferably, in this embodiment, the experimental area is vertically stratified to obtain multiple experimental layers.

[0040] Obtaining the preset parameter default value of the control group and the experimental group of each experimental layer, the preset parameter default value including the first preset parameter default value set for the control group of each experimental layer and the second preset parameter default value set for the control group of each experimental layer; That is, the preset parameter default value includes the first parameter default value and the second parameter default value. Each experimental layer includes an experimental group and a control group.

[0041] Step B: sending access traffic to the control group or experimental group of the experimental layer, and returning the first parameter default value; That is, when sending access traffic to the control group or experimental group of the experimental layer, the first parameter default value is returned. Similarly, the first parameter default value includes the ninth parameter default value returned when sending access traffic to the control group of the experimental layer, and the tenth parameter default value returned when sending access traffic to the experimental group of the experimental layer. That is, the first parameter default value includes the ninth parameter default value and the tenth parameter default value.

[0042] Step C: Based on the preset parameter default value, it is judged whether there is a first parameter default value inconsistent with the preset parameter default value, to verify whether the ABTest system shunting state is abnormal.

[0043] If there is a preset parameter default value consistent with the first parameter default value, it is determined that the access traffic corresponding to the first parameter default value hits the control group or the experimental group of the experimental layer corresponding to the preset parameter default value; if there is no preset parameter default value consistent with the first parameter default value, it is determined that the access traffic corresponding to the first parameter default value does not hit the control group or the experimental group of the experimental layer corresponding to the preset parameter default value. In other words, if there is a preset parameter default value consistent with the first parameter default value, it is determined that the access traffic corresponding to the first parameter default value hits the control group or the experimental group corresponding to the preset parameter default value; if there is no preset parameter default value consistent with the first parameter default value, it is determined that the access traffic corresponding to the first parameter default value does not hit the control group or the experimental group corresponding to the preset parameter default value. Only when all the returned first parameter default values hit the control group or the experimental group of the experimental layer, it can be determined that the ABTest system shunting state is normal, even if there are a few returned first parameter default values that do not hit the control group or the experimental group of the experimental layer, it is determined that the ABTest system shunting state is abnormal.

[0044] In one embodiment, the experimental area is layered, the experimental layers are experimental layer A, experimental layer B and experimental layer C respectively, and the experimental layer A, the experimental layer B and the experimental layer C all include the control group and the experimental group. The preset parameter default values of the control group and the experimental group of the experimental layer A are respectively: if the control group of the experimental layer A is hit, then {DCSYAA#TextColor=black, DCSYAA#iconUrl=DefUrl}, if the experimental group of the experimental layer A is hit, then {DCSYAA#TextColor=red, DCSYAA#iconUrl=NewUrl}; if the control group of the experimental layer B is hit, then {DCSYAA#TextColor=black, DCSYAA#BackgroudColor=white}, if the experimental group of the experimental layer B is hit, then {DCSYAA#TextColor=red, DCSYAA#BackgroudColor=blue}; if the control group of the experimental group C is hit, then {DCSYAA#iconUrl=DefUrl}, if the experimental group of the experimental layer C is hit, then {DCSYAA#iconUrl=OldUrl}; for example, the result obtained at a certain time is {DCSYAA#TextColor=black, DCSYAA#iconUrl=DefUrl, DCSYAA#BackgroudColor=white}, because DCSYAA#TextColor and DCSYAA#iconUrl all hit the control group of the experimental group A, and DCSYAA#BackgroudColor gets the default value, so we can judge that this return combination is hit in the control group of the experimental group A. For example, the result obtained at a certain time is {DCSYAA#TextColor=red, DCSYAA#BackgroudColor=blue, DCSYAA#iconUrl=DefUrl}, because DCSYAA#TextColor and DCSYAA#BackgroudColor all get the experimental group of the experimental group B, and DCSYAA#iconUrl gets the default value, so we can judge that this return combination is hit in the experimental group of the experimental group B. For example, the result obtained at a certain time is {DCSYAA#TextColor=black, DCSYAA#BackgroudColor=white, DCSYAA#iconUrl=DefUrl}, all three parameters get the default value, so we can judge that this time does not hit any experiment. It should be understood that the setting of the parameter default values of the control group and the experimental group of each experimental layer can be based on actual conditions and set by those skilled in the art.With this method of returning parameter default value, all combinations of the first parameter default value that may appear can be listed, and based on the first parameter default value and the preset parameter default value, if the first parameter default value is inconsistent with the returned preset parameter default value, it is determined that the ABTest system shunting state is abnormal.

[0045] In the ABTest system shunting verification method, based on the ABTest system shunting result, it is determined whether the ABTest system shunting state is abnormal; and based on the ABTest system shunting result, the ABTest system shunting process is derived to preliminarily determine the cause of the ABTest system shunting abnormality.

[0046] In one embodiment, verifying whether the ABTest system shunting state is abnormal includes: if there is no first parameter default value inconsistent with the preset parameter default value, it is determined that the ABTest system shunting state is normal; if there is a first parameter default value inconsistent with the preset parameter default value, it is determined that the ABTest system shunting state is abnormal. That is, as long as there is a first parameter default value inconsistent with the preset parameter default value, it can be preliminarily determined that the ABTest system shunting state is abnormal.

[0047] In an embodiment, when there is no first parameter default value inconsistent with the preset parameter default value, the method further comprises: presetting the flow proportion occupied by the control group and the experimental group of the experimental layer; obtaining the flow proportion occupied by the control group and the experimental group hitting the experimental layer; based on the experimental layer, comparing whether the flow proportion of the control group hitting the experimental layer and the preset flow proportion, and the flow proportion of the experimental group hitting the experimental layer and the preset flow proportion are consistent, respectively, to verify whether the ABTest system shunting state is abnormal. For example: in an experimental scenario, experimental group A first divides 20% of the overall flow, so the proportion of hitting experimental group A is 20%, and because experimental group A is divided into experimental group and control group, their proportions are each 50%, so we can get more specific proportions, the experimental group of experimental group A occupies 10% of the overall flow, and the control group of experimental group A occupies 10% of the overall flow. Experimental group B is in 80% of the overall flow, and then only occupies 20% of the flow in the corresponding experimental layer, and its experimental group and control group are each 50%. Then we can get that the control group of experimental group B occupies 80%*20%*50%=8% of the overall flow, and the experimental group of experimental group B is also 8%. Experimental group C is similar to experimental group B, that is, the control group of experimental group C is 8%, and the experimental group of experimental group C is 8%. 100%-10%-10%-8%-8%-8%-8%=48%, the remaining 48% is the default value, that is, the part that does not hit any experimental layer. Finally, the flow proportion of the control group and the experimental group hitting each experimental layer can be obtained, on the one hand, the flow proportion of the control group and the experimental group hitting each experimental layer obtained can be compared with the preset flow proportion occupied by the control group and the experimental group of the experimental layer to determine whether the ABTest system shunting is abnormal; on the other hand, the calculated flow proportion sent to each experimental layer can be used as the preset flow proportion occupied by the control group and the experimental group sent to the experimental layer, and re-used as the input parameter of the ABTest system to determine whether the ABTest system shunting is abnormal.

[0048] Specifically, by calling the ABTest system in batches, the access traffic hitting the experiment layer, the control group and the experiment group is correspondingly taken as the access traffic sent to the experiment group, the control group and the experiment group, which on the one hand guarantees the integrity of data quality and on the other hand guarantees the sufficiency of data quantity. For example, if the access traffic allocated to the corresponding experiment layer is 30% of the access traffic of the entire experiment area, then correspondingly, if the access traffic hitting the corresponding experiment layer is 30% of the access traffic of the entire experiment area, it is determined that the access traffic of the user is consistent with the access traffic hitting the experiment layer; it can also include that the access traffic label of the user is consistent with the access traffic label hitting the experiment layer, such as classifying and labeling different experiment layers, based on the experiment layer, the access traffic of the user allocated to the corresponding experiment layer is consistently labeled, and if there is no user access traffic inconsistent with the layer experiment layer label in each experiment layer in the experiment area, it is determined that the access traffic of the user is consistent with the access traffic hitting the experiment layer.

[0049] In order to further determine whether the ABTest system shunting state is abnormal, the ABTest system shunting verification can be realized based on the ABTest system shunting result, i.e. the proportion of the traffic of the control group and the experiment group based on the experiment layer. It should be understood that the preset proportion of the traffic of the control group and the experiment group of the experiment layer can be obtained based on the previous ABTest system shunting verification result. That is, the determination of whether the ABTest system shunting is abnormal based on the proportion of the traffic of the control group and the experiment group based on the experiment layer can be realized by judging whether the ABTest system shunting verification results before and after are consistent. That is, the ABTest system shunting verification can be realized by using the calling mode in cycles, which guarantees sufficient data request quantity and ensures the integrity and sufficiency of the request base.

[0050] In one embodiment, when verifying whether the ABTest system shunting state is abnormal, the method comprises: if the proportion of the traffic of the control group hitting the experiment layer is consistent with the preset proportion of the traffic of the control group, and the proportion of the traffic of the experiment group hitting the experiment layer is consistent with the preset proportion of the traffic of the experiment group, it is determined that the ABTest system shunting is normal; otherwise, it is determined that the ABTest system shunting is abnormal.

[0051] It needs to be understood that the determination of whether the ABTest system shunting is abnormal can be based only on the preset parameter default value, and the check of whether the ABTest system shunting state is abnormal can be realized by judging whether there is a first parameter default value inconsistent with the preset parameter default value. It can also be based only on the flow proportion of the control group and the experimental group of the preset experiment layer, obtain the flow proportion of the control group and the experimental group of the hit experiment layer, and realize the check of whether the ABTest system shunting state is abnormal by comparing whether the flow proportion of the control group and the experimental group of the hit experiment layer is consistent with the flow proportion of the control group and the experimental group of the preset experiment layer. It can also be a combination of the above two technical solutions, that is, based on the preset parameter default and the first parameter default value, the flow proportion of the control group and the experimental group of the preset experiment layer and the flow proportion of the control group and the experimental group of the hit experiment layer, to realize the check of whether the ABTest system shunting is abnormal.

[0052] In one embodiment, when sending access traffic to the experimental group and the control group of the experiment layer, the method comprises: presetting the data type associated with each experiment layer; based on the data type of the access traffic, judging layer by layer whether the access traffic belongs to the data type associated with the experiment layer; if the access traffic belongs to the data type associated with the experiment layer, the access traffic is sent to the control group or the experimental group of the experiment layer. The data type includes: new / old customers, regions, ages, genders, order quantities, order attributes, and / or membership levels. For example: for online shopping, taking the purchase of household goods as an example, the ABTest system shunting check experiment is performed. The access traffic of users who purchase household goods in the whole country is selected for the experiment, and this part of user access traffic is divided into a control group and an experimental group according to a certain proportion. The users in the experimental group receive the test of the household goods rules, and the users in the control group remain unchanged; or the experiment can include a buy-two-get-one-free preferential promotion experiment, and the access traffic of users whose order data reaches a certain number is selected for the experiment. Then, this part of user access traffic is divided into a control group and an experimental group according to a certain proportion. The users in the experimental group can enjoy the buy-two-get-one-free preferential promotion, and the users in the control group remain unchanged.

[0053] It needs to be understood that the layer-by-layer judgment of whether the user belongs to the data type associated with the experiment layer can be a matching result obtained after traversing all experiment layers, or a matching result obtained in the process of traversing all experiment layers.

[0054] In one embodiment, presetting the proportion of the flow of the control group and the experimental group of the experimental layer includes: obtaining a total amount of the first parameter default value to obtain a first total amount; obtaining a total amount of the parameter default value consistent with the first parameter default value based on the first parameter default value to obtain a second total amount; and obtaining the proportion of the flow of the control group and the experimental group hitting the experimental layer based on the quotient of the second total amount and the first total amount; that is, the proportion of the flow of the control group and the experimental group hitting the experimental layer = the second total amount / the first total amount.

[0055] It should be understood that, although Figures 2-3 the steps in the flowchart of the application are shown in a sequence as indicated by the arrows, the steps are not necessarily executed in the order as indicated by the arrows. Unless otherwise specified herein, the execution of the steps is not strictly limited in sequence, and the steps can be executed in other sequences. Moreover, Figures 2-3 at least part of the steps in the flowchart of the application can include multiple sub-steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution sequence of the sub-steps or stages is not necessarily sequential, but can be executed in rotation or alternation with at least part of other steps or sub-steps or stages of other steps.

[0056] Embodiment two

[0057] In one embodiment, as Figure 4 shown, an ABTest system shunt verification device is provided, comprising: an experimental layering unit for layering an experimental area, obtaining preset parameter default values of a control group and an experimental group of each experimental layer, the number of the experimental layers being at least one;

[0058] a flow distribution unit, the flow distribution acquisition unit being in communication connection with the experimental layering unit to send access flow to the control group or the experimental group of the experimental layer and return a first parameter default value;

[0059] a verification unit, the verification unit being in communication connection with the flow distribution unit to judge whether there is a first parameter default value inconsistent with the preset parameter default value based on the first parameter default value, to verify whether the ABTest system shunt state is abnormal.

[0060] In one embodiment, the flow distribution unit includes: a judgment module for layer-by-layer judging whether the access flow belongs to the data type associated with the experimental layer based on the data type of the access flow;

[0061] a shunt module, if the access flow belongs to the data type associated with the experimental layer, the access flow is sent to the control group or the experimental group of the experimental layer.

[0062] In an embodiment, the apparatus further comprises:

[0063] a flow preset unit configured to preset a flow proportion occupied by the control group and the experiment group of the experiment layer;

[0064] a flow comparison unit configured to compare, based on the experiment layer, whether the access flow of the control group hitting the experiment layer and the preset flow proportion are consistent, and whether the access flow of the experiment group hitting the experiment layer and the preset flow proportion are consistent.

[0065] In an embodiment, the apparatus further comprises:

[0066] a parameter preset unit configured to preset a parameter default value of the control group and the experiment group of the experiment layer.

[0067] The specific limitations of the ABTest system shunt verification apparatus can be referred to the limitations of the ABTest system shunt verification method in the above, which will not be repeated here. Each module in the above ABTest system shunt verification apparatus can be realized by software, hardware and combinations thereof, in whole or in part. The above modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory in the computer device in software form, so as to call and execute the operations corresponding to each module by the processor.

[0068] Embodiment three

[0069] In an embodiment, a computer device is provided, which can be a terminal, and the internal structure diagram thereof can be as shown in Figure 5 The computer device comprises a processor, a memory, a network interface, a display screen and an input device connected through a system bus. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The network interface of the computer device is configured to communicate with external terminals through network connection. The computer program is executed by the processor to implement an ABTest system shunt verification method. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer overlaid on the display screen, or can be a key, trackball or touchpad arranged on the shell of the computer device, or can be an external keyboard, touchpad or mouse, etc.

[0070] Those skilled in the art can understand that, Figure 5The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0071] In one embodiment, a computer device is provided, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, and the processor implements the following steps when executing the computer program:

[0072] Step A: stratifying the experimental area, obtaining the preset parameter default value of the control group and the experimental group of each experimental layer, and the number of the experimental layers is at least one;

[0073] Step B: sending access traffic to the control group or the experimental group of the experimental layer, and returning the first parameter default value;

[0074] Step C: based on the preset parameter default value, determining whether there is a first parameter default value inconsistent with the preset parameter default value, to verify whether the ABTest system shunting state is abnormal; wherein the data type includes: new / old customer, region, age, gender, order quantity, order attribute and / or membership level.

[0075] In one embodiment, the processor further implements the following steps when executing the computer program:

[0076] If there is no first parameter default value inconsistent with the preset parameter default value, it is determined that the ABTest system shunting state is normal; if there is a first parameter default value inconsistent with the preset parameter default value, it is determined that the ABTest system shunting state is abnormal.

[0077] In one embodiment, the processor further implements the following steps when executing the computer program:

[0078] The preset proportion of the traffic of the control group and the experimental group of the experimental layer is obtained; based on the experimental layer, whether the proportion of the traffic hitting the control group of the experimental layer and the preset proportion of the traffic, and the proportion of the traffic hitting the experimental group of the experimental layer and the preset proportion of the traffic are consistent is compared respectively, to verify whether the ABTest system shunting state is abnormal.

[0079] In one embodiment, the processor further implements the following steps when executing the computer program:

[0080] If the proportion of the flow of the control group hitting the experimental layer is consistent with the preset proportion of the flow of the control group, and the proportion of the flow of the experimental group hitting the experimental layer is consistent with the preset proportion of the flow of the experimental group, it is determined that the ABTest system is normal; otherwise, it is determined that the ABTest system is abnormal.

[0081] In one embodiment, the processor, when executing the computer program, also implements the following steps:

[0082] The data type associated with each experimental layer is preset; whether the access flow belongs to the data type associated with the experimental layer is judged layer by layer based on the data type of the access flow; if the access flow belongs to the data type associated with the experimental layer, the access flow is sent to the control group or the experimental group of the experimental layer.

[0083] In one embodiment, the processor, when executing the computer program, also implements the following steps:

[0084] The total amount of the first parameter default value is obtained to obtain a first total amount; the total amount of the parameter default value consistent with the first parameter default value is obtained based on the first parameter default value to obtain a second total amount; and the proportion of the flow of the control group and the experimental group hitting the experimental layer is obtained based on the quotient of the second total amount and the first total amount.

[0085] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments. Any reference to memory, storage, database or other medium used in the embodiments provided by the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0086] Any combination of the technical features in the above embodiments can be made, and for the sake of brevity, not all possible combinations are described above, however, as long as the combination of the technical features does not exist in contradiction, it shall be considered within the scope of the present disclosure.

[0087] The above embodiments only express several implementation manners of the present application, and the description is relatively specific and detailed, but it shall not be understood as a limitation on the patent scope of the present application. It shall be pointed out that, for ordinary skilled persons in the art, several modifications and improvements can be made without departing from the concept of the present application, and these shall be within the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the appended claims.

Claims

1. An ABTest system shunt verification method, characterized in that, The method comprises: stratifying the experimental area, obtaining preset parameter default values of the control group and the experimental group of each experimental layer, the number of the experimental layers being at least one, each experimental layer comprising the control group and the experimental group, the preset parameter default values comprising a first preset parameter default value set for the control group of each experimental layer and a second preset parameter default value set for the control group of each experimental layer; sending access traffic to the control group or the experimental group of the experimental layer and returning the first parameter default value; judging whether there is a first parameter default value inconsistent with the preset parameter default value based on the preset parameter default value to verify whether the ABTest system shunting state is abnormal; if there is no first parameter default value inconsistent with the preset parameter default value, it is determined that the ABTest system shunting state is normal; if there is a first parameter default value inconsistent with the preset parameter default value, it is determined that the ABTest system shunting state is abnormal.

2. The AB Test system shunt verification method of claim 1, wherein, When there is no first parameter default value inconsistent with the preset parameter default value, the method further comprises: presetting the traffic proportion occupied by the control group and the experimental group of the experimental layer; obtaining the traffic proportion hitting the control group and the experimental group of the experimental layer; comparing whether the traffic proportion hitting the control group of the experimental layer and the preset traffic proportion and the traffic proportion hitting the experimental group of the experimental layer and the preset traffic proportion are consistent based on the experimental layer to verify whether the ABTest system shunting state is abnormal.

3. The AB Test system shunt verification method of claim 2, wherein, When verifying whether the ABTest system shunting state is abnormal, the method comprises: if the traffic proportion hitting the control group of the experimental layer and the preset traffic proportion occupied by the control group are consistent, and the traffic proportion hitting the experimental group of the experimental layer and the preset traffic proportion occupied by the experimental group are consistent, it is determined that the ABTest system shunting is normal; otherwise, it is determined that the ABTest system shunting is abnormal.

4. The AB Test system shunt verification method of claim 3, wherein, Presetting the traffic proportion occupied by the control group and the experimental group of the experimental layer comprises: obtaining the total amount of the first parameter default value to obtain a first total amount; obtaining the total amount of the parameter default value consistent with the first parameter default value based on the first parameter default value to obtain a second total amount; obtaining the traffic proportion hitting the control group and the experimental group of the experimental layer based on the quotient of the second total amount and the first total amount.

5. The AB Test system shunt verification method according to any one of claims 1 to 4, characterized in that, When sending access traffic to the experimental group and the control group of the experimental layer, the method comprises: presetting the data type associated with each experimental layer; judging whether the access traffic belongs to the data type associated with the experimental layer layer by layer based on the data type of the access traffic; if the access traffic belongs to the data type associated with the experimental layer, the access traffic is sent to the control group or the experimental group of the experimental layer.

6. The AB Test system shunt verification method of claim 5, wherein, The data type comprises: new / old customers, regions, ages, genders, order quantities, order attributes, and / or membership levels.

7. An apparatus for implementing the method of claim 1 to 6, wherein, The device comprises: an experimental stratification unit for stratifying the experimental area, obtaining preset parameter default values of the control group and the experimental group of each experimental layer, the number of the experimental layers being at least one; A traffic distribution unit, which is in communication connection with the experiment layering unit, to send access traffic to the control group or the experimental group of the experiment layer and return a first parameter default value; A checking unit, which is in communication connection with the traffic distribution unit, to judge whether there is a first parameter default value inconsistent with the preset parameter default value based on the first parameter default value, to check whether the ABTest system shunting state is abnormal.

8. The AB Test system shunt verification apparatus of claim 7, wherein, The traffic distribution unit comprises: A judging module, which is based on the data type of the access traffic, to judge whether the access traffic belongs to the data type associated with the experiment layer layer by layer; A shunting module, which sends the access traffic to the control group or the experimental group of the experiment layer if the access traffic belongs to the data type associated with the experiment layer.

9. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to realize the steps of the method in any one of claims 1 to 6.

Images