RTA multi-layer test method, device, computer equipment and storage medium

Abstract

The present application relates to an RTA multi-layer test method, device, computer equipment and storage medium, and belongs to the field of advertising delivery technology. The present application can form a control group and an experimental group by bucketing by using limited user data provided by the media, and determine the number of experimental groups and the value of adjustment parameters in each test layer according to the number of experimental groups. The user data in the experimental group of the second test layer is obtained by performing hash calculation and bucketing on the user data in the experimental group of the first test layer, that is, the user data in the experimental group of the subsequent test layer is obtained by performing hash calculation and bucketing on the user data in the experimental group of the previous test layer, which can realize the construction of the number of experimental groups on demand, and the control group always remains unchanged, ensuring the accuracy of the test results, and can do multiple groups of experiments at the same time, which improves the efficiency of parameter adjustment experiments.

Description

Technical Field

[0001] The present application relates to the technical field of advertisement placement, and in particular to an RTA multi-layer test method, device, computer equipment and storage medium. Background Art

[0002] In the field of advertising, RTA refers to the real-time screening and bidding intervention of advertising traffic by advertisers through API interfaces. RTA is the abbreviation of Real Time API. In the targeting link, the request for customer identity identification is sent to the advertiser to screen the customer to meet the advertiser's real-time personalized delivery needs. The system on the advertiser side provides an RTA service to the media with an RTA request interface. The media initiates an RTA request to the advertiser based on the advertising space information, asking the advertiser whether a certain user participates in the bidding for the advertising space.

[0003] With the refined operation and intelligent upgrade of the Internet advertising market, the application of RTA (Real Time API) real-time interface in advertising has increasingly highlighted its key value. This technological innovation realizes the real-time bidding purchase of advertising inventory, allowing advertisers to instantly analyze user characteristics, evaluate advertising value, and quickly make bidding decisions every time an impression opportunity arises. Through the RTA real-time interface, the advertiser's system can capture user data streams in real time and accurately locate the target population, thereby improving the relevance and conversion rate of advertising, optimizing media resource allocation, and reducing ineffective consumption. RTA technology not only greatly improves advertising effectiveness, but also builds a transparent, fair and efficient trading environment for the advertising ecosystem. It is a powerful engine that drives the modern digital marketing industry to continue to move forward.

[0004] However, when advertisers use RTA to screen and bid for a certain audience, they do not blindly exclude or bid. Instead, they adjust parameters to conduct comparative experiments. They need to use the results of the experimental and control groups to analyze the experimental group data to obtain parameter values ​​that meet the bidding strategy requirements. However, the amount of data that the media can provide for comparative experiments is small, which can only meet the data volume of one experiment, and it is difficult to meet the data volume for multiple experiments at the same time. Only one experiment can be conducted at the same time, and multiple experiments cannot be conducted at the same time, which greatly reduces the efficiency of parameter adjustment experiments. Summary of the invention

[0005] Based on this, an RTA multi-layer test method, device, computer equipment and storage medium are provided to solve the technical problem that when the current advertising traffic is screened and bids are intervened in real time, the amount of data that the media can provide for comparative experiments through parameter adjustment is small, resulting in the inability to conduct multiple groups of experiments at the same time, which greatly reduces the efficiency of parameter adjustment experiments.

[0006] In one aspect, a RTA multilayer test method is provided, the method comprising:

[0007] Obtain user data provided by the media, and divide the user data into multiple buckets, wherein the amount of user data in all buckets is the same, the buckets are provided with bucket address numbers, each of the buckets contains multiple user data, and each user data is associated with the bucket address number of the bucket in which it is located;

[0008] Obtaining the types of comparative experiments to be performed, obtaining adjustment parameters of user data in each type of comparative experiment, and determining the number of test layers for each type of comparative experiment according to the number of types of the adjustment parameters;

[0009] Select two buckets as control groups for all experimental layers, split the remaining buckets into two buckets each as an experimental group, obtain the number of the experimental groups, and determine the number of experimental groups in each experimental layer according to the number of the experimental groups;

[0010] The control group and the experimental group are used to form a first test layer, the value of the adjustment parameter of the experimental group in the first test layer is set, and a reasonable value of the adjustment parameter in the first test layer is obtained by comparing the test results;

[0011] Performing a hash operation on the user data in the experimental group of the first test layer to calculate the bucket address numbers of the buckets, re-bucketing the user data according to the calculated bucket address numbers to form the experimental group of the second test layer, setting the value of the adjustment parameter of the experimental group in the second test layer, and obtaining a reasonable value of the adjustment parameter in the second test layer by comparing the test results;

[0012] Determine whether the comparative test of all types of adjustment parameters is completed. If so, output the reasonable value of the adjustment parameter in the second test layer. If not, repeat the previous step until all types of comparative tests are completed.

[0013] In one embodiment, the step of re-performing a hash operation on the user data in the experimental group of the first experimental layer to calculate the bucket address numbers of the buckets, and re-bucketing the user data according to the calculated bucket address numbers to form the experimental group of the second experimental layer includes:

[0014] Acquire user data in the experimental group of the first experimental layer;

[0015] Performing a hash operation on the user data;

[0016] The last digit after the hash operation is taken as the bucket address number, and the user data is re-bucketed according to the calculated bucket address number to form the experimental group of the second test layer.

[0017] In one embodiment, the step of performing a hash operation on the user data includes:

[0018] Obtain the device number of the device used by the user corresponding to the user data;

[0019] Obtain a fixed character string preset corresponding to the second test layer, and add the fixed character string to the device number of the user device corresponding to all user data to form a first character string;

[0020] Using the MD5 message digest algorithm to obtain a first hash value corresponding to the first character string;

[0021] A hash operation is performed on the first hash value, and a modulo 10 is calculated.

[0022] In one of the embodiments, the formula for performing a hash operation on the user data is Hash[Md5(device+fixed string)]%10;

[0023] The device is the device number of the device used by the user in the user data;

[0024] The fixed character string is a character string preset for the corresponding test layer, and the fixed character strings of all test layers are set to be different;

[0025] Md5 is used to generate a 128-bit hash value;

[0026] The hash algorithm uses the CRC32 algorithm;

[0027] %10 is the remainder of 10.

[0028] In one embodiment, the steps of obtaining the types of comparative experiments to be performed, obtaining adjustment parameters of user data in each type of comparative experiments, and determining the number of test layers for each type of comparative experiments according to the number of types of adjustment parameters include:

[0029] Obtain the type of comparative experiment to be performed, including exclusion experiment and bidding experiment;

[0030] Obtain adjustment parameters of user data in each type of comparative experiment, wherein the adjustment parameters include a bid coefficient, a user selection willingness score, and a user qualification score;

[0031] The number of types of the adjustment parameters of each type of comparative experiment is obtained, and the number of test layers of each type of comparative experiment is determined according to the number of types of the adjustment parameters.

[0032] In one embodiment, the step of determining whether the comparative test of all types of adjustment parameters is completed, and if so, outputting the reasonable value of the adjustment parameter in the second test layer, and if not, repeating the previous step until all types of comparative tests are completed includes:

[0033] If the comparative test of all types of adjustment parameters has not been completed, the user data in the experimental group of the current experimental layer is obtained;

[0034] Perform hash operation on the user data in the experimental group of the current experimental layer to calculate the bucket address number of the bucket;

[0035] Re-bucketing the user data according to the calculated bucket address numbers to form an experimental group of the next experimental layer, and setting the value of the adjustment parameter of the experimental group in the next experimental layer;

[0036] Obtaining reasonable values ​​of the adjustment parameters by comparing the test results;

[0037] Determine whether the comparative tests of all types of adjustment parameters are completed, if so, output the reasonable values ​​of the adjustment parameters, if not, perform the remaining comparative tests according to the adjustment parameters of the uncompleted comparative tests.

[0038] In one embodiment, after the step of outputting the reasonable numerical value of the adjustment parameter, the method further includes:

[0039] Acquire the reasonable value of the output adjustment parameter, adjust the RTA policy parameter according to the reasonable value of the output adjustment parameter, and bid for the user data according to the adjusted RTA policy.

[0040] On the other hand, there is provided an RTA multilayer test device, the device comprising:

[0041] A user data bucketing module is used to obtain user data provided by the media, and divide the user data into multiple buckets. The number of user data in all buckets is the same. The buckets are provided with bucket address numbers. Each bucket contains multiple user data, and each user data is associated with the bucket address number of the bucket in which it is located;

[0042] An experiment quantity management module is used to obtain the types of comparative experiments to be performed, obtain adjustment parameters of user data in each type of comparative experiment, and determine the number of experimental layers for each type of comparative experiment according to the number of types of adjustment parameters;

[0043] The control group and experimental group setting module is used to select two buckets as the control group of all experimental layers, split the remaining buckets into two buckets each as an experimental group, obtain the number of the experimental groups, and determine the number of experimental groups in each experimental layer according to the number of the experimental groups;

[0044] A first test layer module, used to form a first test layer using the control group and the experimental group, set the value of the adjustment parameter of the experimental group in the first test layer, and obtain a reasonable value of the adjustment parameter in the first test layer by comparing the test results;

[0045] The second test layer module is used to perform a hash operation on the user data in the experimental group of the first test layer to calculate the bucket address number of the bucket, re-bucket the user data according to the calculated bucket address number to form the experimental group of the second test layer, set the value of the adjustment parameter of the experimental group in the second test layer, and obtain the reasonable value of the adjustment parameter in the second test layer by comparing the test results;

[0046] The test termination control module is used to determine whether the comparative test of all types of adjustment parameters is completed. If so, the reasonable value of the adjustment parameter in the second test layer is output. If not, the previous step is repeated until all types of comparative experiments are completed.

[0047] In another aspect, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the following steps are implemented:

[0048] Obtain user data provided by the media, and divide the user data into multiple buckets, wherein the amount of user data in all buckets is the same, the buckets are provided with bucket address numbers, each of the buckets contains multiple user data, and each user data is associated with the bucket address number of the bucket in which it is located;

[0049] Obtaining the types of comparative experiments to be performed, obtaining adjustment parameters of user data in each type of comparative experiment, and determining the number of test layers for each type of comparative experiment according to the number of types of the adjustment parameters;

[0050] Select two buckets as control groups for all experimental layers, split the remaining buckets into two buckets each as an experimental group, obtain the number of the experimental groups, and determine the number of experimental groups in each experimental layer according to the number of the experimental groups;

[0051] The control group and the experimental group are used to form a first test layer, the value of the adjustment parameter of the experimental group in the first test layer is set, and a reasonable value of the adjustment parameter in the first test layer is obtained by comparing the test results;

[0052] Performing a hash operation on the user data in the experimental group of the first test layer to calculate the bucket address numbers of the buckets, re-bucketing the user data according to the calculated bucket address numbers to form the experimental group of the second test layer, setting the value of the adjustment parameter of the experimental group in the second test layer, and obtaining a reasonable value of the adjustment parameter in the second test layer by comparing the test results;

[0053] Determine whether the comparative test of all types of adjustment parameters is completed. If so, output the reasonable value of the adjustment parameter in the second test layer. If not, repeat the previous step until all types of comparative tests are completed.

[0054] In another aspect, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0055] Obtain user data provided by the media, and divide the user data into multiple buckets, wherein the amount of user data in all buckets is the same, the buckets are provided with bucket address numbers, each of the buckets contains multiple user data, and each user data is associated with the bucket address number of the bucket in which it is located;

[0056] Obtaining the types of comparative experiments to be performed, obtaining adjustment parameters of user data in each type of comparative experiment, and determining the number of test layers for each type of comparative experiment according to the number of types of the adjustment parameters;

[0057] Select two buckets as control groups for all experimental layers, split the remaining buckets into two buckets each as an experimental group, obtain the number of the experimental groups, and determine the number of experimental groups in each experimental layer according to the number of the experimental groups;

[0058] The control group and the experimental group are used to form a first test layer, the value of the adjustment parameter of the experimental group in the first test layer is set, and a reasonable value of the adjustment parameter in the first test layer is obtained by comparing the test results;

[0059] Performing a hash operation on the user data in the experimental group of the first test layer to calculate the bucket address numbers of the buckets, re-bucketing the user data according to the calculated bucket address numbers to form the experimental group of the second test layer, setting the value of the adjustment parameter of the experimental group in the second test layer, and obtaining a reasonable value of the adjustment parameter in the second test layer by comparing the test results;

[0060] Determine whether the comparative test of all types of adjustment parameters is completed. If so, output the reasonable value of the adjustment parameter in the second test layer. If not, repeat the previous step until all types of comparative tests are completed.

[0061] The above-mentioned RTA multi-layer test method, apparatus, computer equipment and storage medium form a control group and an experimental group by bucketing by utilizing the limited user data provided by the media, and determine the number of experimental groups in each test layer and the value of the adjustment parameter according to the number of experimental groups. The user data in the experimental group of the second test layer is obtained by performing a hash operation and bucketing on the user data in the experimental group of the first test layer, that is, the user data in the experimental group of the subsequent test layer is obtained by performing a hash operation and bucketing on the user data in the experimental group of the previous test layer. The number of experimental groups can be constructed on demand, and the control group always remains unchanged, thereby ensuring the accuracy of the test results, being able to conduct multiple groups of experiments at the same time, and improving the efficiency of parameter adjustment experiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0063] Figure 1 This is an application environment diagram of the RTA multi-layer test method in one embodiment of the present application;

[0064] Figure 2 This is a schematic diagram of a process of an RTA multi-layer test method in one embodiment of the present application;

[0065] Figure 3 A flowchart of the steps of re-performing a hash operation on the user data in the experimental group of the first test layer to calculate the bucket address numbers of the buckets, and re-bucketing the user data according to the calculated bucket address numbers to form the experimental group of the second test layer in one embodiment of the present application;

[0066] Figure 4 A flowchart of the steps of obtaining the types of comparative experiments to be performed, obtaining adjustment parameters of user data in each type of comparative experiment, and determining the number of test layers for each type of comparative experiment according to the number of types of the adjustment parameters in one embodiment of the present application;

[0067] Figure 5 This is a schematic diagram of a structure for forming a test layer structure in one embodiment of the present application;

[0068] Figure 6 A flowchart of determining whether comparative tests of all types of adjustment parameters are completed in one embodiment of the present application, and if so, outputting reasonable values ​​of the adjustment parameters in the second test layer, and if not, repeating the previous step until all types of comparative tests are completed;

[0069] Figure 7 This is a structural block diagram of an RTA multi-layer test device in one embodiment of the present application;

[0070] Figure 8 This is a diagram of the internal structure of a computer device in one embodiment of the present application. DETAILED DESCRIPTION

[0071] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0072] The RTA multilayer test method provided in this application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the media advertising platform 103 through the network, and the media advertising platform 103 communicates with the advertiser server 104 through the network. The media advertising platform 103 is referred to as media. Among them, the terminal 102 can be but not limited to various personal computers, laptops, smart phones, tablet computers and portable wearable devices. Customers can browse advertising information such as Toutiao and Tencent through the terminal 102. The media advertising platform 103 includes servers such as Toutiao and Tencent. The media advertising platform 103 and the advertiser server 104 can be implemented by an independent server or a server cluster composed of multiple servers.

[0073] When advertisers use RTA to screen and bid for the crowd, they do not blindly exclude or bid, but conduct comparative experiments. They need to use the results of the experimental group (data with added exclusion and bidding strategy interference) and the control group (pure data without any interference) to analyze whether the data of our experimental group meets the bidding strategy requirements. The premise of doing data experiments is that it is necessary to make fixed groups for each person's user data. At this time, the concept of buckets will be used. The media will divide the user data into 10 bucket numbers evenly. The media will divide each user's data into fixed buckets, that is, each request for the user data of the same person will be a fixed bucket address number (ID), and it is difficult to re-obtain a new bucket address number for bucketing. For example, if a user's device number idfa=abcde12fgh, and the corresponding bucket address number is 1, then the group to which this user belongs must be in the bucket with bucket address number 1, and no changes will be made.

[0074] When doing experiments, we also make strategies based on different bucket address numbers. However, the media only provides 10 buckets of data, which can only meet one tenth of the total demand. For these 10 data, the current practice is to take out 2 of them as the control group without any interference. The remaining 8 data are used as the experimental group. These 8 data need to be used for both exclusion and bidding experiments. However, for both exclusion and bidding experiments, only one experiment can be done at the same time, and multiple experiments cannot be done at the same time, which greatly reduces the efficiency of parameter adjustment experiments.

[0075] In one embodiment, Figure 2 As shown, a RTA multi-layer test method is provided, which is applied to Figure 1 Taking the advertiser server 104 in FIG. 1 as an example, the following steps are included:

[0076] Step S1, obtaining user data provided by the media, dividing the user data into multiple buckets, the number of user data in all buckets is the same, the buckets are provided with bucket address numbers, each of the buckets contains multiple user data, and each user data is associated with the bucket address number of the bucket in which it is located;

[0077] Step S2, obtaining the types of comparative experiments to be performed, obtaining adjustment parameters of user data in each type of comparative experiment, and determining the number of test layers for each type of comparative experiment according to the number of types of the adjustment parameters;

[0078] Step S3, selecting two buckets as control groups for all test layers, splitting the remaining buckets into two buckets each as an experimental group, obtaining the number of the experimental groups, and determining the number of experimental groups in each test layer according to the number of the experimental groups;

[0079] Step S4, using the control group and the experimental group to form a first test layer, setting the value of the adjustment parameter of the experimental group in the first test layer, and obtaining a reasonable value of the adjustment parameter in the first test layer by comparing the test results;

[0080] Step S5, performing a hash operation on the user data in the experimental group of the first test layer to calculate the bucket address numbers of the buckets, re-bucketing the user data according to the calculated bucket address numbers to form the experimental groups of the second test layer, setting the values ​​of the adjustment parameters of the experimental groups in the second test layer, and obtaining reasonable values ​​of the adjustment parameters in the second test layer by comparing the test results;

[0081] Step S6, determining whether the comparative test of all types of adjustment parameters is completed, if so, outputting the reasonable values ​​of the adjustment parameters in the second test layer, if not, repeating the previous step until all types of comparative tests are completed.

[0082] The present application utilizes limited user data provided by the media to form a control group and an experimental group through bucketing, and determines the number of experimental groups in each test layer and the values ​​of adjustment parameters according to the number of experimental groups. The user data in the experimental group of the second test layer is obtained by performing a hash operation and bucketing on the user data in the experimental group of the first test layer, that is, the user data in the experimental group of the subsequent test layer is obtained by performing a hash operation and bucketing on the user data in the experimental group of the previous test layer. The number of experimental groups can be constructed on demand, and the control group always remains unchanged, thereby ensuring the accuracy of the test results, being able to conduct multiple groups of experiments at the same time, and improving the efficiency of parameter adjustment experiments.

[0083] like Figure 3As shown, in this embodiment, the steps of re-performing a hash operation on the user data in the experimental group of the first test layer to calculate the bucket address numbers of the buckets, and re-bucketing the user data according to the calculated bucket address numbers to form the experimental group of the second test layer include:

[0084] Step S51, obtaining user data in the experimental group of the first experimental layer;

[0085] Step S52, performing a hash operation on the user data;

[0086] Step S53, taking the last digit after the hash operation as the bucket address number, and re-bucketing the user data according to the calculated bucket address number to form an experimental group of the second test layer.

[0087] In this embodiment, the step of performing a hash operation on the user data includes:

[0088] Obtain the device number of the device used by the user corresponding to the user data;

[0089] Obtain a fixed character string preset corresponding to the second test layer, and add the fixed character string to the device number of the user device corresponding to all user data to form a first character string;

[0090] Using the MD5 message digest algorithm to obtain a first hash value corresponding to the first character string;

[0091] A hash operation is performed on the first hash value, and a modulo 10 is calculated.

[0092] In this embodiment, the formula for performing a hash operation on the user data is Hash[Md5(device+fixed string)]%10;

[0093] The device is the device number of the device used by the user in the user data;

[0094] The fixed character string is a character string preset for the corresponding test layer, and the fixed character strings of all test layers are set to be different;

[0095] Md5 is used to generate a 128-bit (16-byte) hash value;

[0096] The hash algorithm uses the CRC32 algorithm;

[0097] %10 is the remainder of 10.

[0098] It is understandable that different fixed character strings are set for all test layers, and after the fixed character string is added to the device number of the user device corresponding to all user data, a different character string is formed corresponding to each test layer, so that the hash value obtained by using the MD5 information digest algorithm is different, and the bucket address number obtained after the hash operation is different, which overcomes the problem that the user data of the same person will have a fixed bucket address number each time it is requested, and it is difficult to re-obtain a new bucket address number for bucketing, thereby realizing the on-demand construction of experimental groups using limited user data.

[0099] In addition, the method of calculating the bucket address number by taking the modulo 10 of the calculated value of the Hash algorithm can ensure that the data of the experimental group will be evenly distributed to the lower-level experimental groups.

[0100] like Figure 4 As shown, in this embodiment, the steps of obtaining the types of comparative experiments to be performed, obtaining adjustment parameters of user data in each type of comparative experiments, and determining the number of test layers for each type of comparative experiments according to the number of types of the adjustment parameters include:

[0101] Step S21, obtaining the type of comparative experiment to be performed, the types of comparative experiments to be performed include exclusion experiments and bidding experiments;

[0102] Step S22, obtaining adjustment parameters of user data in each type of comparative experiment, wherein the adjustment parameters include bid coefficient, user selection willingness score, and user qualification score;

[0103] Step S23, obtaining the number of types of the adjustment parameters for each type of comparative experiment, and determining the number of test layers for each type of comparative experiment according to the number of types of the adjustment parameters.

[0104] It is understandable that if Figure 5 As shown, an experimental layer structure is formed according to the type of comparative experiment, the number of experimental layers, and the number of types of adjustment parameters. The values ​​of the adjustment parameters of each experimental group are set accordingly in the experimental layer structure. The values ​​of the adjustment parameters of the experimental groups in each experimental layer can be set at fixed intervals. Figure 5 The arrows in the figure indicate the flow direction of user data.

[0105] like Figure 6 As shown, in this embodiment, the step of determining whether the comparative test of all types of adjustment parameters is completed, if yes, outputting the reasonable value of the adjustment parameter in the second test layer, and if no, repeating the previous step until all types of comparative tests are completed includes:

[0106] Step S61, if the comparison test of all types of adjustment parameters has not been completed, then obtain the user data in the experimental group of the current test layer;

[0107] Step S62, performing a hash operation on the user data in the experimental group of the current test layer to calculate the bucket address number of the bucket;

[0108] Step S63, re-bucketing the user data according to the calculated bucket address numbers to form an experimental group of the next experimental layer, and setting the value of the adjustment parameter of the experimental group in the next experimental layer;

[0109] Step S64, obtaining a reasonable value of the adjustment parameter by comparing the test results;

[0110] Step S65, determining whether the comparison test of all types of adjustment parameters is completed, if so, outputting the reasonable value of the adjustment parameter, if not, performing the remaining comparison tests according to the adjustment parameters for which the comparison test is not completed.

[0111] It is understandable that the experimental group in the next test layer is obtained by re-calculating the buckets of the experimental group in the previous test layer through hash operations, ensuring that the traffic data of the user data of the experimental group in each layer is randomly executed downward without any manual processing. The random generation of bucket address numbers ensures the quality of the experimental group. In addition, the adjustment parameters in the previous test layer will be transferred to the next test layer, realizing the comprehensive adjustment of the adjustment parameters, and obtaining reasonable values ​​of the adjustment parameters based on the comparison of test results, which can not only comprehensively adjust the values ​​of all types of adjustment parameters, but also clearly know the test results.

[0112] like Figure 2 As shown, in this embodiment, after the step of outputting the reasonable numerical value of the adjustment parameter, the following steps are also included:

[0113] Step S7, obtaining the reasonable value of the output adjustment parameter, adjusting the RTA policy parameter according to the reasonable value of the output adjustment parameter, and bidding for the user data according to the adjusted RTA policy.

[0114] The experimental details are:

[0115] like Figure 5 As shown, the business personnel used the user data of 10 buckets issued by the media to conduct the first-level experiment in the first test layer. The experiment mainly verified the exclusion of user logic, and used the user data in 2 buckets issued by the media as the control group without any interference. The remaining 8 buckets were divided into 4 groups. The data of the first group will have a bid coefficient of 0.6 for the corresponding willingness score of 0-50, the bid coefficient of 1 for the second group with a willingness score of 50-70, the bid coefficient of 1 for the third group with a willingness score of 70-80, and the bid coefficient of 1.2 for the fourth group with a score greater than 80.

[0116] Because all bucket address numbers are issued by the media, each device number will only appear in the same bucket, ensuring the uniqueness of the experiment.

[0117] At this time, it is necessary to conduct experiments on the user's qualification points, but the first test layer has been occupied and no more data can be allocated for business experiments, so it is necessary to divide the data into buckets based on the device number. The second layer of experiments is based on the experimental data of the 8 buckets in the first test layer. The 4 groups of data are used to conduct experiments on the qualification points. The bid coefficient for qualification points between 0-50 is 0.5, the bid coefficient for 50-60 is set to 0.7, the bid coefficient for 60-70 is set to 1, the bid coefficient for 70-80 is set to 1.2, and the bid coefficient for 80 points and above is set to 1.5.

[0118] The final bid coefficient is determined by the two layers of coefficients.

[0119] Multi-layer experiments were also performed in a similar manner.

[0120] For Hash[Md5(device+fixed string)]%10:

[0121] For example, device = 27cedb28f7eefac63e54d0229d8f12b2;

[0122] Fixed string for the first test layer = first;

[0123] Md5(27cedb28f7eefac63e54d0229d8f12b2first)=

[0124] 0ed1f9b33692ab2d5e290c1cc19d6922;

[0125] Hash(0ed1f9b33692ab2d5e290c1cc19d6922)=3285379787;

[0126] Fixed string for the second test layer = second;

[0127] Md5(27cedb28f7eefac63e54d0229d8f12b2second)=

[0128] bd96158c76721c9b605d82d0ad022acf;

[0129] Hash(bd96158c76721c9b605d82d0ad022acf)=2168330004.

[0130] From the above, it can be seen that the hash values ​​calculated for the same device number after adding different fixed strings are inconsistent. After taking the modulo 10, the last digit is used as the bucket address number. The bucket address number of the first test layer is 7, and the bucket address number of the second test layer is 4.

[0131] The hash algorithm uses the CRC32 algorithm. CRC32 (Cyclic Redundancy Check) is a cyclic redundancy check algorithm, commonly used for error detection in data transmission. It calculates the check value of the data to detect whether the data has been erroneous or tampered with during the transmission process.

[0132] The characteristics of the CRC32 algorithm are as follows:

[0133] 1. Based on binary operations: The CRC32 algorithm performs calculations on binary data and regards the data as a sequence of binary bits.

[0134] 2. Polynomial division: CRC32 uses polynomial division to calculate the check value. It divides the data with a predefined generating polynomial and takes the remainder as the check value.

[0135] 3. Circular shift operation: The CRC32 algorithm uses a circular shift operation to process each bit of the data.

[0136] 4. XOR operation: The CRC32 algorithm uses XOR operation to calculate the intermediate results and the final check value.

[0137] In the above-mentioned RTA multi-layer test method, by utilizing the limited user data provided by the media, a control group and an experimental group are formed by bucketing. The number of experimental groups in each test layer and the values ​​of the adjustment parameters are determined according to the number of experimental groups. The user data in the experimental group of the second test layer is obtained by performing a hash operation and bucketing on the user data in the experimental group of the first test layer, that is, the user data in the experimental group of the subsequent test layer is obtained by performing a hash operation and bucketing on the user data in the experimental group of the previous test layer. The number of experimental groups can be constructed on demand, and the control group always remains unchanged, thereby ensuring the accuracy of the test results. Multiple groups of experiments can be conducted at the same time, which improves the efficiency of parameter adjustment experiments.

[0138] It should be understood that although Figure 2-Figure 6 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 2-Figure 6At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0139] In one embodiment, Figure 7 As shown, an RTA multi-layer test device 10 is provided, including: a user data bucketing module 1, an experiment quantity management module 2, a control group and an experiment group setting module 3, a first experiment layer module 4, a second experiment layer module 5 and an experiment termination control module 6.

[0140] The user data bucketing module 1 is used to obtain user data provided by the media, and divide the user data into multiple buckets. The number of user data in all buckets is the same. The buckets are provided with bucket address numbers. Each bucket contains multiple user data, and each user data is associated with the bucket address number of the bucket in which it is located.

[0141] The experiment quantity management module 2 is used to obtain the types of comparative experiments to be performed, obtain the adjustment parameters of user data in each type of comparative experiments, and determine the number of experimental layers for each type of comparative experiments according to the number of types of the adjustment parameters.

[0142] The control group and experimental group setting module 3 is used to select two buckets as the control group for all experimental layers, split the remaining buckets into two buckets each as an experimental group, obtain the number of the experimental groups, and determine the number of experimental groups in each experimental layer according to the number of the experimental groups.

[0143] The first test layer module 4 is used to form a first test layer using the control group and the experimental group, set the value of the adjustment parameter of the experimental group in the first test layer, and obtain the reasonable value of the adjustment parameter in the first test layer by comparing the test results.

[0144] The second test layer module 5 is used to perform hash operation on the user data in the experimental group of the first test layer to calculate the bucket address numbers of the buckets, re-bucket the user data according to the calculated bucket address numbers to form experimental groups of the second test layer, set the values ​​of the adjustment parameters of the experimental groups in the second test layer, and obtain reasonable values ​​of the adjustment parameters in the second test layer by comparing the test results.

[0145] The test termination control module 6 is used to determine whether the comparative test of all types of adjustment parameters is completed. If so, it outputs the reasonable value of the adjustment parameter in the second test layer. If not, it repeats the previous step until all types of comparative experiments are completed.

[0146] In this embodiment, the steps of re-performing a hash operation on the user data in the experimental group of the first experimental layer to calculate the bucket address numbers of the buckets, and re-bucketing the user data according to the calculated bucket address numbers to form the experimental group of the second experimental layer include:

[0147] Acquire user data in the experimental group of the first experimental layer;

[0148] Performing a hash operation on the user data;

[0149] The last digit after the hash operation is taken as the bucket address number, and the user data is re-bucketed according to the calculated bucket address number to form the experimental group of the second test layer.

[0150] In this embodiment, the step of performing a hash operation on the user data includes:

[0151] Obtain the device number of the device used by the user corresponding to the user data;

[0152] Obtain a fixed character string preset corresponding to the second test layer, and add the fixed character string to the device number of the user device corresponding to all user data to form a first character string;

[0153] Using the MD5 message digest algorithm to obtain a first hash value corresponding to the first character string;

[0154] A hash operation is performed on the first hash value, and a modulo 10 is calculated.

[0155] In this embodiment, the formula for performing a hash operation on the user data is Hash[Md5(device+fixed string)]%10;

[0156] The device is the device number of the device used by the user in the user data;

[0157] The fixed character string is a character string preset for the corresponding test layer, and the fixed character strings of all test layers are set to be different;

[0158] Md5 is used to generate a 128-bit (16-byte) hash value;

[0159] The hash algorithm uses the CRC32 algorithm;

[0160] %10 is the remainder of 10.

[0161] In this embodiment, the steps of obtaining the types of comparative experiments to be performed, obtaining adjustment parameters of user data in each type of comparative experiments, and determining the number of test layers for each type of comparative experiments according to the number of types of the adjustment parameters include:

[0162] Obtain the type of comparative experiment to be performed, including exclusion experiment and bidding experiment;

[0163] Obtain adjustment parameters of user data in each type of comparative experiment, wherein the adjustment parameters include a bid coefficient, a user selection willingness score, and a user qualification score;

[0164] The number of types of the adjustment parameters of each type of comparative experiment is obtained, and the number of test layers of each type of comparative experiment is determined according to the number of types of the adjustment parameters.

[0165] In this embodiment, the step of determining whether the comparative test of all types of adjustment parameters is completed, if yes, outputting the reasonable value of the adjustment parameter in the second test layer, and if no, repeating the previous step until all types of comparative tests are completed includes:

[0166] If the comparative test of all types of adjustment parameters has not been completed, the user data in the experimental group of the current experimental layer is obtained;

[0167] Perform hash operation on the user data in the experimental group of the current experimental layer to calculate the bucket address number of the bucket;

[0168] Re-bucketing the user data according to the calculated bucket address numbers to form an experimental group of the next experimental layer, and setting the value of the adjustment parameter of the experimental group in the next experimental layer;

[0169] Obtaining reasonable values ​​of the adjustment parameters by comparing the test results;

[0170] Determine whether the comparative tests of all types of adjustment parameters are completed, and if so, output the reasonable values ​​of the adjustment parameters, and if not, perform the remaining comparative tests according to the adjustment parameters for which the comparative tests have not been completed.

[0171] In this embodiment, if Figure 7 As shown, the RTA multi-layer test device 10 further includes: a bidding module 7 .

[0172] The bidding module 7 is used to further include: obtaining the output reasonable value of the adjustment parameter after the step of outputting the reasonable value of the adjustment parameter, adjusting the RTA policy parameter according to the output reasonable value of the adjustment parameter, and bidding on the user data according to the adjusted RTA policy.

[0173] In the above-mentioned RTA multi-layer test device, by utilizing the limited user data provided by the media, a control group and an experimental group are formed through bucketing. The number of experimental groups in each test layer and the values ​​of the adjustment parameters are determined according to the number of experimental groups. The user data in the experimental group of the second test layer is obtained by performing a hash operation and bucketing on the user data in the experimental group of the first test layer, that is, the user data in the experimental group of the subsequent test layer is obtained by performing a hash operation and bucketing on the user data in the experimental group of the previous test layer. The number of experimental groups can be constructed on demand, and the control group always remains unchanged, thereby ensuring the accuracy of the test results. Multiple groups of experiments can be performed at the same time, which improves the efficiency of parameter adjustment experiments.

[0174] The specific definition of the RTA multi-layer test device can be found in the definition of the RTA multi-layer test method above, which will not be repeated here. Each module in the above RTA multi-layer test device can be implemented in whole or in part by software, hardware and a combination thereof. The above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0175] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 8 As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store RTA multi-layer test data. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, an RTA multi-layer test method is implemented.

[0176] Those skilled in the art will understand that Figure 8 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0177] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:

[0178] Obtain user data provided by the media, and divide the user data into multiple buckets, wherein the amount of user data in all buckets is the same, the buckets are provided with bucket address numbers, each of the buckets contains multiple user data, and each user data is associated with the bucket address number of the bucket in which it is located;

[0179] Obtaining the types of comparative experiments to be performed, obtaining adjustment parameters of user data in each type of comparative experiment, and determining the number of test layers for each type of comparative experiment according to the number of types of the adjustment parameters;

[0180] Select two buckets as control groups for all experimental layers, split the remaining buckets into two buckets each as an experimental group, obtain the number of the experimental groups, and determine the number of experimental groups in each experimental layer according to the number of the experimental groups;

[0181] The control group and the experimental group are used to form a first test layer, the value of the adjustment parameter of the experimental group in the first test layer is set, and a reasonable value of the adjustment parameter in the first test layer is obtained by comparing the test results;

[0182] Performing a hash operation on the user data in the experimental group of the first test layer to calculate the bucket address numbers of the buckets, re-bucketing the user data according to the calculated bucket address numbers to form the experimental group of the second test layer, setting the value of the adjustment parameter of the experimental group in the second test layer, and obtaining a reasonable value of the adjustment parameter in the second test layer by comparing the test results;

[0183] Determine whether the comparative test of all types of adjustment parameters is completed. If so, output the reasonable value of the adjustment parameter in the second test layer. If not, repeat the previous step until all types of comparative tests are completed.

[0184] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0185] The step of re-performing a hash operation on the user data in the experimental group of the first experimental layer to calculate the bucket address numbers of the buckets, and re-bucketing the user data according to the calculated bucket address numbers to form the experimental group of the second experimental layer includes:

[0186] Acquire user data in the experimental group of the first experimental layer;

[0187] Performing a hash operation on the user data;

[0188] The last digit after the hash operation is taken as the bucket address number, and the user data is re-bucketed according to the calculated bucket address number to form the experimental group of the second test layer.

[0189] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0190] The step of performing a hash operation on the user data comprises:

[0191] Obtain the device number of the device used by the user corresponding to the user data;

[0192] Obtain a fixed character string preset corresponding to the second test layer, and add the fixed character string to the device number of the user device corresponding to all user data to form a first character string;

[0193] Using the MD5 message digest algorithm to obtain a first hash value corresponding to the first character string;

[0194] A hash operation is performed on the first hash value, and a modulo 10 is calculated.

[0195] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0196] The formula for performing hash operation on the user data is Hash[Md5(device+fixed string)]%10;

[0197] The device is the device number of the device used by the user in the user data;

[0198] The fixed character string is a character string preset for the corresponding test layer, and the fixed character strings of all test layers are set to be different;

[0199] Md5 is used to generate a 128-bit (16-byte) hash value;

[0200] The hash algorithm uses the CRC32 algorithm;

[0201] %10 is the remainder of 10.

[0202] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0203] The steps of obtaining the types of comparative experiments to be performed, obtaining adjustment parameters of user data in each type of comparative experiments, and determining the number of test layers for each type of comparative experiments according to the number of types of adjustment parameters include:

[0204] Obtain the type of comparative experiment to be performed, including exclusion experiment and bidding experiment;

[0205] Obtain adjustment parameters of user data in each type of comparative experiment, wherein the adjustment parameters include a bid coefficient, a user selection willingness score, and a user qualification score;

[0206] The number of types of the adjustment parameters of each type of comparative experiment is obtained, and the number of test layers of each type of comparative experiment is determined according to the number of types of the adjustment parameters.

[0207] In one embodiment, when the processor executes the computer program, the following steps are also implemented:

[0208] The step of judging whether the comparative test of all types of adjustment parameters is completed, and if so, outputting the reasonable value of the adjustment parameter in the second test layer, and if not, repeating the previous step until all types of comparative tests are completed includes:

[0209] If the comparative test of all types of adjustment parameters has not been completed, the user data in the experimental group of the current experimental layer is obtained;

[0210] Perform hash operation on the user data in the experimental group of the current experimental layer to calculate the bucket address number of the bucket;

[0211] Re-bucketing the user data according to the calculated bucket address numbers to form an experimental group of the next experimental layer, and setting the value of the adjustment parameter of the experimental group in the next experimental layer;

[0212] Obtaining reasonable values ​​of the adjustment parameters by comparing the test results;

[0213] Determine whether the comparative tests of all types of adjustment parameters are completed, and if so, output the reasonable values ​​of the adjustment parameters, and if not, perform the remaining comparative tests according to the adjustment parameters for which the comparative tests have not been completed.

[0214] In one embodiment, when the processor executes the computer program, the following steps are also implemented:

[0215] After the step of outputting the reasonable numerical value of the adjustment parameter, the following step is further included:

[0216] Acquire the reasonable value of the output adjustment parameter, adjust the RTA policy parameter according to the reasonable value of the output adjustment parameter, and bid for the user data according to the adjusted RTA policy.

[0217] For the specific limitations on the steps implemented when the processor executes the computer program, please refer to the limitations on the RTA multi-layer test method above, which will not be repeated here.

[0218] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0219] Obtain user data provided by the media, and divide the user data into multiple buckets, wherein the amount of user data in all buckets is the same, the buckets are provided with bucket address numbers, each of the buckets contains multiple user data, and each user data is associated with the bucket address number of the bucket in which it is located;

[0220] Obtaining the types of comparative experiments to be performed, obtaining adjustment parameters of user data in each type of comparative experiment, and determining the number of test layers for each type of comparative experiment according to the number of types of the adjustment parameters;

[0221] Select two buckets as control groups for all experimental layers, split the remaining buckets into two buckets each as an experimental group, obtain the number of the experimental groups, and determine the number of experimental groups in each experimental layer according to the number of the experimental groups;

[0222] The control group and the experimental group are used to form a first test layer, the value of the adjustment parameter of the experimental group in the first test layer is set, and a reasonable value of the adjustment parameter in the first test layer is obtained by comparing the test results;

[0223] Performing a hash operation on the user data in the experimental group of the first test layer to calculate the bucket address numbers of the buckets, re-bucketing the user data according to the calculated bucket address numbers to form the experimental group of the second test layer, setting the value of the adjustment parameter of the experimental group in the second test layer, and obtaining a reasonable value of the adjustment parameter in the second test layer by comparing the test results;

[0224] Determine whether the comparative test of all types of adjustment parameters is completed. If so, output the reasonable value of the adjustment parameter in the second test layer. If not, repeat the previous step until all types of comparative tests are completed.

[0225] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0226] The step of re-performing a hash operation on the user data in the experimental group of the first experimental layer to calculate the bucket address numbers of the buckets, and re-bucketing the user data according to the calculated bucket address numbers to form the experimental group of the second experimental layer includes:

[0227] Acquire user data in the experimental group of the first experimental layer;

[0228] Performing a hash operation on the user data;

[0229] The last digit after the hash operation is taken as the bucket address number, and the user data is re-bucketed according to the calculated bucket address number to form the experimental group of the second test layer.

[0230] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0231] The step of performing a hash operation on the user data comprises:

[0232] Obtain the device number of the device used by the user corresponding to the user data;

[0233] Obtain a fixed character string preset corresponding to the second test layer, and add the fixed character string to the device number of the user device corresponding to all user data to form a first character string;

[0234] Using the MD5 message digest algorithm to obtain a first hash value corresponding to the first character string;

[0235] A hash operation is performed on the first hash value, and a modulo 10 is calculated.

[0236] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0237] The formula for performing hash operation on the user data is Hash[Md5(device+fixed string)]%10;

[0238] The device is the device number of the device used by the user in the user data;

[0239] The fixed character string is a character string preset for the corresponding test layer, and the fixed character strings of all test layers are set to be different;

[0240] Md5 is used to generate a 128-bit (16-byte) hash value;

[0241] The hash algorithm uses the CRC32 algorithm;

[0242] %10 is the remainder of 10.

[0243] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0244] The steps of obtaining the types of comparative experiments to be performed, obtaining adjustment parameters of user data in each type of comparative experiments, and determining the number of test layers for each type of comparative experiments according to the number of types of adjustment parameters include:

[0245] Obtain the type of comparative experiment to be performed, including exclusion experiment and bidding experiment;

[0246] Obtain adjustment parameters of user data in each type of comparative experiment, wherein the adjustment parameters include a bid coefficient, a user selection willingness score, and a user qualification score;

[0247] The number of types of the adjustment parameters of each type of comparative experiment is obtained, and the number of test layers of each type of comparative experiment is determined according to the number of types of the adjustment parameters.

[0248] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0249] The step of judging whether the comparative test of all types of adjustment parameters is completed, and if so, outputting the reasonable value of the adjustment parameter in the second test layer, and if not, repeating the previous step until all types of comparative tests are completed includes:

[0250] If the comparative test of all types of adjustment parameters has not been completed, the user data in the experimental group of the current experimental layer is obtained;

[0251] Perform hash operation on the user data in the experimental group of the current experimental layer to calculate the bucket address number of the bucket;

[0252] Re-bucketing the user data according to the calculated bucket address numbers to form an experimental group of the next experimental layer, and setting the value of the adjustment parameter of the experimental group in the next experimental layer;

[0253] Obtaining reasonable values ​​of the adjustment parameters by comparing the test results;

[0254] Determine whether the comparative tests of all types of adjustment parameters are completed, if so, output the reasonable values ​​of the adjustment parameters, if not, perform the remaining comparative tests according to the adjustment parameters of the uncompleted comparative tests.

[0255] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0256] After the step of outputting the reasonable numerical value of the adjustment parameter, the following step is further included:

[0257] Acquire the reasonable value of the output adjustment parameter, adjust the RTA policy parameter according to the reasonable value of the output adjustment parameter, and bid for the user data according to the adjusted RTA policy.

[0258] For the specific definition of the steps implemented when the computer program is executed by the processor, please refer to the definition of the RTA multi-layer test method above, which will not be repeated here.

[0259] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0260] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0261] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A RTA multilayer test method, characterized in that: include: Obtain user data provided by the media, and divide the user data into multiple buckets, wherein the amount of user data in all buckets is the same, the buckets are provided with bucket address numbers, each of the buckets contains multiple user data, and each user data is associated with the bucket address number of the bucket in which it is located; Obtaining the types of comparative experiments to be performed, obtaining adjustment parameters of user data in each type of comparative experiment, and determining the number of test layers for each type of comparative experiment according to the number of types of the adjustment parameters; Select two buckets as control groups for all experimental layers, split the remaining buckets into two buckets each as an experimental group, obtain the number of the experimental groups, and determine the number of experimental groups in each experimental layer according to the number of the experimental groups; The control group and the experimental group are used to form a first test layer, the value of the adjustment parameter of the experimental group in the first test layer is set, and a reasonable value of the adjustment parameter in the first test layer is obtained by comparing the test results; Performing a hash operation on the user data in the experimental group of the first test layer to calculate the bucket address numbers of the buckets, re-bucketing the user data according to the calculated bucket address numbers to form the experimental group of the second test layer, setting the value of the adjustment parameter of the experimental group in the second test layer, and obtaining a reasonable value of the adjustment parameter in the second test layer by comparing the test results; Determine whether the comparative test of all types of adjustment parameters is completed. If so, output the reasonable value of the adjustment parameter in the second test layer. If not, repeat the previous step until all types of comparative tests are completed.

2. The RTA multilayer test method according to claim 1, characterized in that: The step of re-performing a hash operation on the user data in the experimental group of the first experimental layer to calculate the bucket address numbers of the buckets, and re-bucketing the user data according to the calculated bucket address numbers to form the experimental group of the second experimental layer includes: Acquire user data in the experimental group of the first experimental layer; Performing a hash operation on the user data; The last digit after the hash operation is taken as the bucket address number, and the user data is re-bucketed according to the calculated bucket address number to form the experimental group of the second test layer.

3. The RTA multilayer test method according to claim 2, characterized in that: The step of performing a hash operation on the user data comprises: Obtain the device number of the device used by the user corresponding to the user data; Obtain a fixed character string preset corresponding to the second test layer, and add the fixed character string to the device number of the user device corresponding to all user data to form a first character string; Using the MD5 message digest algorithm to obtain a first hash value corresponding to the first character string; A hash operation is performed on the first hash value, and a modulo 10 is calculated.

4. The RTA multilayer test method according to claim 3, characterized in that: The formula for performing hash operation on the user data is Hash[Md5(device+fixed string)]%10; The device is the device number of the device used by the user in the user data; The fixed character string is a character string preset for the corresponding test layer, and the fixed character strings of all test layers are set to be different; Md5 is used to generate a 128-bit hash value; The hash algorithm uses the CRC32 algorithm; %10 is the remainder of 10.

5. The RTA multi-layer test method according to claim 1, characterized in that: The steps of obtaining the types of comparative experiments to be performed, obtaining adjustment parameters of user data in each type of comparative experiments, and determining the number of test layers for each type of comparative experiments according to the number of types of adjustment parameters include: Obtain the type of comparative experiment to be performed, including exclusion experiment and bidding experiment; Obtain adjustment parameters of user data in each type of comparative experiment, wherein the adjustment parameters include a bid coefficient, a user selection willingness score, and a user qualification score; The number of types of the adjustment parameters of each type of comparative experiment is obtained, and the number of test layers of each type of comparative experiment is determined according to the number of types of the adjustment parameters.

6. The RTA multi-layer test method according to claim 1, characterized in that: The step of judging whether the comparative test of all types of adjustment parameters is completed, and if so, outputting the reasonable value of the adjustment parameter in the second test layer, and if not, repeating the previous step until all types of comparative tests are completed includes: If the comparative test of all types of adjustment parameters has not been completed, the user data in the experimental group of the current experimental layer is obtained; Perform hash operation on the user data in the experimental group of the current experimental layer to calculate the bucket address number of the bucket; Re-bucketing the user data according to the calculated bucket address numbers to form an experimental group of the next experimental layer, and setting the value of the adjustment parameter of the experimental group in the next experimental layer; Obtaining reasonable values ​​of the adjustment parameters by comparing the test results; Determine whether the comparative tests of all types of adjustment parameters are completed, and if so, output the reasonable values ​​of the adjustment parameters, and if not, perform the remaining comparative tests according to the adjustment parameters for which the comparative tests have not been completed.

7. The RTA multi-layer test method according to claim 1, characterized in that: After the step of outputting the reasonable numerical value of the adjustment parameter, the following step is further included: Acquire the reasonable value of the output adjustment parameter, adjust the RTA policy parameter according to the reasonable value of the output adjustment parameter, and bid for the user data according to the adjusted RTA policy.

8. An RTA multilayer test device, characterized in that: The device comprises: A user data bucketing module is used to obtain user data provided by the media, and divide the user data into multiple buckets. The number of user data in all buckets is the same. The buckets are provided with bucket address numbers. Each bucket contains multiple user data, and each user data is associated with the bucket address number of the bucket in which it is located; An experiment quantity management module is used to obtain the types of comparative experiments to be performed, obtain adjustment parameters of user data in each type of comparative experiment, and determine the number of experimental layers for each type of comparative experiment according to the number of types of adjustment parameters; The control group and experimental group setting module is used to select two buckets as the control group of all experimental layers, split the remaining buckets into two buckets each as an experimental group, obtain the number of the experimental groups, and determine the number of experimental groups in each experimental layer according to the number of the experimental groups; A first test layer module, used to form a first test layer using the control group and the experimental group, set the value of the adjustment parameter of the experimental group in the first test layer, and obtain a reasonable value of the adjustment parameter in the first test layer by comparing the test results; The second test layer module is used to perform a hash operation on the user data in the experimental group of the first test layer to calculate the bucket address number of the bucket, re-bucket the user data according to the calculated bucket address number to form the experimental group of the second test layer, set the value of the adjustment parameter of the experimental group in the second test layer, and obtain the reasonable value of the adjustment parameter in the second test layer by comparing the test results; The test termination control module is used to determine whether the comparative test of all types of adjustment parameters is completed. If so, the reasonable value of the adjustment parameter in the second test layer is output. If not, the previous step is repeated until all types of comparative experiments are completed.

9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images