Data processing method, device, equipment and storage medium

Abstract

The embodiments of the present disclosure provide a data processing method, apparatus, device and storage medium. The test experiments are screened based on preset screening conditions to obtain candidate test experiments; the composite test indicators of the candidate test experiments are determined; wherein the composite test indicators are determined by basic test indicators; the candidate test experiments are filtered based on the composite test indicators to obtain target test experiments; and the indicator influencing factors are determined based on the target test experiments. The data processing method provided by the embodiments of the present disclosure filters the candidate test experiments based on the composite test indicators, and determines the indicator influencing factors based on the filtered test experiments. The influencing factors that cause experimental losses are automatically determined by the indicators of the test experiments, which can not only reduce costs but also improve efficiency.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of data processing technology, and in particular, to a data processing method, apparatus, device, and storage medium. Background Art

[0002] In the Internet field, experiments are usually conducted on online user groups to verify the benefits of different strategies. The experiment takes effect by dynamically configuring some or all online user groups. Therefore, if a push strategy is used to conduct experiments, it will have a certain impact on the benefits. For some experiments with revenue loss, it is particularly important to determine the cause of the loss. In the prior art, the search for the cause through manual investigation is not only labor-intensive but also inefficient. Summary of the invention

[0003] The embodiments of the present disclosure provide a data processing method, apparatus, device and storage medium, which can automatically determine the influencing factors causing experimental losses by testing the indicators of the experiment, which can not only reduce costs but also improve efficiency.

[0004] In a first aspect, an embodiment of the present disclosure provides a data processing method, including:

[0005] Screening the test experiments based on preset screening conditions to obtain candidate test experiments;

[0006] Determine a composite test indicator of the candidate test experiment; wherein the composite test indicator is determined by a basic test indicator;

[0007] Filter the candidate test experiments based on the composite test indicator to obtain a target test experiment;

[0008] Determine the index influencing factors based on the target test experiment.

[0009] In a second aspect, an embodiment of the present disclosure further provides a data processing device, including:

[0010] A candidate test experiment acquisition module is used to screen the test experiments based on preset screening conditions to obtain candidate test experiments;

[0011] A composite test indicator determination module, used to determine the composite test indicator of the candidate test experiment; wherein the composite test indicator is determined by the basic test indicator;

[0012] A target test experiment acquisition module, used to filter the candidate test experiments based on the composite test indicator to obtain a target test experiment;

[0013] The indicator influencing factor determination module is used to determine the indicator influencing factors based on the target test experiment.

[0014] In a third aspect, an embodiment of the present disclosure further provides an electronic device, the electronic device comprising:

[0015] one or more processors;

[0016] a storage device for storing one or more programs,

[0017] When the one or more programs are executed by the one or more processors, the one or more processors implement the data processing method as described in the embodiment of the present disclosure.

[0018] In a fourth aspect, an embodiment of the present disclosure further provides a storage medium comprising computer executable instructions, which, when executed by a computer processor, are used to execute the data processing method as described in the embodiment of the present disclosure.

[0019] The disclosed embodiments disclose a data processing method, apparatus, device and storage medium, which screen test experiments based on preset screening conditions to obtain candidate test experiments; determine the composite test index of the candidate test experiments; wherein the composite test index is determined by the basic test index; filter the candidate test experiments based on the composite test index to obtain a target test experiment; and determine the index influencing factors based on the target test experiment. The data processing method provided by the disclosed embodiments filters candidate test experiments based on composite test indexes, and determines the index influencing factors based on the filtered test experiments. The influencing factors that cause experimental losses are automatically determined by the test experiment indicators, which can not only reduce costs but also improve efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.

[0021] Figure 1 It is a flowchart of a data processing method provided by an embodiment of the present disclosure;

[0022] Figure 2 is a structural schematic diagram of a data processing device provided by an embodiment of the present disclosure;

[0023] Figure 3 It is a structural schematic diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0024] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0025] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0026] The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0027] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0028] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0029] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0030] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0031] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.

[0032] As an optional but non-limiting implementation, in response to receiving an active request from the user, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0033] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet the relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0034] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and relevant provisions.

[0035] Figure 1 A flow chart of a data processing method provided in an embodiment of the present disclosure is applicable to situations where factors affecting indicators in a test experiment are determined. The method can be executed by a data processing device, which can be implemented in the form of software and / or hardware, and optionally, by an electronic device, which can be a mobile terminal, a PC, or a server, etc.

[0036] like Figure 1 As shown, the method includes:

[0037] S110, screening the test experiments based on the preset screening conditions to obtain candidate test experiments.

[0038] Among them, the screening conditions include at least one of the following: time information, scenario information, operating system, and the difference between the actual amount of the indicator and the standard amount. Time information can be represented by a time period, for example: one day, N consecutive days or a week, etc. The role of time information is to screen the test experiment based on the experimental data in the time information. Scenario information can be understood as the scenario that the test experiment should have, and the scenario information in this embodiment can be understood as the information flow category. The operating system can be the operating system used by the user terminal. The difference between the actual amount of the indicator and the standard amount can be understood as: the difference between the total amount of the indicator and the standard amount of all test experiments. In this embodiment, screening the test experiment based on the difference between the actual amount of the indicator and the standard amount can be: first determine the difference between the indicator of the experimental group and the indicator of the control group in each test experiment, if the difference between the indicator of the experimental group and the indicator of the control group exceeds the difference between the actual amount of the indicator and the standard amount, then the test experiment is screened out. The standard amount can be a pre-set value.

[0039] In this embodiment, the screening conditions are first set, and then the test experiments are screened based on the screening conditions to obtain candidate test experiments, which can reduce the order of the queried test experiments and improve the query efficiency.

[0040] S120, determining composite test indicators of the candidate test experiment.

[0041] Among them, the composite test indicators are determined by the basic test indicators. Basic test indicators can include delivery volume (send), display volume (show), click volume (click) and conversion volume (convert). Composite test indicators include display delivery ratio (ssr), click display ratio (ctr2) and conversion click ratio (cvr). Among them, the display delivery ratio can be understood as the ratio of display volume to delivery volume, the click display ratio can be understood as the ratio of click volume to display volume, and the conversion click ratio can be understood as the ratio of conversion volume to click volume.

[0042] Specifically, the method for determining the composite test indicators of the candidate test experiment can be: obtaining the basic test indicators of at least one experimental group in the candidate test experiment; merging the basic test indicators of at least one experimental group; setting logical operations on the merged basic test indicators to obtain the composite test indicators of the candidate test experiment.

[0043] In this embodiment, if the candidate test experiment includes an experimental group, the process of determining the composite test indicator of the candidate test experiment can be: determining the ratio of the display volume to the delivery volume of the experimental group to obtain the display delivery ratio; determining the ratio of the click volume to the display volume of the experimental group to obtain the click-to-display ratio; determining the ratio of the conversion volume to the click volume of the experimental group to obtain the conversion-to-click ratio.

[0044] If the candidate test experiment includes multiple experimental groups, first accumulate the basic test indicators of each experimental group, and then determine the ratio of the accumulated display volume to the accumulated delivery volume to obtain the display delivery ratio; determine the ratio of the accumulated click volume to the accumulated display volume to obtain the click-display ratio; determine the ratio of the accumulated conversion volume to the accumulated click volume to obtain the conversion-click ratio.

[0045] S130, filtering the candidate test experiments based on the composite test indicator to obtain the target test experiment.

[0046] In this embodiment, the method of filtering the candidate test experiments based on the composite test indicator may be: firstly determining the influence weight of each candidate test experiment based on the composite test indicator, and then filtering the candidate test experiments based on the influence weight.

[0047] Specifically, the candidate test experiments are filtered based on the composite test indicators to obtain the target test experiment in the following manner: obtaining the composite test indicators, the experimental group traffic, and the benchmark quantities corresponding to the composite test indicators of at least one experimental group in the candidate test experiments; determining the impact weight of the candidate test experiment based on the composite test indicators, the experimental group traffic, and the benchmark quantities corresponding to the composite test indicators of at least one experimental group; filtering the candidate test experiments based on the impact weight to obtain the target test experiment.

[0048] Among them, the benchmark is characterized by the composite test index of the control group. The composite test index of the experimental group is determined by the basic test index of the experimental group, and the composite test index of the control group is determined by the basic test index of the control group. The determination method can be referred to the above embodiment and will not be repeated here. The experimental traffic can be understood as the total traffic shared by all experimental groups, which is characterized by the ratio of the user group size of all experimental groups to the total user group size participating in the test experiment, for example: 0.8.

[0049] Specifically, the method for determining the impact weight of a candidate test experiment based on the composite test indicator of at least one experimental group, the experimental group traffic and the benchmark quantity corresponding to the composite test indicator can be: first determine the difference between the composite test indicator of each experimental group and the composite test indicator of the control group, then add up the differences, divide the accumulated differences by the experimental traffic, and obtain the impact weight of the candidate test experiment.

[0050] Optionally, the candidate test experiments are filtered based on the impact weights, and the method for obtaining the target test experiment can be: determine the proportion of the impact weights of each candidate test experiment, and determine the candidate test experiments whose proportions exceed a set threshold as target test experiments. The set threshold can be pre-set, for example: set to any value between 80% and 90%. Exemplarily, assuming that there are three candidate test experiments, and the impact weights are: a1, a2, and a3, respectively, then the proportions are a1 / (a1+a2+a3), a2 / (a1+a2+a3), and a3 / (a1+a2+a3), respectively. Assuming that a2 / (a1+a2+a3) exceeds the set threshold, the corresponding candidate test experiment is determined as the target test experiment.

[0051] Optionally, the candidate test experiments are filtered based on the impact weights to obtain the target test experiment by: sorting the impact weights in descending order, and determining the candidate test experiments corresponding to a set number of impact weights that are ranked higher as the target test experiments; or, sorting the impact weights in ascending order, and determining the candidate test experiments corresponding to a set number of impact weights that are ranked lower as the target test experiments.

[0052] In this embodiment, the above operation is performed based on each composite test indicator (respectively: ssr, ctr2 and cvr), and the target test experiment corresponding to each composite indicator can be determined.

[0053] S140, determining the influencing factors of the indicator based on the target test experiment.

[0054] Among them, the factors affecting the indicators may include client factors, server factors, client multi-call factors, etc.

[0055] Specifically, the method of determining the influencing factors of indicators based on the target test experiment can be: determining the offset information between the composite test indicators of the target test experiment and the benchmark quantity as the first offset information; determining the offset information of the basic test indicators based on the first offset information as the second offset information; and determining the influencing factors based on the second offset information.

[0056] The first offset information includes an offset direction. The second offset information includes an offset direction and an offset weight. The offset direction may include being high or low. The offset weight of a basic test indicator may be represented by the number of times a certain offset direction appears. For example, for "click volume", if the number of times the high click volume appears is 3, the weight of the high click volume is 3.

[0057] In this embodiment, since the composite test indicator is obtained by setting a logical operation based on two basic test indicators, the offset direction of the composite test indicator will affect the offset direction of the two corresponding basic test indicators. Specifically, the offset information of the basic test indicator determined according to the first offset information includes the following situations: if the ssr is high, the display volume is high or the delivery volume is low; if the ssr is low, the display volume is low or the delivery volume is high; if the ctr2 is high, the click volume is high or the display volume is low; if the ctr2 is low, the click volume is low or the display volume is high; if the cvr is high, the conversion volume is high or the click volume is low; if the cvr is low, the conversion volume is low or the click volume is high.

[0058] The composite test index of the target test experiment may be the corresponding test index when the target test experiment is filtered out. For example, if the target test experiment A is filtered out based on ssr, the composite test index of the target test experiment A is ssr.

[0059] Specifically, first determine the offset direction of the composite test indicators of each target test experiment relative to the benchmark, then determine the offset direction and offset weight of the basic test indicators based on the offset direction of the composite test indicators relative to the benchmark, and finally determine the indicator influencing factors based on the offset direction and offset weight of the basic test indicators.

[0060] In this embodiment, a correspondence between the index influencing factors and the basic test index combination can be established in advance, and the index influencing factors can be determined based on the correspondence. For example, if the final result is: show is too high, click is too high, and convert is too high, then the index influencing factor is a client factor, that is, there is a problem with the client. If the final result is send is too high or too low, then the index influencing factor is a server factor, that is, there is a problem with the server.

[0061] Optionally, before determining the composite test indicators of the candidate test experiment, the following steps are also included: obtaining the control group delivery volume, experimental group delivery volume, grayscale delivery volume and experimental traffic of the candidate test experiment; dividing the candidate test experiment into grayscale experiments and formal experiments according to the control group delivery volume, experimental group delivery volume, experimental version delivery volume and experimental traffic.

[0062] Among them, the control group's delivery amount can be understood as the delivery amount for the control group in the candidate test experiment; the experimental group's delivery amount can be understood as the delivery amount for the experimental group in the candidate test experiment; and the grayscale delivery amount can be all delivery amounts for the candidate test experiment.

[0063] The delivery volume of the control group can be expressed as x1, the delivery volume of the experimental group can be expressed as x2, the grayscale delivery volume can be expressed as x3, and the experimental flow rate is expressed as C. If the delivery volume of the control group, the delivery volume of the experimental group, the grayscale delivery volume and the experimental flow rate all meet the following conditions: x1+x2<a*x3, C>b%, then the candidate test experiment is a grayscale experiment, where a and b are set values, for example, a is 1.03, and b is 1. If the delivery volume of the control group, the delivery volume of the experimental group, the grayscale delivery volume and the experimental flow rate all meet the following conditions: x1+x2≥a*x3, then the candidate test experiment is a formal experiment.

[0064] Optionally, for formal experiments, a normality check is performed on the composite test indicators within a set period in the formal experiment; if the composite test indicators do not conform to the normal distribution, the change rates of multiple composite test indicators within a set period in the formal experiment are determined; if the maximum composite test indicator change rate is greater than a set value, the time period corresponding to the maximum composite test indicator standardized rate is determined as an abnormal time period.

[0065] Among them, the normality check can be understood as whether the composite test indicators of the formal experiment within the set period meet the normal distribution. If they meet the normal distribution, the data of the formal experiment is normal. If they do not meet the normal distribution, it means that the data of the formal experiment within the set period is abnormal. At this time, it is necessary to further determine the abnormal period. The abnormal period is the period within the set period.

[0066] Assuming that the set period is 7 days, i.e. one week, the method for determining the change rates of multiple composite test indicators within the set period in the formal experiment can be: extract the composite test indicators of any two days within the set period, and take the difference between the two composite test indicators to find the number of days between the two days, and then obtain a composite test indicator change rate. The calculation formula for the composite test indicator change rate can be expressed as: Among them, i∈[0,7), j∈(0,7].

[0067] Specifically, after obtaining multiple composite test index change rates, if the maximum composite test index change rate is greater than the set value, the time period corresponding to the maximum composite test index normalization rate is determined as the abnormal time period. For example, assuming that the set period is one week, and the composite test index normalization rate between Wednesday and Friday is the largest and greater than the set value, Wednesday to Friday is the abnormal time period.

[0068] Optionally, after obtaining the abnormal time period, the test experiments are screened based on preset screening conditions to obtain candidate test experiments in the following manner: the formal experiments are screened according to the abnormal time period to obtain candidate formal experiments.

[0069] In this embodiment, experiments are screened based on experimental data within an abnormal period, which can further narrow the query scope and reduce the amount of calculation.

[0070] The technical solution of the disclosed embodiment screens the test experiments based on preset screening conditions to obtain candidate test experiments; determines the composite test indicators of the candidate test experiments; wherein the composite test indicators are determined by basic test indicators; filters the candidate test experiments based on the composite test indicators to obtain target test experiments; and determines the indicator influencing factors based on the target test experiments. The data processing method provided by the disclosed embodiment filters the candidate test experiments based on the composite test indicators, and determines the indicator influencing factors based on the filtered test experiments. By automatically determining the influencing factors that cause experimental losses through the indicators of the test experiments, it can not only reduce costs but also improve efficiency.

[0071] Figure 2 A structural diagram of a data processing device provided by an embodiment of the present disclosure is shown in FIG. Figure 2 As shown, the device comprises:

[0072] A candidate test experiment acquisition module 210 is used to screen the test experiments based on preset screening conditions to obtain candidate test experiments;

[0073] A composite test indicator determination module 220 is used to determine the composite test indicator of the candidate test experiment; wherein the composite test indicator is determined by the basic test indicator;

[0074] A target test experiment acquisition module 230 is used to filter the candidate test experiments based on the composite test indicator to obtain a target test experiment;

[0075] The indicator influencing factor determination module 240 is used to determine the indicator influencing factors based on the target test experiment.

[0076] Optionally, the composite test indicator determination module 220 is further configured to:

[0077] Obtaining basic test indicators of at least one experimental group in the candidate test experiment, wherein the basic test indicator includes at least one of the following: delivery volume, display volume, click volume, and conversion volume;

[0078] Combining the basic test indicators of the at least one experimental group;

[0079] A logical operation is performed on the combined basic test indicators to obtain a composite test indicator of the candidate test experiment.

[0080] Optionally, the target test experiment acquisition module 230 is further used to:

[0081] Obtaining a composite test indicator of at least one experimental group in the candidate test experiment, the experimental group flow, and a reference quantity corresponding to the composite test indicator; wherein the reference quantity is represented by the composite test indicator of the control group;

[0082] Determine the influence weight of the candidate test experiment based on the composite test indicator of the at least one experimental group, the experimental group traffic, and the benchmark amount corresponding to the composite test indicator;

[0083] The candidate test experiments are filtered based on the impact weights to obtain a target test experiment.

[0084] Optionally, the indicator influencing factor determination module 240 is further used to:

[0085] Determine the offset information between the composite test index of the target test experiment and the reference quantity as the first offset information; wherein the first offset information includes an offset direction;

[0086] Determine the offset information of the basic test indicator according to the first offset information as the second offset information; wherein the second offset information includes an offset direction and an offset weight;

[0087] An influencing factor is determined according to the second offset information.

[0088] Optionally, it also includes: an experiment partitioning module, which is used to:

[0089] Obtaining the delivery volume of the control group, the delivery volume of the experimental group, the grayscale delivery volume and the experimental traffic of the candidate test experiment;

[0090] The candidate test experiments are divided into grayscale experiments and formal experiments according to the delivery volume of the control group, the delivery volume of the experimental group, the delivery volume of the experimental version and the experimental traffic.

[0091] Optionally, it further includes: an abnormal period determination module, which is used to:

[0092] For the formal experiment, performing normality check on the composite test indicators within a set period in the formal experiment;

[0093] If the composite test index does not conform to the normal distribution, determining the change rates of multiple composite test indexes within a set period in the formal experiment;

[0094] If the maximum composite test indicator change rate is greater than a set value, the time period corresponding to the maximum composite test indicator normalization rate is determined as an abnormal time period.

[0095] Optionally, the candidate test experiment acquisition module 210 is further configured to:

[0096] The formal experiments are screened according to the abnormal time period to obtain candidate formal experiments.

[0097] The data processing device provided in the embodiments of the present disclosure can execute the data processing method provided in any embodiment of the present disclosure, and has the corresponding functional modules and beneficial effects of the execution method.

[0098] It is worth noting that the various units and modules included in the above-mentioned device are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the protection scope of the embodiments of the present disclosure.

[0099] Figure 3 This is a schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure. Figure 3 , which shows an electronic device (eg, Figure 3 The terminal device in the embodiment of the present disclosure may include but is not limited to mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 3 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0100] like Figure 3 As shown, the electronic device 500 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage device 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the electronic device 500 are also stored. The processing device 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An edit / output (I / O) interface 505 is also connected to the bus 504.

[0101] Typically, the following devices may be connected to the I / O interface 505: an input device 506 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 507 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 508 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 509. The communication device 509 may allow the electronic device 500 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 3 The electronic device 500 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0102] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 509, or installed from a storage device 508, or installed from a ROM 502. When the computer program is executed by the processing device 501, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0103] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0104] The electronic device provided by the embodiment of the present disclosure and the data processing method provided by the above embodiment belong to the same inventive concept. The technical details not fully described in this embodiment can be referred to the above embodiment, and this embodiment has the same beneficial effects as the above embodiment.

[0105] The embodiments of the present disclosure provide a computer storage medium on which a computer program is stored. When the program is executed by a processor, the data processing method provided by the above embodiments is implemented.

[0106] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0107] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0108] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0109] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device:

[0110] The above-mentioned computer-readable medium carries one or more programs. When the above-mentioned one or more programs are executed by the electronic device, the electronic device: screens the test experiments based on preset screening conditions to obtain candidate test experiments; determines the composite test indicators of the candidate test experiments; wherein the composite test indicators are determined by basic test indicators; filters the candidate test experiments based on the composite test indicators to obtain target test experiments; and determines indicator influencing factors based on the target test experiments.

[0111] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including, but not limited to, object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0112] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0113] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware. The name of a unit does not limit the unit itself in some cases. For example, the first acquisition unit may also be described as a "unit for acquiring at least two Internet Protocol addresses".

[0114] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0115] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0116] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

[0117] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0118] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

Claims

1. A data processing method, characterized in that: include: Screening the test experiments based on preset screening conditions to obtain candidate test experiments; Determine a composite test indicator of the candidate test experiment; wherein the composite test indicator is determined by a basic test indicator; Filter the candidate test experiments based on the composite test indicator to obtain a target test experiment; Determine the index influencing factors based on the target test experiment.

2. The method according to claim 1, characterized in that Determine the composite test indicators of the candidate test experiment, including: Obtaining basic test indicators of at least one experimental group in the candidate test experiment, wherein the basic test indicator includes at least one of the following: delivery volume, display volume, click volume, and conversion volume; Combining the basic test indicators of the at least one experimental group; A logical operation is performed on the combined basic test indicators to obtain a composite test indicator of the candidate test experiment.

3. The method according to claim 1, characterized in that Filtering the candidate test experiments based on the composite test indicator to obtain a target test experiment includes: Obtaining a composite test indicator of at least one experimental group in the candidate test experiment, the experimental group flow, and a reference quantity corresponding to the composite test indicator; wherein the reference quantity is represented by the composite test indicator of the control group; Determine the influence weight of the candidate test experiment based on the composite test indicator of the at least one experimental group, the experimental group traffic, and the benchmark amount corresponding to the composite test indicator; The candidate test experiments are filtered based on the impact weights to obtain a target test experiment.

4. The method according to claim 3, characterized in that Based on the target test experiment, the influencing factors of the indicator are determined, including: Determine the offset information between the composite test index of the target test experiment and the reference quantity as the first offset information; wherein the first offset information includes an offset direction; Determine the offset information of the basic test indicator according to the first offset information as the second offset information; wherein the second offset information includes an offset direction and an offset weight; An influencing factor is determined according to the second offset information.

5. The method according to claim 2, characterized in that: Before determining the composite test index of the candidate test experiment, it also includes: Obtaining the delivery volume of the control group, the delivery volume of the experimental group, the grayscale delivery volume and the experimental traffic of the candidate test experiment; The candidate test experiments are divided into grayscale experiments and formal experiments according to the delivery volume of the control group, the delivery volume of the experimental group, the delivery volume of the experimental version and the experimental traffic.

6. The method according to claim 5, characterized in that For the formal experiment, performing normality check on the composite test indicators within a set period in the formal experiment; If the composite test index does not conform to the normal distribution, determining the change rates of multiple composite test indexes within a set period in the formal experiment; If the maximum composite test indicator change rate is greater than a set value, the time period corresponding to the maximum composite test indicator normalization rate is determined as an abnormal time period.

7. The method according to claim 6, characterized in that The test experiments are screened based on the preset screening conditions to obtain candidate test experiments, including: The formal experiments are screened according to the abnormal time period to obtain candidate formal experiments.

8. A data processing device, characterized in that: include: A candidate test experiment acquisition module is used to screen the test experiments based on preset screening conditions to obtain candidate test experiments; A composite test indicator determination module, used to determine the composite test indicator of the candidate test experiment; wherein the composite test indicator is determined by the basic test indicator; A target test experiment acquisition module, used to filter the candidate test experiments based on the composite test indicator to obtain a target test experiment; The indicator influencing factor determination module is used to determine the indicator influencing factors based on the target test experiment.

9. An electronic device, characterized in that: The electronic device comprises: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the data processing method according to any one of claims 1 to 7.

10. A storage medium comprising computer executable instructions, wherein the computer executable instructions are used to perform the data processing method according to any one of claims 1 to 7 when executed by a computer processor.

Images