A method, device, equipment and medium for obtaining information delivery effect data

Abstract

The embodiments of the present invention disclose a method, device, equipment and medium for obtaining information delivery effect data. The method comprises: obtaining a log file of a terminal device, wherein the log file contains element behavior data for characterizing the user's operation behavior on a target delivery information element displayed on the terminal device; determining a characterization parameter value of a delivery effect characterization parameter corresponding to the target delivery information element based on the element behavior data; determining information delivery effect data corresponding to the target delivery information element based on the characterization parameter value; wherein the information delivery effect data comprises parameter characterization effect data, and the parameter characterization effect data is used to characterize the effect data corresponding to the delivery effect characterization parameter. The embodiments of the present invention solve the problem of inherent defects in existing delivery effect characterization parameters by determining parameter characterization effect data for characterizing the effect data corresponding to the delivery effect characterization parameter based on the characterization parameter value.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of information delivery technology, and in particular, to a method, device, equipment and medium for acquiring information delivery effect data. Background Art

[0002] Information placement means that the information provider places product information on an information platform, and the information platform displays the product information so that users can browse the product information through the information platform.

[0003] Among them, information delivery test refers to determining the delivery effect of the delivered information based on the user behavior data corresponding to the delivered information. There are two existing methods for determining the delivery effect of the delivered information: the first is to determine the delivery effect of the delivered information based on the parameter value of a delivery effect characterization parameter, and the other is to accumulate the parameter values ​​of multiple delivery effect characterization parameters based on weights, and determine the delivery effect of the delivered information based on the result of the accumulated calculation.

[0004] In the process of implementing the present invention, it is found that there are at least the following technical problems in the prior art:

[0005] In the first method mentioned above, since the parameter characterizing the delivery effect is single, the delivery effect determination result is one-sided. In the second method, there may be a large correlation between the multiple delivery effect characterization parameters, resulting in the problem of multicollinearity in the accumulated results. Summary of the invention

[0006] The embodiments of the present invention provide a method, device, equipment and medium for obtaining information delivery effect data to overcome the inherent defects of existing delivery effect characterization parameters, so that the information delivery effect data can better reflect the delivery effect of the delivered information.

[0007] In a first aspect, an embodiment of the present invention provides a method for obtaining information delivery effect data, the method comprising:

[0008] Acquire a log file of a terminal device, wherein the log file contains element behavior data for characterizing an operation behavior of a user corresponding to a target delivery information element on the terminal device;

[0009] Based on the element behavior data, determining a representation parameter value of a delivery effect representation parameter corresponding to the target delivery information element;

[0010] Based on the representation parameter value, information delivery effect data corresponding to the target delivery information element is determined; wherein the information delivery effect data includes parameter representation effect data, and the parameter representation effect data is used to represent effect data corresponding to the delivery effect representation parameter.

[0011] In a second aspect, an embodiment of the present invention further provides a device for acquiring information delivery effect data, the device comprising:

[0012] A log file acquisition module, used to acquire a log file of a terminal device, wherein the log file contains element behavior data for characterizing an operation behavior of a user corresponding to a target delivery information element on the terminal device;

[0013] A characterization parameter value determination module, used to determine a characterization parameter value of a delivery effect characterization parameter corresponding to the target delivery information element based on the element behavior data;

[0014] The information delivery effect data determination module is used to determine the information delivery effect data corresponding to the target delivery information element based on the representation parameter value; wherein the information delivery effect data includes parameter representation effect data, and the parameter representation effect data is used to represent the effect data corresponding to the delivery effect representation parameter.

[0015] In a third aspect, an embodiment of the present invention further provides an electronic device, the electronic device comprising:

[0016] one or more processors;

[0017] A memory for storing one or more programs;

[0018] When the one or more programs are executed by the one or more processors, the one or more processors implement any of the above-mentioned methods for acquiring information delivery effect data.

[0019] In a fourth aspect, an embodiment of the present invention further provides a storage medium comprising computer executable instructions, wherein the computer executable instructions, when executed by a computer processor, are used to execute any of the above-mentioned methods for acquiring information delivery effect data.

[0020] The embodiments of the above invention have the following advantages or beneficial effects:

[0021] The embodiment of the present invention solves the problem of inherent defects of existing delivery effect characterization parameters by determining parameter characterization effect data used to characterize effect data corresponding to the delivery effect characterization parameters based on the characterization parameter values, so that the information delivery effect data better reflects the delivery effect of the delivery information, thereby helping to provide a reference for users to adjust their information delivery strategies. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 This is a flowchart of a method for obtaining information delivery effect data provided in Example 1 of the present invention.

[0023] Figure 2 This is a flow chart of a method for acquiring information delivery effect data provided in Embodiment 2 of the present invention.

[0024] Figure 3 This is a flowchart of a method for obtaining information delivery effect data provided in Embodiment 3 of the present invention.

[0025] Figure 4 This is a flow chart of a method for obtaining statistical data provided in Embodiment 3 of the present invention.

[0026] Figure 5 It is a schematic diagram of a device for acquiring information delivery effect data provided by Embodiment 4 of the present invention.

[0027] Figure 6 It is a structural schematic diagram of an electronic device provided in Embodiment 5 of the present invention. DETAILED DESCRIPTION

[0028] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0029] Embodiment 1

[0030] Figure 1 This is a flowchart of a method for obtaining information delivery effect data provided by the first embodiment of the present invention. This embodiment is applicable to the case of obtaining information delivery effect data during the delivery information test. The method can be executed by an information delivery effect data acquisition device, which can be implemented in software and / or hardware, and can be configured in a terminal device. Specifically, it includes the following steps:

[0031] S110: Obtain a log file of the terminal device.

[0032] Exemplarily, the terminal device may be a mobile terminal, a laptop computer, a tablet computer, a desktop computer, a server or other intelligent terminal.

[0033] Specifically, the log file contains the test setting data of the information delivery test and the test data generated during the test process. The test setting data can be used to characterize the test parameters of the information delivery test. Specifically, the test parameters include the information delivery experiment ID (Identity document), the information delivery plan ID, the information delivery unit ID and the information delivery element ID. Each information delivery experiment ID corresponds to at least one information delivery plan ID, each information delivery plan ID corresponds to at least one information delivery unit ID, and each information delivery unit ID corresponds to at least one information delivery element. Exemplarily, the information delivery element can refer to a certain material in the delivery information, such as pictures, text, voice or video clips, etc. The information delivery element can also be the target population for the delivery information, such as the population of a certain city, men or women, etc. The specific content of the information delivery element is not limited here, and can be set according to actual test needs.

[0034] The test data may be the data recorded during the user characterization test. In this embodiment, the log file contains element behavior data for characterizing the user's operation behavior on the terminal device corresponding to the target delivery information element. Specifically, the element behavior data includes but is not limited to element display data, element click data, element browsing data, element purchase data, element order data, and element like data at each moment.

[0035] S120. Determine, based on the element behavior data, a representation parameter value of a delivery effect representation parameter corresponding to the target delivery information element.

[0036] In one embodiment, optionally, based on the element behavior data, behavior statistical data corresponding to the element behavior data is determined, and based on the behavior statistical data, a characterization parameter value of a delivery effect characterization parameter corresponding to the target delivery information element is determined.

[0037] Specifically, the element behavior data within the preset delivery time window is counted to obtain the behavior statistics data. For example, assuming that the preset delivery time window is 8:00-8:30, and the element behavior data is element click data, the data volume of the element click data within the preset delivery time window is used as the behavior statistics data, and for example, the behavior statistics data is the number of clicks.

[0038] In one embodiment, the delivery effect characterization parameter may include click-through rate and / or conversion rate. The characterization parameter value of the click-through rate satisfies the formula: click-through rate = number of clicks / number of impressions, and the characterization parameter value of the conversion rate satisfies the formula: conversion rate = order line / number of clicks.

[0039] S130. Determine information delivery effect data corresponding to the target delivery information element based on the representation parameter value.

[0040] In this embodiment, the information delivery effect data includes parameter representation effect data, and the parameter representation effect data is used to represent effect data corresponding to the delivery effect representation parameter.

[0041] In one embodiment, optionally, the parameter characterization effect data includes at least one of a confidence interval, a winning probability and an effect identifier corresponding to the delivery effect characterization parameter; wherein the confidence interval is determined based on the characterization parameter value of the target delivery information element corresponding to at least one information delivery unit, the winning probability is determined based on the characterization parameter value corresponding to the target delivery information element and the characterization parameter value corresponding to the reference delivery information element in the information delivery unit other than the target delivery information element, and the effect identifier is determined based on the winning probability; the information delivery unit includes at least two delivery information elements.

[0042] In one embodiment, optionally, based on the characterization parameter value, information delivery effect data corresponding to the target delivery information element is determined, including: taking at least one characterization parameter value corresponding to the delivery effect characterization parameter as the parameter average value corresponding to the delivery effect characterization parameter, and determining the parameter variance corresponding to the delivery effect characterization parameter based on each characterization parameter value; and determining the parameter characterization effect data of the delivery effect characterization parameter corresponding to the target delivery information element based on each parameter average value and each parameter variance.

[0043] Specifically, the parameter average value satisfies the formula: parameter average value=characterization parameter value. The parameter variance satisfies the formula: parameter variance=characterization parameter value*(1-characterization parameter value).

[0044] In one embodiment, for the characterization parameter value of the target delivery information element corresponding to each information delivery unit, the characterization parameter value is taken as the parameter average value, and the parameter variance is determined based on the characterization parameter value; based on the parameter average value and parameter variance corresponding to at least one information delivery unit, the confidence interval of the delivery effect characterization parameter corresponding to the target delivery information element is determined.

[0045] Specifically, a probability distribution function is constructed based on the average value of each parameter and the variance of each parameter. In one embodiment, when the number of information delivery units is two, for example, assuming that X i and Y i denote the delivery effect characterization parameters of the i-th delivery information element corresponding to the first information delivery unit and the second information delivery unit, respectively. Let θ = E[X i ]-E[Y i ],E[X i ] and E[Y i] represent the expected values ​​of the parameters representing the effect of the delivery. Among them, the hypothesis test is H 0 :θ=0, the alternative hypothesis is H 1 :θ≠0. Then the probability distribution function satisfies the formula:

[0046]

[0047] in, represents the average value of the parameters corresponding to the i-th delivery information element in the first information delivery unit, represents the average value of the parameters corresponding to the i-th information element in the second information delivery unit, Var n x i represents the parameter variance corresponding to the i-th information element in the first information delivery unit, Var n y i represents the parameter variance corresponding to the i-th delivery information element in the second information delivery unit, n A The parameter sample size of the parameter representation effect data corresponding to the first information delivery unit, n B represents the parameter sample size of the parameter characterization effect data corresponding to the second information delivery unit, and τ represents the adjustment parameter.

[0048] Exemplarily, when the parameter characterizing the effect data is click-through rate, the parameter sample size is the number of impressions; and when the parameter characterizing the effect data is conversion rate, the parameter sample size is the number of clicks.

[0049] When the confidence level is ε, we can get:

[0050]

[0051] Right now

[0052]

[0053] In one embodiment, optionally, the confidence interval includes a difference confidence interval of delivery effect characterization parameters of target delivery information elements corresponding to two information delivery units and / or a confidence interval of delivery effect characterization parameters of the target delivery information element corresponding to the target information delivery unit.

[0054] Specifically, the difference confidence interval satisfies the formula:

[0055]

[0056] Specifically, the confidence interval of the delivery effect characterization parameter of the target delivery information element corresponding to the target information delivery unit satisfies the formula:

[0057]

[0058] The advantage of such a setting is that the data delay caused by the information delivery system may cause temporary problems in the delivery effect characterization parameters. Taking the click-through rate as an example, if you only pay attention to the characterization parameter value of the click-through rate, when the number of displays in the early stage of information delivery is small, a small amount of click data may make a certain information delivery unit have a very high characterization parameter value. Even if the number of displays is large enough, due to the randomness of the delivery experiment diversion, it is difficult to ensure that the characterization parameter value obtained by the test is completely consistent with the characterization parameter value after the actual delivery. The confidence interval corresponding to the delivery effect characterization parameter provided in this embodiment can effectively overcome the above-mentioned defects. The confidence interval represents that the true parameter value of the delivery effect characterization parameter falls within the interval with a certain probability, so that the confidence interval is an effective supplement to the existing single characterization parameter value.

[0059] In one embodiment, optionally, a test report is generated based on the information delivery effect data and the test report is output, wherein the test report includes element behavior data, unit behavior data, behavior statistics data, click-through rate, conversion rate, return on investment, confidence interval, winning probability and winning mark, etc.

[0060] The technical solution of this embodiment solves the problem of inherent defects in existing delivery effect characterization parameters by determining parameter characterization effect data used to characterize effect data corresponding to delivery effect characterization parameters based on characterization parameter values, so that the information delivery effect data better reflects the delivery effect of the delivery information, thereby helping to provide a reference for users to adjust their information delivery strategies.

[0061] Embodiment 2

[0062] Figure 2 It is a flow chart of a method for obtaining information delivery effect data provided by the second embodiment of the present invention. The technical solution of this embodiment is a further refinement based on the above embodiment. Optionally, based on the characterization parameter value, the information delivery effect data corresponding to the target delivery information element is determined, including: taking at least one characterization parameter value corresponding to the delivery effect characterization parameter as the parameter average value corresponding to the delivery effect characterization parameter, and determining the parameter variance corresponding to the delivery effect characterization parameter based on each characterization parameter value; based on each parameter average value and each parameter variance, determining the parameter characterization effect data of the delivery effect characterization parameter corresponding to the target delivery information element.

[0063] The specific implementation steps of this embodiment include:

[0064] S210: Obtain the log file of the terminal device.

[0065] In this embodiment, the log file contains element behavior data corresponding to at least two delivery information elements in the information delivery unit.

[0066] S220: Determine, based on the element behavior data, a representation parameter value of a delivery effect representation parameter corresponding to the target delivery information element.

[0067] S230. For the representation parameter value of the delivery effect representation parameter corresponding to each delivery information element in the information delivery unit, take the representation parameter value as the parameter average value, and determine the parameter variance based on the representation parameter value.

[0068] The information delivery unit includes a target delivery information element and at least one reference delivery information element. Exemplarily, when the target delivery information element is picture A, the reference delivery information element may be picture B, picture C, and picture D. When the target delivery information element is crowd A, the reference delivery information element may be crowd B, crowd C, and crowd D.

[0069] S240: Determine the winning probability of the delivery effect characterization parameter corresponding to the target delivery information element based on the parameter average values ​​and parameter variances corresponding to at least two delivery information elements in the information delivery unit.

[0070] In one embodiment, optionally, the winning probability corresponding to the delivery effect characterization parameter satisfies the formula:

[0071]

[0072] Among them, q i represents the winning probability corresponding to the i-th delivery information element, Indicates the average value of the parameter corresponding to the i-th delivery information element, Var n x i represents the parameter variance corresponding to the i-th information element, δ i represents the adjustment parameter corresponding to the i-th delivery information element, n i represents the parameter sample size corresponding to the parameter representation effect data of the i-th delivery information element, and n represents the number of delivery information elements contained in the information delivery unit.

[0073] Exemplarily, when the parameter characterizing the effect data is click-through rate, the parameter sample size is the number of impressions; and when the parameter characterizing the effect data is conversion rate, the parameter sample size is the number of clicks.

[0074] On the basis of the above embodiment, optionally, when the log file contains element behavior data corresponding to a delivery time window, the parameter characterization effect data of the delivery effect characterization parameter corresponding to the target delivery information element is determined based on the average value of each parameter and the variance of each parameter, including: after determining the winning probability of the delivery effect characterization parameter corresponding to the target delivery information element based on the average value of each parameter and the variance of each parameter, if the winning probability is greater than a preset probability threshold, the effect identifier corresponding to the delivery effect characterization parameter is set to a winning identifier; if the winning probability is less than or equal to the preset probability threshold, the effect identifier corresponding to the delivery effect characterization parameter is set to a failure identifier.

[0075] The delivery time window represents a time period, and exemplarily, the time interval corresponding to the delivery time window may be 10 minutes. The preset probability threshold may exemplarily be 90%.

[0076] On the basis of the above embodiment, optionally, the log file contains element behavior data corresponding to at least two delivery time windows respectively, and correspondingly, the winning probability includes the winning probability corresponding to the at least two delivery time windows respectively, and the parameter characterization effect data of the delivery effect characterization parameter corresponding to the target delivery information element is determined based on the parameter average value and the parameter variance respectively corresponding to the at least two delivery time windows, including: after determining at least two winning probabilities corresponding to the target delivery information element based on the parameter average value and the parameter variance respectively corresponding to the at least two delivery time windows, judging whether the winning probabilities corresponding to the delivery time windows of a preset number of consecutive windows are all greater than a preset probability threshold; if so, setting the effect identifier corresponding to the delivery effect characterization parameter to a winning identifier; if not, setting the effect identifier corresponding to the delivery effect characterization parameter to a failure identifier.

[0077] Exemplarily, the number of preset continuous windows can be 10 or 36. Specifically, the delivery time window of the preset number of continuous windows does not include the time of pausing the test, such as starting the test at 8:00, pausing the test at 9:00, and restarting the test at 10:00. Assuming that the time interval of the delivery time window is 10 minutes and the preset number of continuous windows is 36, the delivery time window of the preset number of continuous windows includes 8:00-9:00 and 10:00-13:00.

[0078] In the process of information delivery, due to various random factors, there are many reversals in the effect of information delivery, that is, the information delivery elements with excellent delivery effects in the early stage may not necessarily win. The technical solution of this embodiment determines the winning probability corresponding to the delivery effect characterization parameter, and further judges whether the winning probability corresponding to the delivery time window of the preset number of continuous windows is greater than the preset probability threshold, and determines the effect identifier corresponding to the delivery effect characterization parameter, which solves the problem that the characterization parameter value of a single delivery effect characterization parameter in the prior art does not accurately represent the delivery effect. The effect identifier of this embodiment can reflect the delivery effect represented by the delivery effect characterization parameter under a long time series, and has a certain stability, thereby providing a more reliable reference value for users to adjust information delivery strategies.

[0079] Embodiment 3

[0080] Figure 3 It is a flow chart of a method for obtaining information delivery effect data provided by Example 3 of the present invention. The technical solution of this embodiment is a further refinement based on the above embodiment. Optionally, the log file of the terminal device is obtained, including: obtaining unit behavior data and / or information delivery data of the information delivery unit corresponding to the target delivery information element in the previous delivery time window; based on the unit behavior data and / or information delivery data, adjusting the information delivery parameters corresponding to the information delivery unit in the current delivery time window; after the delivery of the information delivery unit is completed based on the adjusted information delivery parameters, executing the step of obtaining the element behavior data corresponding to the target delivery information element in the current delivery time window and determining the log file.

[0081] The specific implementation steps of this embodiment include:

[0082] S310: Acquire unit behavior data and / or information delivery data of the information delivery unit corresponding to the target delivery information element in the previous delivery time window.

[0083] Specifically, the entire information delivery test process is divided into at least two test processes corresponding to the delivery time windows. Exemplarily, the entire information delivery test process is 8:00-10:00. When the time interval corresponding to the delivery time window is one hour, the entire information delivery test process includes the 8:00-9:00 test process and the 9:00-10:00 test process. Specifically, when the current delivery time window is 9:00-10:00, the previous delivery time window is 8:00-9:00.

[0084] Specifically, the unit behavior data is used to characterize the behavior data of the user's operation behavior on the terminal device corresponding to the information delivery unit. The information delivery data is the cost data corresponding to the information delivery unit. Exemplarily, the information delivery data includes but is not limited to cost consumption data and reserved cost data.

[0085] S320: Based on the unit behavior data and / or the information delivery data, adjust the information delivery parameters corresponding to the information delivery unit in the current delivery time window.

[0086] Specifically, statistics are performed on unit behavior data in the previous delivery time window to obtain statistical behavior data, and / or statistics are performed on information delivery data in the previous time window to obtain statistical delivery data, and based on the statistical behavior data and / or statistical delivery data, information delivery parameters corresponding to the information delivery unit in the current delivery time window are adjusted.

[0087] Figure 4 It is a flow chart of a method for obtaining statistical data provided in Example 3 of the present invention. Specifically, the unit behavior data and information delivery data in the previous delivery time window are uploaded to the Kafka cluster system through the real-time data stream Flume tool, so that the Flink stream computing platform in the Kafka cluster system performs real-time calculations on the received element behavior data and information delivery data, and the calculated data is fully dumped (backed up) to obtain behavior statistics data and statistical delivery data. Behavior statistics data and statistical delivery data can be directly output to the network disk or output as a Hive table. Among them, exemplarily, Hive (data warehouse tool) can be used to map behavior statistics data and statistical delivery data to Hive tables. Among them, the Flume tool is a distributed and highly available system for massive log collection, aggregation and transmission. The Kafka cluster system is a distributed, partitioned, multi-copy and zookeeper-coordinated distributed messaging system that can process large amounts of data in real time to meet various demand scenarios. The Flink stream computing platform is an open source distributed stream computing and batch processing platform.

[0088] Specifically, in the process of adjusting information delivery parameters, behavioral statistical data and / or statistical delivery data in the network disk are obtained, and in the process of determining information delivery effect data, behavioral statistical data in the Hive table are read.

[0089] In this embodiment, the information delivery parameters include playback probability.

[0090] In one embodiment, optionally, based on the unit behavior data and / or the information delivery data, the information delivery parameters corresponding to the information delivery unit in the current delivery time window are adjusted, including: determining a first playback probability corresponding to the information delivery unit in the current time window based on the unit behavior data, and / or determining a second playback probability corresponding to the information delivery unit in the current time window based on the information delivery data; adjusting the information delivery parameters of the information delivery unit in the current time window based on the first playback probability and / or the second playback probability.

[0091] On the basis of the above embodiment, optionally, the unit behavior data includes unit click data and unit exposure data corresponding to the information delivery unit. Accordingly, based on the unit behavior data, the first playback probability corresponding to the information delivery unit in the current time window is determined, including: determining a first variable based on the unit click data, and determining a second variable based on the unit click data and the unit exposure data; based on the first variable and the second variable, determining the first playback probability corresponding to the information delivery unit in the current time window.

[0092] Specifically, the first variable α satisfies the formula: α=1+clicks, where clicks represents the number of clicks determined based on the unit click data. The second variable β satisfies the formula: β=1+imps-clicks, where imps represents the number of exposures determined based on the unit exposure data.

[0093] The first playback probability satisfies the probability density function of the Beta distribution. Specifically, the probability density function of the Beta distribution is:

[0094]

[0095] Among them, f(x; α, β) represents the first playback probability, is a Γ function, which represents a normalization coefficient. Wherein, exemplarily, when α=1, β=1, the Beta distribution decays to a 0-1 uniform distribution.

[0096] On the basis of the above embodiment, optionally, the information delivery data includes cost consumption data and delivery reservation ratio corresponding to the information delivery unit, and accordingly, based on the information delivery data, determining the second playback probability corresponding to the information delivery unit in the current time window includes: obtaining total cost consumption data corresponding to the information delivery plan; wherein the information delivery plan includes at least one information delivery unit; based on the total cost consumption data, the cost consumption data of the information delivery unit and the delivery reservation ratio, determining the second playback probability corresponding to the information delivery unit in the current time window; wherein the delivery reservation ratio is used to characterize the proportion of the delivery reserved data of the information delivery unit to the total delivery reserved data corresponding to the information delivery plan.

[0097] Specifically, the second playback probability satisfies the formula:

[0098]

[0099] Among them, F AG (D t,k ) represents the second playback probability, α represents the adjustment parameter, S t,k represents the cost consumption data of the kth information delivery unit in the information delivery plan within the previous delivery time window, S t,j represents the cost consumption data of the jth information delivery unit in the information delivery plan within the previous delivery time window, t represents the time interval corresponding to the previous delivery time window, j represents the number of information delivery units in the information delivery plan, and q k It represents the proportion of the reserved delivery data of the kth information delivery unit to the total reserved delivery data corresponding to the information delivery plan.

[0100] In one embodiment, the first play probability and the second play probability are determined based on the unit behavior data and the information delivery data, respectively, and the final play probability satisfies the formula:

[0101] P=f(x;α,β)*ρ+F AG *(1-ρ)

[0102] Wherein, ρ represents a probability adjustable parameter, and its value range is [0,1]. In one embodiment, optionally,

[0103] The advantage of this setting is that when the reserved delivery data is sufficient, the corresponding playback probability of the information delivery unit with a high click rate is higher, so that the information delivery unit with a high click rate will not consume the reserved delivery data in a short time, so that the test time of the information delivery unit is extended. For information delivery units with a low reserved delivery ratio or information delivery units with a low click rate, the playback probability will become smaller and smaller as the test progresses, thereby avoiding the problem of overplay and saving reserved delivery data.

[0104] S330: After the information delivery unit is delivered based on the adjusted information delivery parameters, element behavior data corresponding to the target delivery information element in the current delivery time window is obtained.

[0105] S340, determine whether the current delivery time window is the last delivery time window, if yes, execute S360, if not, execute S350.

[0106] S350: Use the current delivery time window as the previous delivery time window, and execute S310.

[0107] S360: Generate a log file based on the element behavior data corresponding to the target delivery information element in at least two delivery time windows.

[0108] S370. Based on at least two element behavior data in the log file, respectively determine the representation parameter values ​​of the delivery effect representation parameters corresponding to the target delivery information elements.

[0109] S380. Determine information delivery effect data corresponding to the target delivery information element based on each characterization parameter value.

[0110] Specifically, when the parameter characterizing the effect data in the information delivery effect data is a confidence interval or a winning probability, the information delivery effect data includes the confidence interval or the winning probability corresponding to at least two delivery time windows. When the parameter characterizing the effect data in the information delivery effect data is an effect identifier, the information delivery effect data includes the effect identifier corresponding to at least two delivery time windows.

[0111] The technical solution of this embodiment adjusts the information delivery parameters of the information delivery unit corresponding to the current delivery time window according to the unit behavior data and / or information delivery data, and obtains the element behavior data corresponding to the target delivery information element in the information delivery unit that has been delivered based on the adjusted information delivery parameters within the current delivery time window, thereby solving the problem of unreasonable delivery parameters of the information delivery unit, allowing information delivery units with sufficient reserved delivery data or a high click-through rate to have more display opportunities, thereby enabling the information delivery unit to win as soon as possible, saving information delivery costs and information delivery time, and improving information delivery efficiency.

[0112] Embodiment 4

[0113] Figure 5 4 is a schematic diagram of a device for acquiring information delivery effect data provided by Embodiment 4 of the present invention. This embodiment can be applied to the case of acquiring information delivery effect data during the delivery information test. The device can be implemented in software and / or hardware, and the device can be configured in a terminal device. The device for acquiring information delivery effect data includes: a log file acquisition module 410, a characterization parameter value determination module 420, and an information delivery effect data determination module 430.

[0114] The log file acquisition module 410 is used to acquire a log file of a terminal device, wherein the log file contains element behavior data for representing the operation behavior of a user on the terminal device corresponding to the target delivery information element;

[0115] The characterization parameter value determination module 420 is used to determine the characterization parameter value of the delivery effect characterization parameter corresponding to the target delivery information element based on the element behavior data;

[0116] The information delivery effect data determination module 430 is used to determine the information delivery effect data corresponding to the target delivery information element based on the representation parameter value; wherein the information delivery effect data includes parameter representation effect data, and the parameter representation effect data is used to represent the effect data corresponding to the delivery effect representation parameter.

[0117] The technical solution of this embodiment solves the problem of inherent defects in existing delivery effect characterization parameters by determining parameter characterization effect data used to characterize effect data corresponding to delivery effect characterization parameters based on characterization parameter values, so that the information delivery effect data better reflects the delivery effect of the delivery information, thereby helping to provide a reference for users to adjust their information delivery strategies.

[0118] On the basis of the above technical solution, optionally, the parameter characterization effect data includes at least one of a confidence interval, a winning probability and an effect identifier corresponding to the delivery effect characterization parameter; wherein the confidence interval is determined based on the characterization parameter value of the target delivery information element corresponding to at least one information delivery unit, the winning probability is determined based on the characterization parameter value corresponding to the target delivery information element and the characterization parameter value corresponding to the reference delivery information element in the information delivery unit other than the target delivery information element, and the effect identifier is determined based on the winning probability; the information delivery unit includes at least two delivery information elements.

[0119] On the basis of the above technical solution, optionally, the information delivery effect data determination module 430 includes:

[0120] a parameter average value determining unit, configured to use at least one characterization parameter value corresponding to the delivery effect characterization parameter as a parameter average value corresponding to the delivery effect characterization parameter, and determine a parameter variance corresponding to the delivery effect characterization parameter based on each characterization parameter value;

[0121] The parameter representation effect data determination unit is used to determine the parameter representation effect data of the delivery effect representation parameter corresponding to the target delivery information element based on the average value of each parameter and the variance of each parameter.

[0122] On the basis of the above technical solution, optionally, the confidence interval corresponding to the delivery effect characterization parameter satisfies the formula:

[0123]

[0124] in, represents the average value of the parameter corresponding to the i-th information element in the first information delivery unit, Var n x i represents the parameter variance corresponding to the i-th information element in the first information delivery unit, n AIt represents the parameter sample size of the parameter characterization effect data corresponding to the first information delivery unit, ε represents the confidence level, and τ represents the adjustment parameter.

[0125] On the basis of the above technical solution, optionally, the winning probability corresponding to the delivery effect characterization parameter satisfies the formula:

[0126]

[0127] Among them, q i represents the winning probability corresponding to the i-th delivery information element, Indicates the average value of the parameter corresponding to the i-th delivery information element, Var n x i represents the parameter variance corresponding to the i-th information element, δ i represents the adjustment parameter corresponding to the i-th delivery information element, n i represents the parameter sample size corresponding to the parameter representation effect data of the i-th delivery information element, and n represents the number of delivery information elements contained in the information delivery unit.

[0128] On the basis of the above technical solution, optionally, the log file includes element behavior data corresponding to at least two delivery time windows respectively, and accordingly, the winning probability includes the winning probability corresponding to at least two delivery time windows respectively, and the parameter characterization effect data determination unit is specifically used to:

[0129] After determining at least two winning probabilities corresponding to the target delivery information element based on the parameter averages and parameter variances respectively corresponding to at least two delivery time windows, it is determined whether the winning probabilities corresponding to the delivery time windows of a preset number of consecutive windows are all greater than a preset probability threshold;

[0130] If yes, the effect flag corresponding to the delivery effect representation parameter is set as the winning flag;

[0131] If not, the effect flag corresponding to the delivery effect characterization parameter is set to a failure flag.

[0132] Based on the above technical solution, optionally, the log file acquisition module 410 includes:

[0133] A unit behavior data acquisition unit, used to acquire unit behavior data and / or information delivery data of an information delivery unit corresponding to a target delivery information element in a previous delivery time window;

[0134] An information delivery parameter adjustment unit, configured to adjust information delivery parameters corresponding to information delivery units within a current delivery time window based on unit behavior data and / or information delivery data;

[0135] The log file determination unit is used to execute the steps of obtaining element behavior data corresponding to the target delivery information element in the current delivery time window and determining the log file after the delivery of the information delivery unit is completed based on the adjusted information delivery parameters.

[0136] On the basis of the above technical solution, optionally, the information delivery parameter adjustment unit includes:

[0137] A first play probability determination subunit is used to determine a first play probability corresponding to the information delivery unit in the current time window based on the unit behavior data; and / or,

[0138] A second playback probability determination subunit, configured to determine a second playback probability corresponding to the information delivery unit within the current time window based on the information delivery data;

[0139] The information delivery parameter subunit is used to adjust the information delivery parameters of the information delivery unit in the current time window based on the first playback probability and / or the second playback probability.

[0140] On the basis of the above technical solution, optionally, the unit behavior data includes unit click data and unit exposure data corresponding to the information delivery unit, and accordingly, the first playback probability determination subunit is specifically used to:

[0141] determining a first variable based on the unit click data, and determining a second variable based on the unit click data and the unit exposure data;

[0142] Based on the first variable and the second variable, a first playback probability corresponding to the information delivery unit in the current time window is determined.

[0143] On the basis of the above technical solution, optionally, the information delivery data includes cost consumption data and delivery reservation ratio corresponding to the information delivery unit, and accordingly, the second playback probability determination subunit is specifically used to:

[0144] Acquire total cost consumption data corresponding to the information delivery plan; wherein the information delivery plan includes at least one information delivery unit;

[0145] Based on the total cost consumption data, the cost consumption data of the information delivery unit and the delivery reservation ratio, the second playback probability corresponding to the information delivery unit in the current time window is determined; wherein the delivery reservation ratio is used to represent the ratio of the delivery reserved data of the information delivery unit to the total delivery reserved data corresponding to the information delivery plan.

[0146] The device for acquiring information delivery effect data provided by the embodiment of the present invention can be used to execute the method for acquiring information delivery effect data provided by the embodiment of the present invention, and has the corresponding functions and beneficial effects of the execution method.

[0147] It is worth noting that in the embodiment of the above-mentioned device for obtaining information delivery effect data, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0148] Embodiment 5

[0149] Figure 6 It is a structural diagram of an electronic device provided by Embodiment 5 of the present invention. The embodiment of the present invention provides services for the implementation of the method for obtaining information delivery effect data of the above embodiment of the present invention, and can configure the device for obtaining information delivery effect data in the above embodiment. Figure 6 A block diagram of an exemplary electronic device 12 suitable for use in implementing embodiments of the present invention is shown. Figure 6 The electronic device 12 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0150] like Figure 6 As shown, the electronic device 12 is in the form of a general purpose computing device. The components of the electronic device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 that connects various system components (including the system memory 28 and the processing unit 16).

[0151] Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, or a local bus using any of a variety of bus architectures. These architectures include, by way of example, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MAC) bus, an Enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.

[0152] The electronic device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the electronic device 12, including volatile and non-volatile media, removable and non-removable media.

[0153] The system memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. The electronic device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 34 may be used to read and write non-removable, non-volatile magnetic media ( Figure 6not shown, usually called a "hard drive"). Although Figure 6 Not shown in the figure, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 18 via one or more data medium interfaces. The memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present invention.

[0154] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in the memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. The program modules 42 generally perform the functions and / or methods of the embodiments described herein.

[0155] The electronic device 12 may also communicate with one or more external devices 14 (e.g., keyboards, pointing devices, displays 24, etc.), one or more devices that enable a user to interact with the electronic device 12, and / or any device that enables the electronic device 12 to communicate with one or more other computing devices (e.g., network cards, modems, etc.). Such communication may be performed through an input / output (I / O) interface 22. Furthermore, the electronic device 12 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) through a network adapter 20. Figure 6 As shown, the network adapter 20 communicates with other modules of the electronic device 12 via the bus 18. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with the electronic device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0156] The processing unit 16 executes various functional applications and data processing by running the programs stored in the system memory 28, such as implementing the method for acquiring information delivery effect data provided in the embodiment of the present invention.

[0157] Through the above electronic device, the inherent defects of existing delivery effect characterization parameters are solved, so that the information delivery effect data can better reflect the delivery effect of the delivered information, thereby helping to provide a reference for users to adjust their information delivery strategies.

[0158] Embodiment 6

[0159] Embodiment 6 of the present invention further provides a storage medium including computer executable instructions, and when the computer executable instructions are executed by a computer processor, they are used to execute a method for obtaining information delivery effect data, the method comprising:

[0160] Obtaining a log file of a terminal device, wherein the log file contains element behavior data for characterizing an operation behavior of a user on the terminal device corresponding to a target delivery information element;

[0161] Determine, based on the element behavior data, a representation parameter value of a delivery effect representation parameter corresponding to the target delivery information element;

[0162] Based on the representation parameter value, information delivery effect data corresponding to the target delivery information element is determined; wherein the information delivery effect data includes parameter representation effect data, and the parameter representation effect data is used to represent effect data corresponding to the delivery effect representation parameter.

[0163] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, a device or a device or used in combination with it.

[0164] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0165] The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0166] Computer program code for performing the operation of the present invention may be written in one or more programming languages ​​or combinations thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0167] Of course, the storage medium containing computer executable instructions provided by an embodiment of the present invention, whose computer executable instructions are not limited to the above method operations, can also execute related operations in the method for obtaining information delivery effect data provided by any embodiment of the present invention.

[0168] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A method for obtaining information delivery effect data, characterized in that: include: Acquire a log file of a terminal device, wherein the log file contains element behavior data for characterizing an operation behavior of a user corresponding to a target delivery information element on the terminal device; Based on the element behavior data, determining a representation parameter value of a delivery effect representation parameter corresponding to the target delivery information element; Based on the representation parameter value, determine the information delivery effect data corresponding to the target delivery information element; wherein the information delivery effect data includes parameter representation effect data, and the parameter representation effect data is used to represent the effect data corresponding to the delivery effect representation parameter; The parameter characterization effect data includes at least one of a confidence interval, a winning probability and an effect identifier corresponding to the delivery effect characterization parameter; wherein the confidence interval is determined based on the characterization parameter value of the target delivery information element corresponding to at least one information delivery unit, the winning probability is determined based on the characterization parameter value corresponding to the target delivery information element and the characterization parameter value corresponding to the reference delivery information element in the information delivery unit other than the target delivery information element, and the effect identifier is determined based on the winning probability; the information delivery unit includes at least two delivery information elements.

2. The method according to claim 1, characterized in that The determining, based on the characterization parameter value, information delivery effect data corresponding to the target delivery information element comprises: taking at least one characterization parameter value corresponding to the delivery effect characterization parameter as the parameter average value corresponding to the delivery effect characterization parameter, and determining the parameter variance corresponding to the delivery effect characterization parameter based on each of the characterization parameter values; Based on the parameter averages and the parameter variances, parameter characterization effect data of the delivery effect characterization parameter corresponding to the target delivery information element is determined.

3. The method according to claim 2, characterized in that The confidence interval corresponding to the delivery effect characterization parameter satisfies the formula: in, represents the average value of the parameter corresponding to the i-th information element in the first information delivery unit, Var n x i represents the parameter variance corresponding to the i-th information element in the first information delivery unit, n A It represents the parameter sample size of the parameter characterization effect data corresponding to the first information delivery unit, ε represents the confidence level, and τ represents the adjustment parameter.

4. The method according to claim 2, characterized in that: The winning probability corresponding to the delivery effect characterization parameter satisfies the formula: Among them, q i represents the winning probability corresponding to the i-th delivery information element, Indicates the average value of the parameter corresponding to the i-th delivery information element, Var n x i represents the parameter variance corresponding to the i-th information element, δ i represents the adjustment parameter corresponding to the i-th delivery information element, n i represents the parameter sample size corresponding to the parameter representation effect data of the i-th delivery information element, and n represents the number of delivery information elements contained in the information delivery unit.

5. The method according to claim 4, characterized in that The log file includes element behavior data corresponding to at least two delivery time windows respectively. Accordingly, the winning probability includes the winning probability corresponding to at least two delivery time windows respectively. The parameter characterization effect data for determining the delivery effect characterization parameter corresponding to the target delivery information element based on the average value of each parameter and the variance of each parameter includes: After determining at least two winning probabilities corresponding to the target delivery information element based on the parameter averages and parameter variances respectively corresponding to at least two delivery time windows, determining whether the winning probabilities corresponding to the delivery time windows of a preset number of consecutive windows are all greater than a preset probability threshold; If yes, then setting the effect identifier corresponding to the delivery effect characterization parameter as the winning identifier; If not, the effect flag corresponding to the delivery effect characterization parameter is set to a failure flag.

6. The method according to claim 5, characterized in that The step of obtaining the log file of the terminal device includes: Acquire unit behavior data and / or information delivery data of the information delivery unit corresponding to the target delivery information element in the previous delivery time window; Based on the unit behavior data and / or information delivery data, adjusting the information delivery parameters corresponding to the information delivery unit in the current delivery time window; After the information delivery unit is delivered based on the adjusted information delivery parameters, the steps of obtaining element behavior data corresponding to the target delivery information element within the current delivery time window and determining the log file are performed.

7. The method according to claim 6, characterized in that The adjusting of information delivery parameters corresponding to the information delivery unit in the current delivery time window based on the unit behavior data and / or information delivery data includes: Determine, based on the unit behavior data, a first playback probability corresponding to the information delivery unit in the current time window, and / or, based on the information delivery data, determine a second playback probability corresponding to the information delivery unit in the current time window; Based on the first playback probability and / or the second playback probability, the information delivery parameters of the information delivery unit in the current time window are adjusted.

8. The method according to claim 7, characterized in that The unit behavior data includes unit click data and unit exposure data corresponding to the information delivery unit. Accordingly, determining the first playback probability corresponding to the information delivery unit in the current time window based on the unit behavior data includes: determining a first variable based on the unit click data, and determining a second variable based on the unit click data and the unit exposure data; Based on the first variable and the second variable, a first playback probability corresponding to the information delivery unit in the current time window is determined.

9. The method according to claim 7, characterized in that: The information delivery data includes cost consumption data and delivery reservation ratio corresponding to the information delivery unit. Accordingly, determining the second playback probability corresponding to the information delivery unit in the current time window based on the information delivery data includes: Acquire total cost consumption data corresponding to an information delivery plan; wherein the information delivery plan includes at least one information delivery unit; Based on the total cost consumption data, the cost consumption data of the information delivery unit and the delivery reservation ratio, the second playback probability corresponding to the information delivery unit in the current time window is determined; wherein the delivery reservation ratio is used to represent the ratio of the delivery reserved data of the information delivery unit to the total delivery reserved data corresponding to the information delivery plan.

10. A device for acquiring information delivery effect data, characterized in that: include: A log file acquisition module, used to acquire a log file of a terminal device, wherein the log file contains element behavior data for characterizing an operation behavior of a user corresponding to a target delivery information element on the terminal device; A characterization parameter value determination module, used to determine a characterization parameter value of a delivery effect characterization parameter corresponding to the target delivery information element based on the element behavior data; An information delivery effect data determination module, used to determine the information delivery effect data corresponding to the target delivery information element based on the representation parameter value; wherein the information delivery effect data includes parameter representation effect data, and the parameter representation effect data is used to represent the effect data corresponding to the delivery effect representation parameter; The parameter characterization effect data includes at least one of a confidence interval, a winning probability and an effect identifier corresponding to the delivery effect characterization parameter; wherein the confidence interval is determined based on the characterization parameter value of the target delivery information element corresponding to at least one information delivery unit, the winning probability is determined based on the characterization parameter value corresponding to the target delivery information element and the characterization parameter value corresponding to the reference delivery information element in the information delivery unit other than the target delivery information element, and the effect identifier is determined based on the winning probability; the information delivery unit includes at least two delivery information elements.

11. An electronic device, characterized in that: The electronic device comprises: one or more processors; A memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method for acquiring information delivery effect data as described in any one of claims 1-9.

12. A storage medium containing computer executable instructions, characterized in that: The computer executable instructions, when executed by a computer processor, are used to execute the method for obtaining information delivery effect data as described in any one of claims 1-9.

Images