Method, device and equipment for calibrating advertisement estimated conversion rate

Abstract

The present disclosure provides a method, device and equipment for calibrating an estimated conversion rate of an advertisement. The method for calibrating the estimated conversion rate of the advertisement comprises: determining a calibration time period for calibrating an estimated conversion rate of a target advertisement; determining a plurality of advertisement sets associated with the target advertisement based on at least part of a plurality of advertisement attributes of the target advertisement, wherein the advertisements in each of the plurality of advertisement sets have at least one common attribute, and at least part of the at least one common attribute is included in the plurality of advertisement attributes; obtaining an estimated conversion rate and an actual conversion rate of each of the plurality of advertisement sets; determining a reference advertisement set used as a calibration reference from the plurality of advertisement sets, and determining a calibration parameter using the estimated conversion rate and the actual conversion rate of the reference advertisement set; and calibrating the estimated conversion rate of the target advertisement in the calibration time period using the calibration parameter.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of multimedia, and more particularly, to a method and apparatus for calibrating predicted conversion rate of an advertisement, a computer readable storage medium, and a computer program product. BACKGROUND

[0002] In the field of advertisements, advertisement conversion rate prediction plays an important role in realizing accurate advertisement delivery, evaluating advertisement delivery effect, etc. Although the predicted conversion rate (PCVR) of an advertisement obtained by using a current conversion rate prediction model has good overall performance, the accuracy of the PCVR is not high for specific industries, specific audiences, specific content, etc. Therefore, after obtaining the PCVR of an advertisement, it is often necessary to calibrate the PCVR of the advertisement in combination with various factors such as the industry, audience, content, brand, etc. of the advertisement, for example, to calibrate the PCVR for a specific industry in a targeted manner to obtain the most accurate PCVR value.

[0003] In addition, for a single advertisement, the PCVR of the advertisement in some time period in the entire delivery period of the advertisement can have a large deviation, thereby affecting the accuracy of the overall PCVR of the advertisement. How to locate the time period in which the PCVR of the advertisement has a large deviation and how to calibrate the PCVR of the advertisement in the time period in a targeted manner are also problems that need to be considered. SUMMARY

[0004] To solve the above problems, the present disclosure provides a method and apparatus for calibrating predicted conversion rate of an advertisement, a computer readable storage medium, and a computer program product.

[0005] According to an aspect of an embodiment of the present disclosure, a method for calibrating predicted conversion rate of an advertisement is provided, including: determining a calibration time period for calibrating predicted conversion rate of a target advertisement; determining a plurality of advertisement sets associated with the target advertisement based on at least part of a plurality of advertisement attributes of the target advertisement, wherein the advertisements in each of the plurality of advertisement sets have at least one common attribute, and at least part of the at least one common attribute is included in the plurality of advertisement attributes; obtaining predicted conversion rate and actual conversion rate of each of the plurality of advertisement sets; determining a reference advertisement set used as a calibration reference from the plurality of advertisement sets, and determining a calibration parameter using the predicted conversion rate and the actual conversion rate of the reference advertisement set; and calibrating the predicted conversion rate of the target advertisement in the calibration time period using the calibration parameter.

[0006] According to an example of the embodiments of the present disclosure, determining the reference ad set used as the calibration reference from the plurality of ad sets comprises: determining the reference ad set based on at least a data validity of each of the plurality of ad sets, wherein the data validity indicates whether a total consumption of ads of the ad set is greater than a predetermined threshold.

[0007] According to an example of the embodiments of the present disclosure, determining the reference ad set used as the calibration reference from the plurality of ad sets further comprises: determining the ad set with the highest priority and data validity from the plurality of ad sets as the reference ad set.

[0008] According to an example of the embodiments of the present disclosure, obtaining the estimated conversion rate and the actual conversion rate of each of the plurality of ad sets comprises: for each of the plurality of ad sets: obtaining the estimated conversion rate and the actual conversion rate of each of the ads in the ad set in each of a plurality of time periods; and calculating a total estimated conversion rate and a total actual conversion rate of the ads in the ad set in the plurality of time periods by cumulatively summing the estimated conversion rate and the actual conversion rate of each of the ads, as the estimated conversion rate and the actual conversion rate of the ad set.

[0009] According to an example of the embodiments of the present disclosure, determining the calibration parameter using the estimated conversion rate and the actual conversion rate of the reference ad set comprises: determining a quotient of the actual conversion rate and the estimated conversion rate of the reference ad set as the calibration parameter.

[0010] According to an example of the embodiments of the present disclosure, the plurality of ad attributes comprises at least one of: an advertiser, an ad product, an ad product brand, an ad targeting, and an ad region.

[0011] According to an example of the embodiments of the present disclosure, determining the calibration time period for calibrating the estimated conversion rate of the target ad comprises: obtaining a distribution of estimated conversion rate deviations of sample ads in a sample ad set in a plurality of time periods; and determining the calibration time period for calibrating the estimated conversion rate of the target ad based on the distribution, wherein the sample ads in the sample ad set have at least one common attribute with the target ad, and the sample ad set can be different from or the same as the ad set in the plurality of ad sets.

[0012] According to an example of the embodiments of the present disclosure, wherein the obtaining the distribution of the estimated conversion rate bias of the sample ads in the sample ad set over the plurality of time periods comprises: calculating a statistical mean of the estimated conversion rate bias of the sample ads in the sample ad set in each time period of the plurality of time periods, and wherein the determining the calibration time period for calibrating the estimated conversion rate of the target ad based on the distribution comprises: determining the time period with the largest statistical mean of the estimated conversion rate bias of the sample ad set as the calibration time period.

[0013] According to an example of the embodiments of the present disclosure, wherein the calculating the statistical mean of the estimated conversion rate bias of the sample ads in the sample ad set in each time period of the plurality of time periods comprises: for each time period of the plurality of time periods: obtaining the actual conversion rate and the estimated conversion rate of each sample ad in the sample ad set in the time period; calculating the estimated conversion rate bias of each sample ad in the time period based on the actual conversion rate and the estimated conversion rate; and calculating the statistical mean of the estimated conversion rate bias of the sample ads in the sample ad set in the time period.

[0014] According to an example of the embodiments of the present disclosure, wherein the calculating the statistical mean of the estimated conversion rate bias of the sample ads in the sample ad set in the time period comprises: obtaining the ad consumption of each sample ad in the sample ad set in the time period; determining the weight of each sample ad in the sample ad set based on the ad consumption; and calculating the statistical mean in the time period by multiplying the absolute value of the estimated conversion rate bias of each sample ad in the sample ad set by its corresponding weight and accumulating the products.

[0015] According to an example of the embodiments of the present disclosure, wherein the obtaining the distribution of the estimated conversion rate bias of the sample ads in the sample ad set over the plurality of time periods comprises: calculating the distribution proportion of the estimated conversion rate bias of the sample ads in the sample ad set in each time period of the plurality of time periods in different evaluation intervals, the evaluation intervals comprising at least one overestimation interval and at least one underestimation interval, and wherein the determining the calibration time period for calibrating the estimated conversion rate of the target ad based on the distribution comprises: determining the time period with the largest proportion of the estimated conversion rate bias of the sample ads falling into the overestimation interval or the time period with the largest proportion of the estimated conversion rate bias of the sample ads falling into the underestimation interval as the calibration time period.

[0016] According to another aspect of the embodiments of the present disclosure, there is provided an advertisement ranking method, comprising: obtaining a calibrated estimated conversion rate of each of a plurality of to-be-launched advertisements; calculating an estimated revenue of each of the plurality of to-be-launched advertisements according to the calibrated estimated conversion rate of each of the plurality of to-be-launched advertisements; and ranking the plurality of to-be-launched advertisements based on the estimated revenue of each of the plurality of to-be-launched advertisements, and launching the plurality of to-be-launched advertisements in sequence according to the ranking result, wherein the obtaining of the calibrated estimated conversion rate of each of the plurality of to-be-launched advertisements comprises: calibrating the estimated conversion rate of each of the plurality of to-be-launched advertisements by using the calibration method as described in the above aspect, to obtain the calibrated estimated conversion rate of each of the plurality of to-be-launched advertisements.

[0017] According to another aspect of the embodiments of the present disclosure, there is provided an apparatus for calibrating an estimated conversion rate of an advertisement, comprising: a first determining unit configured to determine a calibration time period for calibrating an estimated conversion rate of a target advertisement, and determine a plurality of advertisement sets associated with the target advertisement based on at least part of a plurality of advertisement attributes of the target advertisement, wherein the advertisements in each of the plurality of advertisement sets have at least one common attribute, and at least part of the at least one common attribute is included in the plurality of advertisement attributes; an obtaining unit configured to obtain an estimated conversion rate and an actual conversion rate of each of the plurality of advertisement sets; a second determining unit configured to determine a reference advertisement set used as a calibration reference from the plurality of advertisement sets, and determine a calibration parameter by using the estimated conversion rate and the actual conversion rate of the reference advertisement set; and a calibration unit configured to calibrate the estimated conversion rate of the target advertisement in the calibration time period by using the calibration parameter.

[0018] According to one example of the embodiments of the present disclosure, the second determining unit is further configured to determine the reference advertisement set based on at least a data validity of each of the plurality of advertisement sets, wherein the data validity indicates whether a total consumption of the advertisements in the advertisement set is greater than a predetermined threshold.

[0019] According to one example of the embodiments of the present disclosure, the second determining unit is further configured to determine, as the reference advertisement set, an advertisement set that is data valid and has a highest priority among the plurality of advertisement sets.

[0020] According to an example of the embodiments of the present disclosure, the obtaining unit is further configured to: for each of the plurality of advertisement sets, obtain a predicted conversion rate and an actual conversion rate of each advertisement in the advertisement set in each time period of a plurality of time periods; and calculate a total predicted conversion rate and a total actual conversion rate of the advertisements in the advertisement set in the plurality of time periods by accumulating and summing the predicted conversion rate and the actual conversion rate of each advertisement, as the predicted conversion rate and the actual conversion rate of the advertisement set.

[0021] According to an example of the embodiments of the present disclosure, the second determining unit is further configured to: determine a quotient of the actual conversion rate and the predicted conversion rate of the reference advertisement set as the calibration parameter.

[0022] According to an example of the embodiments of the present disclosure, the plurality of advertisement attributes include at least one of the following: an advertiser, an advertisement commodity, an advertisement commodity brand, an advertisement orientation, and an advertisement region.

[0023] According to an example of the embodiments of the present disclosure, the first determining unit is further configured to: obtain a distribution of the predicted conversion rate deviation of the sample advertisements in the sample advertisement set in the plurality of time periods; and determine a calibration time period for calibrating the predicted conversion rate of the target advertisement based on the distribution, wherein the sample advertisements in the sample advertisement set have at least one common attribute with the target advertisement, and the sample advertisement set can be different from or the same as the advertisement set in the plurality of advertisement sets.

[0024] According to an example of the embodiments of the present disclosure, the first determining unit is further configured to: calculate a statistical mean of the predicted conversion rate deviation of the sample advertisements in the sample advertisement set in each time period of the plurality of time periods, and determine a time period with a maximum statistical mean of the predicted conversion rate deviation of the sample advertisement set as the calibration time period.

[0025] According to an example of the embodiments of the present disclosure, the first determining unit is further configured to: for each time period of the plurality of time periods, obtain an actual conversion rate and a predicted conversion rate of each sample advertisement in the sample advertisement set in the time period; calculate a predicted conversion rate deviation of each sample advertisement in the time period based on the actual conversion rate and the predicted conversion rate; and calculate a statistical mean of the predicted conversion rate deviation of the sample advertisements in the sample advertisement set in the time period.

[0026] According to an example of an embodiment of the present disclosure, the first determining unit is further configured to: obtain the advertisement consumption of each sample advertisement in the sample advertisement set in the time period; determine the weight of each sample advertisement in the sample advertisement set based on the advertisement consumption; and calculate the statistical mean in the time period by multiplying the absolute value of the estimated conversion rate bias of each sample advertisement in the sample advertisement set by its corresponding weight, and accumulating and summing the products.

[0027] According to an example of an embodiment of the present disclosure, the first determining unit is further configured to: calculate the distribution proportion of the estimated conversion rate bias of the sample advertisement in the sample advertisement set in each time period of the plurality of time periods in different evaluation intervals, the evaluation intervals including at least one overestimation interval and at least one underestimation interval, and determine the time period in which the proportion of the estimated conversion rate bias of the sample advertisement falling into the overestimation interval is the largest or the time period in which the proportion of the estimated conversion rate bias of the sample advertisement falling into the underestimation interval is the largest as the calibration time period.

[0028] According to another aspect of an embodiment of the present disclosure, an advertisement ranking device is provided, comprising: an obtaining unit configured to obtain the calibrated estimated conversion rate of each of a plurality of to-be-launched advertisements; a revenue estimation unit configured to calculate the estimated revenue of each of the plurality of to-be-launched advertisements according to the calibrated estimated conversion rate of each of the plurality of to-be-launched advertisements; and a ranking unit configured to rank the plurality of to-be-launched advertisements based on the estimated revenue of each of the plurality of to-be-launched advertisements, and launch the plurality of to-be-launched advertisements in turn according to the ranking result, wherein the obtaining unit is configured to calibrate the estimated conversion rate of each of the plurality of to-be-launched advertisements by using the calibration method as described in the above aspect, to obtain the calibrated estimated conversion rate of each of the plurality of to-be-launched advertisements.

[0029] According to another aspect of an embodiment of the present disclosure, a calibration device for advertisement estimated conversion rate is provided, comprising: one or more processors; and one or more memories, wherein the memories store computer readable codes which, when executed by the one or more processors, cause the one or more processors to perform the method as described in any one of the above aspects of the present disclosure.

[0030] According to another aspect of an embodiment of the present disclosure, a computer readable storage medium is provided, which stores computer readable instructions, the computer readable instructions, when executed by a processor, cause the processor to perform the method as described in any one of the above aspects of the present disclosure.

[0031] According to another aspect of the embodiments of the present disclosure, there is provided a computer program product comprising computer readable instructions which, when executed by a processor, cause the processor to perform the method according to any one of the aspects of the present disclosure.

[0032] By using the calibration method and the advertisement ranking method for estimated conversion rate of advertisements according to the aspects of the present disclosure, the specific time period for calibrating the estimated conversion rate of the target advertisement can be determined and analyzed, and the estimated conversion rate of the target advertisement can be calibrated by using the set of reference advertisements associated with the target advertisement for the specific time period, so that a more accurate estimated conversion rate of the target advertisement can be obtained; the calibration can be performed by analyzing the characteristics of the estimated conversion rate of advertisements of a specific industry, a specific audience, a specific content, etc., so that the advertisements can be more accurately put, the cost of putting the advertisements can be controlled, the effect of putting the advertisements can be predicted, etc.; and the estimated conversion rate of the advertisements can be calibrated to estimate the income of putting the advertisements, so that the advertisements with higher estimated income can be put preferentially to maximize the income of the advertisement platform. BRIEF DESCRIPTION OF DRAWINGS

[0033] The above and other objects, features and advantages of the embodiments of the present disclosure will be more apparent from the following detailed description taken in conjunction with the accompanying drawings, in which like reference characters refer to the like elements or steps throughout. The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure. In the drawings:

[0034] Figure 1 A flowchart of a calibration method for estimated conversion rate of advertisements according to an embodiment of the present disclosure is shown;

[0035] Figure 2 A flowchart of a calibration method for estimated conversion rate of advertisements according to an embodiment of the present disclosure is shown;

[0036] Figure 3 A flowchart of calculating a statistical mean of estimated conversion rate bias according to an embodiment of the present disclosure is shown;

[0037] Figure 4 A statistical mean of estimated conversion rate bias of a set of sample advertisements under an advertisement dimension according to an embodiment of the present disclosure is shown;

[0038] Figure 5 A distribution ratio of estimated conversion rate bias of a set of sample advertisements under an advertisement dimension according to an embodiment of the present disclosure is shown;

[0039] Figure 6 a statistical mean of the estimated conversion rate bias of the sample ad set under the advertiser dimension according to an example of an embodiment of the present disclosure is shown;

[0040] Figure 7 a distribution proportion of the estimated conversion rate bias of the sample ad set under the advertiser dimension according to an example of an embodiment of the present disclosure is shown;

[0041] Figure 8 a statistical mean of the calibrated estimated conversion rate bias of the sample ad set under the ad dimension according to an example of an embodiment of the present disclosure is shown;

[0042] Figure 9 a distribution proportion of the calibrated estimated conversion rate bias of the sample ad set under the ad dimension according to an example of an embodiment of the present disclosure is shown;

[0043] Figure 10 a statistical mean of the calibrated estimated conversion rate bias of the sample ad set under the advertiser dimension according to an example of an embodiment of the present disclosure is shown;

[0044] Figure 11 a distribution proportion of the calibrated estimated conversion rate bias of the sample ad set under the advertiser dimension according to an example of an embodiment of the present disclosure is shown;

[0045] Figure 12 a flowchart of an ad ranking method according to an embodiment of the present disclosure is shown;

[0046] Figure 13 a schematic diagram of an application scenario of ad delivery according to an embodiment of the present disclosure is shown;

[0047] Figure 14 a structural schematic diagram of an ad estimated conversion rate calibration apparatus according to an embodiment of the present disclosure is shown;

[0048] Figure 15 a structural schematic diagram of an ad ranking apparatus according to an embodiment of the present disclosure is shown;

[0049] Figure 16 a schematic diagram of an architecture of an exemplary computing device according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0050] The technical solutions in the embodiments of the present disclosure will be described clearly and completely below with reference to the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those of ordinary skill in the art without any creative effort belong to the scope of protection of the present disclosure.

[0051] In the present disclosure, an advertiser generally refers to a party that funds the placement of an advertisement, an advertising platform refers to a party that helps the advertiser to place, analyze, predict, etc. advertisements using its own platform or technology, and an audience refers to a user group that watches the advertisements. After an advertisement is placed through an advertising platform, the following processes usually occur: first, a user sees the advertisement, which is referred to as exposure; after exposure, a user who is interested in the advertisement may click to browse the goods contained in the advertisement, for example, by clicking a product link in the advertisement to jump to a product page for browsing, which is referred to as a click; after jumping to the product page through the click, the user may purchase the product on the product page or download and install an application on the product page, etc., which is referred to as conversion. In the present disclosure, the ratio of the number of clicks to the number of exposures of an advertisement can be referred to as the click-through rate (CTR) of the advertisement, and the ratio of the number of conversions to the number of clicks of the advertisement can be referred to as the conversion rate (CVR) of the advertisement.

[0052] To achieve the accurate placement of advertisements and evaluate the placement effect of advertisements, a model is usually used to estimate the conversion rate after the placement of an advertisement to obtain the estimated conversion rate (PCVR) of the advertisement. With the development of deep learning, current models for estimating the conversion rate of an advertisement are mostly based on neural networks, such as a Deep Crossing model, a product-based neural network (PNN), a factorization machine (FM), etc. However, the estimated conversion rate of an advertisement obtained by using these models has good overall performance, but the accuracy for specific industries, specific audiences, specific content, etc. is not high, and therefore, the estimated conversion rate of an advertisement needs to be calibrated accordingly.

[0053] In addition, for a single advertisement, the estimated conversion rate of the advertisement in certain time periods in the entire placement period of the advertisement can have a large deviation, thereby affecting the accuracy of the overall estimated conversion rate of the advertisement. For example, for a certain advertisement, the exposure of the advertisement in the initial placement period (e.g., from 0 o'clock to 6 o'clock in the morning) is small during the entire day of placement exposure, so that the estimated conversion rate in this period has a large deviation; as time progresses, the exposure of the advertisement can gradually increase, and the accuracy of the estimated conversion rate also increases. If the specific time period in which the estimated conversion of the advertisement has a large deviation can be located, and the estimated conversion rate is calibrated accordingly in this time period, a more accurate estimated conversion rate can be obtained.

[0054] The present disclosure provides a method for calibrating the estimated conversion rate of an advertisement, which can analyze and determine a specific time period for calibrating the estimated conversion rate of a target advertisement, and calibrate the estimated conversion rate of the target advertisement in combination with multiple attributes of the target advertisement for the specific time period, thereby obtaining a more accurate estimated conversion rate of the target advertisement. Hereinafter, the method for calibrating the estimated conversion rate of an advertisement will be described in detail with reference to the accompanying drawings. Figure 1A calibration method of an advertisement estimated conversion rate according to an embodiment of the present disclosure is described.

[0055] Figure 1 A flowchart of the calibration method 100 of the advertisement estimated conversion rate according to an embodiment of the present disclosure is shown. As shown in the figure, Figure 1 a calibration time period for calibrating the estimated conversion rate of the target advertisement is determined in step S110. Here, the target advertisement can be any advertisement whose estimated conversion rate needs to be calibrated, and embodiments of the present disclosure do not make specific limitations thereto. The calibration time period can be a time period determined based on a predetermined rule, for example, a time period in which the estimated conversion rate of the target advertisement deviates greatly, as will be further described below; or can also be any time period specified based on actual needs, and embodiments of the present disclosure do not make specific limitations thereto.

[0056] After the calibration time period is determined, a reference advertisement set used as a calibration reference for the estimated conversion rate of the target advertisement needs to be obtained, so as to calibrate the estimated conversion rate of the target advertisement by using the estimated conversion rate and the actual conversion rate of the reference advertisement set.

[0057] Specifically, first, in step S120, a plurality of advertisement sets associated with the target advertisement are determined based on at least part of a plurality of advertisement attributes of the target advertisement. Among them, the advertisement attributes of the target advertisement can include the advertiser, the advertisement commodity, the advertisement commodity brand, the advertisement targeting, the advertisement region, and the like. The advertisements in each advertisement set associated with the target advertisement have at least one common attribute, for example, belong to the same advertiser, contain the same advertisement commodity, have the same advertisement commodity brand, and the like. And at least part of the at least one common attribute of each advertisement set associated with the target advertisement is included in the plurality of advertisement attributes of the target advertisement. For example, a certain advertisement set associated with the target advertisement can have one common attribute, and the common attribute can be the same as a certain advertisement attribute of the target advertisement. For another example, another advertisement set associated with the target advertisement can have two common attributes, and one of the two common attributes can be the same as a certain advertisement attribute of the target advertisement.

[0058] The following is described in conjunction with specific examples. For example, a target advertisement can have three advertisement attributes of an advertiser A, an advertised product B, and an advertised product brand C, based on which three advertisement sets associated with the target advertisement can be determined. Among them, all advertisements in the first advertisement set belong to the advertiser A, i.e., have a common attribute A, all advertisements in the second advertisement set include the advertised product B, i.e., have a common attribute B, and all products in the advertisements in the third advertisement set belong to the brand C, i.e., have a common attribute C; or, the advertisements in the first advertisement set have the common attribute A, the advertisements in the second advertisement set have the common attribute B and another common attribute D (e.g., all have the same advertisement targeting D), the advertisements in the third advertisement set have the common attribute C and another common attribute E (e.g., all have the same advertisement region E), and the like. It should be noted that the target advertisement has three advertisement attributes and the associated first, second, and third advertisement sets are only examples, and in the embodiments of the present disclosure, the target advertisement can have more or fewer advertisement attributes, and a greater or smaller number of advertisement sets associated with the target advertisement can be determined based thereon, which is not specifically limited in the embodiments of the present disclosure.

[0059] Subsequently, in step S130, the estimated conversion rate and the actual conversion rate of each of the plurality of advertisement sets associated with the target advertisement are obtained. For example, the estimated conversion rate and the actual conversion rate of each advertisement set can be directly obtained from an advertisement database pre-statistically stored on a server. The advertisement database may, for example, include the estimated conversion rates and the actual conversion rates of different advertisement sets obtained according to different aggregation dimensions, such as the total estimated conversion rate and the total actual conversion rate of all advertisements under different advertisers, the total estimated conversion rate and the total actual conversion rate of all advertisements under different advertised products, the total estimated conversion rate and the total actual conversion rate of all advertisements under different advertised product brands, and the like. Alternatively, the estimated conversion rate and the actual conversion rate of each advertisement set can also be statistically obtained, as will be further described below.

[0060] Specifically, for each advertisement set, first, the estimated conversion rate and the actual conversion rate of each advertisement in the advertisement set in each of a plurality of time periods are obtained. Here, the plurality of time periods may, for example, be different time periods in a day determined in advance, the estimated conversion rate of each advertisement may, for example, be obtained by a conversion rate estimation model such as PNN, FM, etc. as described above, and the actual conversion rate may, for example, come from the statistical data of the advertiser. After obtaining the estimated conversion rate and the actual conversion rate of all advertisements in the advertisement set, the estimated conversion rate and the actual conversion rate of each advertisement in the plurality of time periods are cumulatively summed to calculate the total estimated conversion rate and the total actual conversion rate of all advertisements in the advertisement set in the plurality of time periods, and the calculated total estimated conversion rate and the total actual conversion rate are taken as the estimated conversion rate and the actual conversion rate of the advertisement set, respectively.

[0061] For example, in the above example, the target advertisement has three advertisement attributes of advertiser A, advertised commodity B and advertised commodity brand C, and the advertisements in the first advertisement set associated with the target advertisement have the common attribute A, the advertisements in the second advertisement set have the common attribute B, and the advertisements in the third advertisement set have the common attribute C. Then, the total estimated conversion rate and the total actual conversion rate of all the advertisements in the first advertisement set can be counted as the estimated conversion rate PCVR-A and the actual conversion rate CVR-A of the first advertisement set, respectively, and the estimated conversion rate PCVR-B and the actual conversion rate CVR-B of the second advertisement set and the estimated conversion rate PCVR-C and the actual conversion rate CVR-C of the third advertisement set can be similarly obtained.

[0062] In step S140, a reference advertisement set used as a calibration reference is determined from the above-mentioned multiple advertisement sets associated with the target advertisement, and the calibration parameters are determined by using the estimated conversion rate and the actual conversion rate of the reference advertisement set.

[0063] According to one example of an embodiment of the present disclosure, the reference advertisement set can be determined based on the data validity of each of the multiple advertisement sets, and one or more advertisement sets with valid data are determined as the reference advertisement set. Here, the data validity of an advertisement set can indicate whether the data such as the estimated conversion rate and the reference conversion rate of the advertisement set has reference value.

[0064] For example, the data validity of an advertisement set can be determined according to the total consumption of the advertisements in the advertisement set. Here, the consumption of an advertisement refers to the fee to be charged to the advertiser for a certain amount or a certain time of advertisement placement. The consumption of an advertisement can reflect the click volume or the conversion volume of the advertisement, and can reflect whether the advertisement has been exposed enough, so as to be able to be used to determine the data validity of the advertisement set. According to an example of an embodiment of the present disclosure, when the total consumption of the advertisements in the advertisement set is greater than a predetermined threshold, it can be determined that the data of the advertisement set is valid; otherwise, it is determined that the data of the advertisement set is invalid. The predetermined threshold can be set according to the actual situation, for example, the predetermined threshold can be set according to the target conversion cost of the advertiser. The target conversion cost of the advertiser is the fee required by the advertiser for each conversion of a single advertisement. For example, when the total consumption of the advertisements in the advertisement set is greater than 4 times the target conversion cost, it can be determined that the data of the advertisement set is valid; otherwise, it is determined that the data of the advertisement set is invalid.

[0065] It should be noted that although the above describes an example of determining the data validity of an advertisement set according to the total consumption of the advertisements, the embodiments of the present disclosure are not limited thereto, and the data validity of an advertisement set can also be determined according to, for example, the estimated conversion rate, the actual conversion rate, the click volume, the click rate, etc.

[0066] According to another example of the embodiments of the present disclosure, the reference ad set can be determined based on the priority of each of the plurality of ad sets, and the ad set with the highest priority can be determined as the reference ad set. The priority of the ad set may, for example, be determined in advance according to the relevance of the ad set to the target ad, or according to actual needs. For example, in the above example in which the first, second, and third ad sets associated with the target ad have the common attributes A, B, and C respectively, if it is determined that the ad in the second ad set with the same attribute B as the target ad is most relevant to the target ad, the second ad set can be determined to have the highest priority, and be determined as the reference ad set.

[0067] According to another example of the embodiments of the present disclosure, the reference ad set can be determined as the ad set with valid data and the highest priority among the plurality of ad sets associated with the target ad. For example, in the above example, the data of the first and third ad sets is determined to be valid according to the total ad consumption of the respective ad sets, but the third ad set has the highest priority, and thus the third ad set can be determined as the reference ad set.

[0068] After the reference ad set used as the calibration reference for the target ad is determined using the method in the above example, the calibration parameter can be determined using the estimated conversion rate and the actual conversion rate of the reference ad set. Since the ads in the reference ad set have at least one attribute in common with the target ad, the estimated conversion rate and / or the actual conversion rate of the ads in the reference ad set and the target ad have similar characteristics, and thus the estimated conversion rate and the actual conversion rate of the reference ad set can be used as the calibration reference for the target ad. According to an example of the embodiments of the present disclosure, the quotient of the actual conversion rate and the estimated conversion rate of the reference ad set can be determined as the calibration parameter. For example, if the second ad set with the common attribute B as the target ad is determined as the reference ad set in the above step, the calibration parameter a = CVR-B / PCVR-B can be determined based on the estimated conversion rate PCVR-B and the actual conversion rate CVR-B of the second ad set. Alternatively, the product of the quotient of the actual conversion rate and the estimated conversion rate of the reference ad set and a predetermined weight factor can also be used as the calibration parameter. Alternatively, the calibration parameter can also be determined by more complex processing of the estimated conversion rate and the actual conversion rate of the reference ad set, and the embodiments of the present disclosure do not make specific limitations in this regard.

[0069] Next, in step S150, the predicted conversion rate of the target advertisement is calibrated within the calibration time period determined in step S110 using the calibration parameter described above. For example, the predicted conversion rate can be calibrated by multiplying the predicted conversion rate obtained by the conversion rate prediction model such as PNN, FM, etc. by the calibration parameter; or the calibration parameter can be used as a weighting factor of the conversion rate prediction model such as PNN, FM, etc. to directly generate the calibrated predicted conversion rate.

[0070] It should be noted that although in the foregoing, the predicted conversion rate of the target advertisement is calibrated by using the data of the reference advertisement set associated with the target advertisement, the embodiments of the present disclosure are not limited thereto. In the case where the target advertisement has been launched for a sufficient time to generate sufficient data, the target advertisement itself can also be calibrated using the historical data of the same launch period, i.e., the predicted conversion rate and the actual conversion rate of the target advertisement in an earlier time period of the same launch period are used to calibrate the predicted conversion rate of the target advertisement in a later time period of the same launch period. Alternatively, the target advertisement can also be calibrated in combination with the data of the reference advertisement set associated with the target advertisement and the historical data of the target advertisement itself. In these cases, the steps of obtaining the calibration parameter and calibrating the target advertisement are similar to the steps described in detail above, and thus repeated description is omitted here.

[0071] The following refers to Figure 2 The step of determining the calibration time period for calibrating the predicted conversion rate of the target advertisement is described in further detail. Figure 2 A flowchart of an advertisement predicted conversion rate calibration method 200 according to an example of the embodiments of the present disclosure is shown. Since the details of steps S220-S250 of the calibration method 200 shown are the same as the details of steps S120-S150 of the calibration method 100 described in detail above with reference to Figure 2 the details of steps S220-S250 of the calibration method 200 shown are the same as the details of steps S120-S150 of the calibration method 100 described in detail above with reference to Figure 1 the details of steps S220-S250 of the calibration method 200 shown are the same as the details of steps S120-S150 of the calibration method 100 described in detail above with reference to

[0072] As Figure 2As shown, the calibration time period for the target advertisement can be determined based on a distribution of the estimated conversion rates of the sample advertisements in the sample advertisement set. Wherein, the sample advertisement set is a set of sample advertisements that have at least one common attribute with the target advertisement, for example, all sample advertisements in the sample advertisement set are targeted to the same advertisement, for example, are all targeted to an online shopping platform, so that the estimated conversion rate bias of the sample advertisements and the target advertisement has similar time distribution characteristics. And the actual conversion rate and the estimated conversion rate of the sample advertisement are known, for example, have been stored as historical data in the database. Therefore, by analyzing the distribution characteristics of the estimated conversion rate bias of the sample advertisements in the sample advertisement set, the calibration time period for calibrating the estimated conversion rate of the target advertisement can be determined. Here, the sample advertisement set can be different from or the same as the advertisement set in the plurality of advertisement sets described above, and the present disclosure does not make specific limitations thereto.

[0073] Specifically, as shown in step S211, the distribution of the estimated conversion rate bias of the sample advertisements in the sample advertisement set in a plurality of time periods can be obtained, and in step S212, the calibration time period for calibrating the estimated conversion rate of the target advertisement is determined based on the distribution. As described above, the plurality of time periods may, for example, be different time periods in a day determined in advance, and the estimated conversion rate bias represents the bias between the estimated conversion rate and the actual conversion rate. Figure 2

[0074] According to one example of an embodiment of the present disclosure, obtaining the distribution of the estimated conversion rate bias of the sample advertisements in the sample advertisement set in a plurality of time periods can include calculating the statistical mean of the estimated conversion rate bias of the sample advertisements in the sample advertisement set in each time period of the plurality of time periods. Specifically, for each time period in the plurality of time periods, step S211 can further include steps S2111, S2112 and S2113, as shown in Figure 3 Figure 3 A flowchart of calculating the statistical mean of the estimated conversion rate bias according to an example of an embodiment of the present disclosure is shown.

[0075] First, in step S2111, the actual conversion rate and the estimated conversion rate of each sample advertisement in the sample advertisement set in the time period are obtained. As described previously, for example, the actual conversion rate and the estimated conversion rate of the sample advertisement can be obtained from the historical data stored in the database in advance.

[0076] ​​Subsequently, in step S2112, the estimated conversion rate bias of the sample ad is calculated based on the actual conversion rate and the estimated conversion rate of each ad. For example, the quotient of the estimated conversion rate and the actual conversion rate (which can be denoted as BIAS) can be taken as the estimated conversion rate bias of the sample ad. Alternatively, the absolute value of the quotient of the estimated conversion rate and the actual conversion rate minus 1 (which can be denoted as |BIAS-1|) can be taken as the estimated conversion rate bias of the sample ad. Or, the estimated conversion rate bias can be determined by further processing the estimated conversion rate and the actual conversion rate.

[0077] After obtaining the estimated conversion rate bias of each sample ad in the time period, in step S2113, the statistical mean of the estimated conversion rate biases of all sample ads in the sample ad set in the time period can be calculated. Here, the statistical mean can be, for example, the average value, the mean square value, the root mean square value, the weighted value, etc. of the estimated conversion rate biases of all sample ads. Taking the weighted value as an example, the statistical mean of the estimated conversion rate biases of all sample ads in the sample ad set can be calculated by multiplying the estimated conversion rate bias of each sample ad by its corresponding weight and accumulating the products.

[0078] For example, the weight of each sample ad can be determined based on the ad consumption of the sample ad. Specifically, first, the ad consumption of each sample ad in the sample ad set in the current time period is obtained. As mentioned earlier, the ad consumption can reflect the number of clicks or the number of conversions of the ad, so the sample ad with larger ad consumption has a relatively larger contribution to the statistical mean of the estimated conversion rate bias. Therefore, the sample ad with larger ad consumption can be assigned a larger weight. For example, the normalized ad consumption of each sample ad can be taken as its corresponding weight by normalizing the ad consumption of all sample ads. It should be understood that the embodiments of the present disclosure are not limited to determining the weight of each sample ad according to the ad consumption, but any other appropriate method can be used to determine the weight.

[0079] In this example, after obtaining the statistical mean of the estimated conversion rate biases of the sample ad set, step S212 can further include determining a calibration time period for calibrating the estimated conversion rate of the target ad based on the statistical mean of the estimated conversion rate biases of the sample ad set. For example, the time period with the largest statistical mean of the estimated conversion rate biases of the sample ad set can be determined as the calibration time period.

[0080] According to another example of the embodiments of the present disclosure, obtaining the distribution of the estimated conversion rate bias of the sample ads in the sample ad set over the plurality of time periods can include calculating the distribution proportion of the estimated conversion rate bias of the sample ads in the sample ad set in each time period of the plurality of time periods in different evaluation intervals, and the like. The evaluation interval can indicate the accuracy of the estimated conversion rate of the sample ad. For example, the evaluation interval can include at least one overestimation interval and at least one underestimation interval. If the estimated conversion rate bias of the sample ad in a certain time period falls into the overestimation interval, it means that the conversion rate is overestimated. If it falls into the underestimation interval, it means that the conversion rate is underestimated. The method for obtaining the estimated conversion rate bias of each ad is the same as the method described above with reference to step S202, and will not be described here in detail. Figure 2 The details of the steps described above are the same, and will not be described here in detail.

[0081] Specifically, for each time period of the plurality of time periods, the distribution proportion of the estimated conversion rate bias of each sample ad in the sample ad set in the time period in different evaluation intervals can be calculated, for example, the proportion of the number of sample ads whose estimated conversion rate bias falls into each of the at least one overestimation interval and the at least one underestimation interval to the total number of sample ads in the sample ad set. In this example, after obtaining the distribution proportion of the estimated conversion rate bias of the sample ad set, step S212 can further include determining a calibration time period for calibrating the estimated conversion rate of the target ad based on the distribution proportion of the estimated conversion rate bias of the sample ad set. For example, the time period in which the proportion of the estimated conversion rate bias of the sample ad falling into the overestimation interval is the largest can be determined as the calibration time period; alternatively, the time period in which the proportion of the estimated conversion rate bias of the sample ad falling into the underestimation interval is the largest can be determined as the calibration time period, and the like.

[0082] By the above method of determining the calibration time period, the calibration time period for calibrating the estimated conversion rate of the target ad can be determined by analyzing the distribution characteristics of the estimated conversion rate of the sample ad set, so as to calibrate the estimated conversion rate of the target ad in the calibration time period, thereby achieving the most ideal calibration effect.

[0083] The calibration methods 100 and 200 according to the embodiments of the present disclosure will be further described in detail below with reference to specific examples.

[0084] In this example, for the target advertisement to be calibrated, the calibration time period is first determined by analyzing the distribution characteristics of the estimated conversion rates of the sample advertisement set having the same orientation as the target advertisement. For example, the target advertisement and the sample advertisements in the sample advertisement set are both placed on an online shopping platform, and the sample advertisements in the sample advertisement set have accumulated rich historical placement data. That is, in this example, the sample advertisement set is selected as all advertisements having the same orientation as the target advertisement, and at this time the sample advertisement set can be referred to as a sample advertisement set under the advertisement dimension. In addition, since in actual applications, advertisers often expect to obtain comprehensive analysis results of all advertisements placed by them, therefore, all advertisements of the same advertiser can also be comprehensively analyzed, and all advertisements of the same advertiser are regarded as one advertisement, that is, the results of these advertisements are counted as one value, and in this analysis scenario, the sample advertisement set can be referred to as a sample advertisement set under the advertiser dimension. Next, the sample advertisement set under the advertisement dimension and the sample advertisement set under the advertiser dimension will be discussed respectively.

[0085] In addition, in this example, the plurality of time periods can be 4 time periods obtained by dividing the entire day of advertisement placement, wherein the first time period is 0-6 o'clock, the second time period is 6-12 o'clock, the third time period is 12-18 o'clock, and the fourth time period is 18-24 o'clock; and the value of the estimated conversion rate bias is divided into 8 evaluation intervals, in turn (1.5, ∞), (1.2, 1.5], (1.1, 1.2], (1, 1.1], (0.9, 1], (0.8, 0.9], (0.5, 0.8], (0, 0.5], and the proportion of the number of sample advertisements whose estimated conversion rate bias falls into each interval to the total number of sample advertisements in the sample advertisement set is calculated. Among the 8 evaluation intervals, each interval greater than 1 can be referred to as a high estimation interval, and each interval less than 1 can be referred to as a low estimation interval. If the estimated conversion rate bias of the sample advertisement falls into the high estimation interval, it indicates that the conversion rate of the sample advertisement is overestimated; if the estimated conversion rate bias of the sample advertisement falls into the low estimation interval, it indicates that the conversion rate of the sample advertisement is underestimated.

[0086] For the sample advertisement set under the advertisement dimension, the distribution of the estimated conversion rate bias thereof can be counted by the following steps.

[0087] (1) For each advertisement, the estimated conversion number, the actual conversion number and the advertisement consumption of the advertisement in each hour of a day are obtained respectively;

[0088] (2) The total estimated conversion rate PCVR and the total actual conversion rate CVR of each advertisement in each time period from the first to the fourth time period are counted, and the estimated conversion rate bias BIAS = |PCVR / CVR-1| in the time period is calculated;

[0089] (3) Normalizing the advertisement consumption of each advertisement, and taking the normalized advertisement consumption as the corresponding weight w of each advertisement;

[0090] (4) Multiplying the estimated conversion rate bias BIAS of each advertisement by its corresponding weight w respectively, and accumulating and summing, thereby obtaining the statistical mean E of the estimated conversion rate bias =∑(|PCVR / CVR-1|*w), as shown in the table in Figure 4

[0091] (5) In each time period, the proportion of the number of sample advertisements whose estimated conversion rate bias falls into each of the above-mentioned 8 intervals in the total number of sample advertisements is calculated, to obtain the distribution proportion of the estimated conversion rate bias, as shown in the table in Figure 5

[0092] For the sample advertisement set under the advertiser dimension, the process of calculating the distribution of the estimated conversion rate bias is similar to the above-mentioned steps, and the only difference is that in this case, all advertisements under the same advertiser are regarded as one advertisement, i.e., the results of all advertisements of the same advertiser are counted as one value. According to the above-mentioned steps, the statistical mean and the distribution proportion of the estimated conversion rate bias under the advertiser dimension are shown in the tables in Figure 6 and Figure 7

[0093] According to Figure 4 and Figure 6 , in the first time period (i.e., 0-6 o'clock), the estimated conversion rate bias of the sample advertisement set is the largest, and the estimated conversion rate biases of the second, third and fourth time periods have decreased. According to the analysis of Figure 5 and Figure 7 , whether in the advertisement dimension or in the advertiser dimension, the proportion of the estimated conversion rate bias falling into the overestimation interval (for example, the first interval) decays much less than the proportion of the estimated conversion rate bias falling into the underestimation interval (for example, the eighth interval) as time elapses, i.e., as time goes by, the conversion rate is still mainly overestimated.

[0094] Based on the above analysis, the first time period with the largest statistical mean of the estimated conversion rate bias can be determined as the calibration time period; or, through analysis, it can be known that in the fourth time period, the sum of the proportions of the estimated conversion rate bias falling into the overestimation interval (i.e., the sum of falling into the first interval to the fourth interval) is the largest, so the fourth time period can be determined as the calibration time period; or, through analysis, it can be known that in the first time period, the sum of the proportions of the estimated conversion rate bias falling into the underestimation interval (i.e., the sum of falling into the fifth interval to the eighth interval) is the largest, so the first time period can be determined as the calibration time period, and so on. In this example, the first time period is taken as the calibration time period as an example for illustration.

[0095] ​​​After determining the first time period as the calibration time period, the target advertisement can be calibrated in the first time period. In this example, for the convenience of illustrating the effect of the calibration method according to the embodiments of the present disclosure, each sample advertisement in the sample advertisement set under the advertisement dimension is sequentially taken as the target advertisement, and the estimated conversion rate of each sample advertisement is calibrated in the first time period of the new delivery cycle of the sample advertisement. Since in the first time period, the data generated by the target advertisement itself is not sufficient, the data of the advertisement set associated with the target advertisement can be used to calibrate the estimated conversion rate of the target advertisement.

[0096] In this example, for each target advertisement, a first advertisement set having the same advertiser A as the target advertisement, a second advertisement set having the same advertisement commodity B as the target advertisement, and a third advertisement set having the same advertisement commodity brand C as the target advertisement are obtained respectively, and the total estimated conversion rate and the total actual conversion rate of all advertisements in the first, second and third advertisement sets are respectively counted as the estimated conversion rate and the actual conversion rate of the first, second and third advertisement sets. For example, the estimated conversion rate and the actual conversion rate of the first advertisement set are denoted as PCVR-A and CVR-A respectively; the estimated conversion rate and the actual conversion rate of the second advertisement set are denoted as PCVR-B and CVR-B respectively; and the estimated conversion rate and the actual conversion rate of the third advertisement set are denoted as PCVR-C and CVR-C respectively.

[0097] Then, a reference advertisement set used as the calibration reference of the target advertisement can be determined from the first, second and third advertisement sets. For example, as described above, the data validity in the first, second and third advertisement sets can be judged according to the total consumption of the advertisements in the first, second and third advertisement sets, and one of the advertisement sets with data validity is selected as the reference advertisement set; or the advertisement set with the highest priority in the first, second and third advertisement sets can be selected as the reference set; or the advertisement set with data validity and the highest priority in the first, second and third advertisement sets can be selected as the reference advertisement set. In this example, for each target advertisement, the advertisement set with data validity and the highest priority in the multiple advertisement sets associated with the target advertisement can be determined as the reference advertisement set of the target advertisement.

[0098] In this example, for the current target advertisement, if it is determined that the second advertisement set associated therewith is a reference advertisement set used as a calibration reference, then a calibration parameter can be determined according to the estimated conversion rate PCVR-B and the actual conversion rate CVR-B of the second advertisement set, for example, the calibration parameter a = CVR-B / PCVR-B can be determined. Subsequently, in the first time period of a new delivery cycle (e.g., a new delivery day) of the target advertisement, the estimated conversion rate of the current target advertisement can be calibrated based on the calibration parameter, for example, the estimated conversion rate obtained by a conversion rate estimation model such as PNN, FM, etc. can be multiplied by the calibration parameter to generate a calibrated estimated conversion rate.

[0099] Similarly, each sample advertisement in the sample advertisement set under the advertisement dimension is sequentially taken as a target advertisement, a reference advertisement set of the target advertisement is determined, and a calibration parameter is determined based on the estimated conversion rate and the actual conversion rate of the reference advertisement set. Then, in the first time period of the new delivery cycle of the target advertisement, the estimated conversion rate of the target advertisement is calibrated based on the calibration parameter. After completing the new delivery cycle, the distribution result of the calibrated estimated conversion rate deviation of the sample advertisement set under the advertisement dimension is obtained by using the method as described above, as shown in Figure 8 to Figure 9 , and the distribution result of the calibrated estimated conversion rate deviation of the sample advertisement set under the advertiser dimension can be further obtained, as shown in Figure 10 to Figure 11 . Wherein, Figure 8 shows the statistical mean of the calibrated estimated conversion rate deviation of the sample advertisement set under the advertisement dimension according to an example of an embodiment of the present disclosure; Figure 9 shows the distribution proportion of the calibrated estimated conversion rate deviation of the sample advertisement set under the advertisement dimension according to an example of an embodiment of the present disclosure; Figure 10 shows the statistical mean of the calibrated estimated conversion rate deviation of the sample advertisement set under the advertiser dimension according to an example of an embodiment of the present disclosure; Figure 11 shows the distribution proportion of the calibrated estimated conversion rate deviation of the sample advertisement set under the advertiser dimension according to an example of an embodiment of the present disclosure.

[0100] Comparing Figure 8 to Figure 11 with Figure 4 to Figure 7 It can be seen that after the estimated conversion rates of various sample advertisements are calibrated in the first time period of the new delivery cycle by using the calibration method of the estimated conversion rate of the advertisement according to an embodiment of the present disclosure, the estimated conversion rate deviations in the first to fourth time periods are all effectively improved, and the calibration effect for the first time period is particularly obvious. For example, before calibration, the statistical mean of the estimated conversion rate deviation of the sample advertisement set under the advertisement dimension in the first time period is 0.421252, and after calibration, this value is reduced to 0.313339. In addition, by comparing Figure 9 andFigure 11 As can be seen, after calibration, the proportion of predicted conversion rates falling into the overestimation range was effectively reduced. In other words, by calibrating specifically for the first time period, the possibility of predicted conversion rates being overestimated was effectively suppressed. This is also based on the above... Figure 4 to Figure 7 The distribution characteristics of the predicted conversion rate deviation determine that overestimation of conversion rate remains the main problem over time. Therefore, after calibration, the possibility of overestimation of conversion rate is effectively reduced.

[0101] By utilizing the calibration method for estimated conversion rates of advertisements according to the above embodiments of this disclosure, a specific time period for calibrating the estimated conversion rate of a target advertisement can be analyzed and determined. For this specific time period, a set of reference advertisements associated with the target advertisement is used to calibrate the estimated conversion rate of the target advertisement, thereby obtaining a more accurate estimated conversion rate for the target advertisement. Furthermore, by utilizing the calibration method according to the embodiments of this disclosure, targeted calibration can be performed by analyzing the characteristics of the estimated conversion rates of advertisements for specific industries, specific audiences, specific content, etc., so as to more accurately conduct advertisement placement, control advertisement placement costs, predict placement effects, etc.

[0102] Furthermore, the calibrated estimated conversion rate obtained using the calibration method for estimated conversion rate of advertisements according to the above embodiments of this disclosure can be used to sort the advertisements to be delivered, so as to prioritize the delivery of advertisements with higher estimated revenue, thereby maximizing the revenue of the advertising platform. See below for reference. Figure 12 This disclosure describes an advertisement sorting method according to embodiments of the present disclosure. Figure 12 A flowchart of an advertising sorting method 1200 according to an embodiment of the present disclosure is shown.

[0103] like Figure 12 As shown, in step S1210, the calibrated estimated conversion rate of each of the multiple ads to be delivered is obtained. The multiple ads to be delivered can be any ads to be delivered; this embodiment does not impose specific limitations on this. In this embodiment, for example, the above reference can be used... Figure 1 The described calibration method for advertising conversion rate calibration calibrates the estimated conversion rate of each of a plurality of ads to be delivered, thereby obtaining a calibrated estimated conversion rate for each of the plurality of ads to be delivered. However, the embodiments of this disclosure are not limited to this, and the calibrated estimated conversion rate of each ad to be delivered can also be obtained by other means. The estimated conversion rate of each ad to be delivered can be obtained, for example, through conversion rate prediction models such as PNN and FM as described above, or through any other method; the embodiments of this disclosure do not impose specific limitations on this. Since the above description has already detailed the use of... Figure 1The illustrated calibration method calibrates the predicted conversion rate of the advertisement, and thus, for the sake of simplicity, the same description is omitted here.

[0104] In step S1220, the predicted revenue of each of the plurality of to-be-launched advertisements is calculated according to the calibrated predicted conversion rate of each of the plurality of to-be-launched advertisements. For example, the predicted revenue of a to-be-launched advertisement can be measured by effective Cost Per Mile (eCPM), but the embodiments of the present disclosure are not limited thereto, and can also be measured by any other index such as Return on Investment (ROI). Generally, the eCPM of an advertisement can depend on the predicted conversion rate (PCVR), the predicted click-through rate (PCTR), and the bid of the advertiser. Among them, the bid of the advertiser can refer to the fee paid by the advertiser for one advertisement conversion; the predicted click-through rate refers to the predicted click-through rate of the advertisement after the advertisement is launched. Thus, the eCPM of the advertisement can be expressed as:

[0105] eCPM = PCVR x PCTR x bid

[0106] As can be seen from the above formula, the higher the predicted conversion rate PCVR of the advertisement, the higher the predicted revenue eCPM thereof, thereby bringing higher revenue to the advertisement platform. After the calibrated predicted conversion rate of each to-be-launched advertisement is obtained in step S1210, a more accurate predicted revenue eCPM of each to-be-launched advertisement can be calculated by using the above formula. Among them, the predicted click-through rate PCTR can be obtained by using the prediction method known in the art, and the bid of the advertiser depends on the real-time bid of the advertiser.

[0107] After the predicted revenue such as eCPM of each to-be-launched advertisement is obtained, in step S1230, the plurality of to-be-launched advertisements can be sorted based on the predicted revenue of each to-be-launched advertisement. For example, the to-be-launched advertisement with higher predicted revenue can be arranged in a more forward position, and the to-be-launched advertisement with lower predicted revenue can be arranged in a more backward position. After the sorting is completed, the plurality of to-be-launched advertisements can be launched in sequence in a new launch period according to the sorting result, wherein the new launch period can refer to a new launch time period, a new launch day, and the like. For example, one or more to-be-launched advertisements with the highest ranking can be selected for launching; or, the to-be-launched advertisement with the highest ranking can be selected for launching first, then the to-be-launched advertisement with the second highest ranking can be selected for launching, and so on.

[0108] By using the advertisement ranking method according to the above-mentioned embodiments of the present disclosure, the estimated revenue of the to-be-launched advertisement can be more accurately calculated based on the calibrated estimated conversion rate of the to-be-launched advertisement, the multiple to-be-launched advertisements are ranked based on the estimated revenue, and thus the to-be-launched advertisement with a higher estimated revenue can be preferentially launched, so as to maximize the revenue of the advertisement platform.

[0109] The process of launching an advertisement by using the advertisement estimated conversion rate calibration method 100 and the advertisement ranking method 1200 according to the above-mentioned embodiments of the present disclosure can be implemented in an application scenario as shown in FIG. 13, for example. Figure 13 Figure 13 A schematic diagram of an application scenario 1300 of launching an advertisement according to an embodiment of the present disclosure is shown. Figure 13 A server 1310 and multiple terminals 1320 are shown.

[0110] The server 1310 can be an independent server for performing advertisement analysis, or can be a server cluster composed of multiple physical servers or a distributed system, or can be a cloud server providing basic cloud computing services such as cloud service, cloud database, cloud computing, cloud function, cloud storage, network service, cloud communication, middleware service, domain name service, security service, positioning service, and big data and artificial intelligence platform, etc., and the present disclosure does not make specific limitation thereon. Each of the multiple terminals 1320 can be a fixed terminal such as a desktop computer, a mobile terminal such as a smart phone, a tablet computer, a portable computer, a handheld device, a personal digital assistant, a smart wearable device, or any combination thereof, and the present disclosure does not make specific limitation thereon.

[0111] The advertisement estimated conversion rate calibration method 100 and the advertisement ranking method 1200 according to the above-mentioned embodiments of the present disclosure can be loaded on the server 1310. After the calibrated estimated conversion rates of the multiple to-be-launched advertisements are obtained by using the calibration method 100, the multiple to-be-launched advertisements can be ranked based on the estimated revenue by using the advertisement ranking method 1200, and one or more to-be-launched advertisements with the highest estimated revenue are selected to be pushed to the multiple terminals 1320 for display in a new launching period, or each to-be-launched advertisement in the multiple to-be-launched advertisements can be sequentially pushed to the multiple terminals 1320 for display based on the ranking result, etc.

[0112] Next, the calibration device for the advertisement estimated conversion rate according to an embodiment of the present disclosure is described with reference to Figure 14 Figure 14 A structural schematic diagram of the calibration device for the advertisement estimated conversion rate 1200 according to an embodiment of the present disclosure is shown. As shown in FIG. 12, the calibration device 1200 can include a processor 1201, a memory 1202, and a communication interface 1203. Figure 14 ​​As shown, the calibration apparatus 1400 comprises a first determining unit 1410, an obtaining unit 1420, a second determining unit 1430 and a calibration unit 1440. In addition to the four units, the calibration apparatus 1400 can comprise other components, however, since these components are irrelevant to the content of the embodiments of the present disclosure, the illustration and description thereof are omitted here. In addition, since the functions of the calibration apparatus 1400 are similar to those described above with reference to the calibration method 100, the repeated description of part of the content is omitted here for the sake of simplicity. Figure 1 The details of the steps of the calibration method 100 described above are similar, and therefore, for the sake of simplicity, the repeated description of part of the content is omitted here.

[0113] The first determining unit 1410 is configured to determine a calibration time period for calibrating the estimated conversion rate of a target advertisement. Here, the target advertisement can be any advertisement for which the estimated conversion rate needs to be calibrated, and the embodiments of the present disclosure do not make specific limitations thereon. The calibration time period can be a time period determined based on a predetermined rule, for example, a time period in which the estimated conversion rate of the target advertisement deviates greatly, as will be further described below; or can also be any time period specified based on actual needs, and the embodiments of the present disclosure do not make specific limitations thereon.

[0114] After the first determining unit 1410 determines the calibration time period, it is necessary to obtain a reference advertisement set used as a calibration reference for the estimated conversion rate of the target advertisement, so as to calibrate the estimated conversion rate of the target advertisement by using the estimated conversion rate and the actual conversion rate of the reference advertisement set.

[0115] Specifically, first, the first determining unit 1410 is configured to determine a plurality of advertisement sets associated with the target advertisement based on at least part of a plurality of advertisement attributes of the target advertisement. Among them, the advertisement attributes of the target advertisement can include the advertiser, the advertisement commodity, the advertisement commodity brand, the advertisement targeting, the advertisement region, etc. The advertisements in each advertisement set associated with the target advertisement have at least one common attribute, for example, belong to the same advertiser, contain the same advertisement commodity, have the same advertisement commodity brand, etc. And at least part of the at least one common attribute of each advertisement set associated with the target advertisement is included in the plurality of advertisement attributes of the target advertisement. For example, a certain advertisement set associated with the target advertisement can have one common attribute, and the common attribute can be the same as a certain advertisement attribute of the target advertisement. For another example, another advertisement set associated with the target advertisement can have two common attributes, and one of the two common attributes can be the same as a certain advertisement attribute of the target advertisement.

[0116] The following is described in connection with specific examples. For example, a target advertisement can have three advertisement attributes of an advertiser A, an advertised product B, and an advertised product brand C, based on which three advertisement sets associated with the target advertisement can be determined. Among them, all advertisements in the first advertisement set belong to the advertiser A, i.e., have a common attribute A, all advertisements in the second advertisement set include the advertised product B, i.e., have a common attribute B, and all advertisements in the third advertisement set include the product of the brand C, i.e., have a common attribute C; or, the advertisements in the first advertisement set have a common attribute A, the advertisements in the second advertisement set have a common attribute B and another common attribute D (e.g., all have the same advertisement targeting D), the advertisements in the third advertisement set have a common attribute C and another common attribute E (e.g., all have the same advertisement area E), and so on. It should be noted that the target advertisement has three advertisement attributes and the associated first, second, and third advertisement sets are only examples. In the embodiments of the present disclosure, the target advertisement can have more or fewer advertisement attributes, and more or fewer numbers of advertisement sets associated with the target advertisement can be determined based thereon, and the embodiments of the present disclosure do not specifically limit this.

[0117] Subsequently, the obtaining unit 1420 obtains the estimated conversion rate and the actual conversion rate of each of the plurality of advertisement sets associated with the target advertisement. For example, the obtaining unit 1420 can directly obtain the estimated conversion rate and the actual conversion rate of each advertisement set from an advertisement database pre-statistically stored on a server. The advertisement database may, for example, include the estimated conversion rates and the actual conversion rates of different advertisement sets obtained according to different aggregation dimensions, such as the total estimated conversion rate and the total actual conversion rate of all advertisements under different advertisers, the total estimated conversion rate and the total actual conversion rate of all advertisements under different advertised products, the total estimated conversion rate and the total actual conversion rate of all advertisements under different advertised product brands, and so on. Alternatively, the estimated conversion rate and the actual conversion rate of each advertisement set can also be statistically obtained, as will be further described below.

[0118] Specifically, for each ad set, the obtaining unit 1420 can first obtain the predicted conversion rate and the actual conversion rate of each ad in the ad set in each time period of a plurality of time periods. Here, the plurality of time periods can be, for example, different time periods in a day determined in advance, the predicted conversion rate of each ad is obtained by, for example, a conversion rate prediction model such as PNN, FM, etc. as described above, and the actual conversion rate can come from the statistics of the advertiser. After obtaining the predicted conversion rate and the actual conversion rate of all ads in the ad set, the obtaining unit 1420 can cumulatively sum the predicted conversion rate and the actual conversion rate of each ad in the plurality of time periods to calculate the total predicted conversion rate and the total actual conversion rate of all ads in the ad set in the plurality of time periods, and take the calculated total predicted conversion rate and the total actual conversion rate as the predicted conversion rate and the actual conversion rate of the ad set, respectively.

[0119] For example, in the above example, the target ad has three ad attributes of advertiser A, ad commodity B and ad commodity brand C, and the ads in the first ad set associated with the target ad have the common attribute A, the ads in the second ad set have the common attribute B, and the ads in the third ad set have the common attribute C. Then the total predicted conversion rate and the total actual conversion rate of all ads in the first ad set can be counted to be the predicted conversion rate PCVR-A and the actual conversion rate CVR-A of the first ad set, respectively, and the predicted conversion rate PCVR-B and the actual conversion rate CVR-B of the second ad set and the predicted conversion rate PCVR-C and the actual conversion rate CVR-C of the third ad set can be obtained similarly.

[0120] The second determining unit 1430 is configured to determine a reference ad set used as a calibration reference from the plurality of ad sets associated with the target ad as described above, and determine the calibration parameter using the predicted conversion rate and the actual conversion rate of the reference ad set.

[0121] According to one example of an embodiment of the present disclosure, the second determining unit 1430 can determine the reference ad set based on the data validity of each ad set in the plurality of ad sets, and determine one or more ad sets with valid data as the reference ad set. Here, the data validity of the ad set can indicate whether the data such as the predicted conversion rate and the reference conversion rate of the ad set has reference value.

[0122] For example, the second determining unit 1430 can determine the data validity of the advertisement set according to the total consumption of the advertisements in the advertisement set. Here, the consumption of the advertisements refers to the fee required to be charged to the advertiser for a certain amount or a certain time of advertisement delivery. The consumption of the advertisements can reflect the click volume or the conversion volume of the advertisements, and can reflect whether the advertisements are exposed enough, so as to be used to determine the data validity of the advertisement set. According to an example of the embodiment of the present disclosure, when the total consumption of the advertisements in the advertisement set is greater than a predetermined threshold, it can be determined that the data of the advertisement set is valid; otherwise, it is determined that the data of the advertisement set is invalid. The predetermined threshold can be set according to the actual situation, for example, the predetermined threshold can be set according to the target conversion cost of the advertiser. The target conversion cost of the advertiser is the fee required for each conversion of a single advertisement expected by the advertiser. For example, when the total consumption of the advertisements in the advertisement set is greater than 4 times the target conversion cost, it can be determined that the data of the advertisement set is valid; otherwise, it is determined that the data of the advertisement set is invalid.

[0123] It should be noted that although the above is described by taking the example of determining the data validity of the advertisement set according to the total consumption of the advertisements, the embodiment of the present disclosure is not limited thereto, and the data validity of the advertisement set can also be determined according to, for example, the estimated conversion rate, the actual conversion rate, the click volume, the click rate, etc.

[0124] According to another example of the embodiment of the present disclosure, the second determining unit 1430 can determine the reference advertisement set based on the priority of each of the plurality of advertisement sets, and determine the advertisement set with the highest priority as the reference advertisement set. The priority of the advertisement set can be determined in advance according to, for example, the relevance of the advertisement set to the target advertisement, or according to the actual demand. For example, in the above example that the first, second and third advertisement sets associated with the target advertisement have the common attributes A, B and C respectively, if it is determined that the advertisements in the second advertisement set with the same attribute B as the target advertisement are most relevant to the target advertisement, it can be determined that the priority of the second advertisement set is the highest, and the second advertisement set is determined as the reference advertisement set.

[0125] According to another example of the embodiment of the present disclosure, the second determining unit 1430 can determine the reference advertisement set as the advertisement set with valid data and the highest priority among the plurality of advertisement sets associated with the target advertisement. For example, in the above example, it is determined that the data of the first and third advertisement sets is valid according to the total consumption of the advertisements in the advertisement sets, but the priority of the third advertisement set is the highest, so the third advertisement set can be determined as the reference advertisement set.

[0126] After determining the reference ad set used as the calibration reference for the target ad, the second determining unit 1430 can determine the calibration parameter by using the estimated conversion rate and the actual conversion rate of the reference ad set. Since the ads in the reference ad set have at least one attribute in common with the target ad, the estimated conversion rate and / or the actual conversion rate of the ads in the reference ad set and the target ad have similar characteristics, and thus the estimated conversion rate and the actual conversion rate of the reference ad set can be used as the calibration reference for the target ad. According to examples of embodiments of the present disclosure, the quotient of the actual conversion rate and the estimated conversion rate of the reference ad set can be determined as the calibration parameter. For example, if the second determining unit 1430 determines that the second ad set having the common attribute B with the target ad is the reference ad set, the calibration parameter a = CVR-B / PCVR-B can be determined based on the estimated conversion rate PCVR-B and the actual conversion rate CVR-B of the second ad set. Alternatively, the product of the quotient of the actual conversion rate and the estimated conversion rate of the reference ad set and a predetermined weight factor can also be used as the calibration parameter. Or, the calibration parameter can also be determined by more complex processing of the estimated conversion rate and the actual conversion rate of the reference ad set, which is not specifically limited by embodiments of the present disclosure.

[0127] Next, the calibration unit 1440 calibrates the estimated conversion rate of the target ad in the calibration time period determined by the first determining unit 1410 by using the calibration parameter described above. For example, the estimated conversion rate can be calibrated by multiplying the estimated conversion rate obtained by a conversion rate estimation model such as PNN, FM, etc. by the calibration parameter; or the calibration parameter can be used as a weight factor of the conversion rate estimation model such as PNN, FM, etc. to directly generate the calibrated estimated conversion rate.

[0128] It should be noted that although the estimated conversion rate of the target ad is calibrated by using the data of the reference ad set associated with the target ad in the foregoing, embodiments of the present disclosure are not limited thereto. In the case where the target ad has been launched for a sufficient time to generate sufficient data, the target ad itself can also be calibrated by using the historical data of the same launch period, i.e., the estimated conversion rate and the actual conversion rate of the target ad in an earlier time period of the same launch period are used to calibrate the estimated conversion rate of the target ad in a later time period of the same launch period. Alternatively, the target ad can also be calibrated in combination with the data of the reference ad set associated with the target ad and the historical data of the target ad itself. In these cases, the steps of obtaining the calibration parameter and calibrating the target ad are similar to the steps described in detail above, which will not be described here.

[0129] The step of determining the calibration time period for calibrating the estimated conversion rate of the target ad by the first determining unit 1410 will be described in further detail below.

[0130] The first determining unit 1410 can determine the calibration time period for the target advertisement based on a distribution of the estimated conversion rates of the sample advertisements in the sample advertisement set. Wherein, the sample advertisement set is a set of sample advertisements that have at least one common attribute with the target advertisement, for example, all the sample advertisements in the sample advertisement set are targeted to the same advertisement, for example, are all targeted to an online shopping platform, so that the estimated conversion rate bias of the sample advertisements and the target advertisement has similar time distribution characteristics. And the actual conversion rate and the estimated conversion rate of the sample advertisements are known, for example, have been stored in the database as historical data. Therefore, by analyzing the distribution characteristics of the estimated conversion rate bias of the sample advertisements in the sample advertisement set, the calibration time period for calibrating the estimated conversion rate of the target advertisement can be determined. Here, the sample advertisement set can be different from or the same as the advertisement set in the plurality of advertisement sets described above, and the present disclosure does not make specific limitations thereto.

[0131] Specifically, the first determining unit 1410 can obtain the distribution of the estimated conversion rate bias of the sample advertisements in the sample advertisement set in a plurality of time periods, and determine the calibration time period for calibrating the estimated conversion rate of the target advertisement based on the distribution. As described above, the plurality of time periods may, for example, be different time periods in a day determined in advance, and the estimated conversion rate bias represents the bias between the estimated conversion rate and the actual conversion rate.

[0132] According to one example of an embodiment of the present disclosure, the process of the first determining unit 1410 obtaining the distribution of the estimated conversion rate bias of the sample advertisements in the sample advertisement set in a plurality of time periods can include calculating the statistical mean of the estimated conversion rate bias of the sample advertisements in the sample advertisement set in each time period of the plurality of time periods. Specifically, for each time period in the plurality of time periods, the statistical mean of the estimated conversion rate bias in the time period can be calculated by the following process.

[0133] First, the first determining unit 1410 obtains the actual conversion rate and the estimated conversion rate of each sample advertisement in the sample advertisement set in the time period. As described previously, the actual conversion rate and the estimated conversion rate of the sample advertisement may, for example, be obtained from the historical data stored in the database in advance.

[0134] Subsequently, the first determining unit 1410 calculates the estimated conversion rate bias of the sample ad based on the actual conversion rate and the estimated conversion rate of each ad. For example, the quotient of the estimated conversion rate and the actual conversion rate (which can be denoted as BIAS) can be taken as the estimated conversion rate bias of the sample ad. Alternatively, the absolute value of the quotient of the estimated conversion rate and the actual conversion rate minus 1 (which can be denoted as |BIAS-1|) can be taken as the estimated conversion rate bias of the sample ad. Or, the estimated conversion rate bias can be determined by further processing the estimated conversion rate and the actual conversion rate.

[0135] After obtaining the estimated conversion rate bias of each sample ad in the time period, the first determining unit 1410 can calculate the statistical mean of the estimated conversion rate biases of all sample ads in the sample ad set in the time period. Here, the statistical mean can be, for example, the average value, the mean square value, the root mean square value, the weighted value, etc. of the estimated conversion rate biases of all sample ads. Taking the weighted value as an example, the statistical mean of the estimated conversion rate biases of all sample ads in the sample ad set can be calculated by multiplying the estimated conversion rate bias of each sample ad by its corresponding weight and accumulating the products.

[0136] For example, the first determining unit 1410 can determine the corresponding weight of each sample ad based on the ad consumption of the sample ad. Specifically, first, the ad consumption of each sample ad in the sample ad set in the current time period is obtained. As mentioned earlier, the ad consumption can reflect the number of clicks or conversions of the ad, so the sample ad with larger ad consumption has a relatively larger contribution to the statistical mean of the estimated conversion rate bias. Therefore, the sample ad with larger ad consumption can be assigned a larger weight. For example, the normalized ad consumption of each sample ad can be taken as its corresponding weight by normalizing the ad consumption of all sample ads. It should be understood that the embodiments of the present disclosure are not limited to determining the weight of each sample ad based on the ad consumption, but any other appropriate method can be used to determine the weight.

[0137] In this example, after obtaining the statistical mean of the estimated conversion rate biases of the sample ad set, the first determining unit 1410 can also be configured to determine a calibration time period for calibrating the estimated conversion rate of the target ad based on the statistical mean of the estimated conversion rate biases of the sample ad set. For example, the time period with the largest statistical mean of the estimated conversion rate biases of the sample ad set can be determined as the calibration time period.

[0138] According to another example of the embodiments of the present disclosure, the process that the first determining unit 1410 obtains the distribution of the estimated conversion rate bias of the sample advertisements in the sample advertisement set in the plurality of time periods can include: calculating the distribution proportion of the estimated conversion rate bias of the sample advertisements in the sample advertisement set in each time period of the plurality of time periods in different evaluation intervals, and the like. The evaluation interval can indicate the accuracy of the estimated conversion rate of the sample advertisement. For example, the evaluation interval can include at least one overestimation interval and at least one underestimation interval. If the estimated conversion rate bias of the sample advertisement in a certain time period falls into the overestimation interval, it means that the conversion rate is overestimated. If it falls into the underestimation interval, it means that the conversion rate is underestimated. The method for obtaining the estimated conversion rate bias of each advertisement is the same as the above description of the method for obtaining the estimated conversion rate bias of the sample advertisement, which will not be repeated here. Figure 2 The details of the steps described are the same as those described above, which will not be repeated here.

[0139] Specifically, for each time period of the plurality of time periods, the first determining unit 1410 can calculate the distribution proportion of the estimated conversion rate bias of each sample advertisement in the sample advertisement set in the time period in different evaluation intervals, for example, the proportion of the number of sample advertisements whose estimated conversion rate bias falls into each of the at least one overestimation interval and the at least one underestimation interval to the total number of sample advertisements in the sample advertisement set. In this example, after obtaining the distribution proportion of the estimated conversion rate bias of the sample advertisement set, the first determining unit 1410 can be further configured to determine the calibration time period for calibrating the estimated conversion rate of the target advertisement based on the distribution proportion of the estimated conversion rate bias of the sample advertisement set. For example, the time period in which the proportion of the estimated conversion rate bias of the sample advertisement falling into the overestimation interval is the largest can be determined as the calibration time period; alternatively, the time period in which the proportion of the estimated conversion rate bias of the sample advertisement falling into the underestimation interval is the largest can be determined as the calibration time period, and the like.

[0140] Through the above process of the first determining unit 1410, the calibration time period for calibrating the estimated conversion rate of the target advertisement can be determined by analyzing the distribution characteristics of the estimated conversion rate of the sample advertisement set, so as to calibrate the estimated conversion rate of the target advertisement in the calibration time period, thereby achieving the most ideal calibration effect.

[0141] By using the calibration apparatus for estimated conversion rates of advertisements according to the above embodiments of the present disclosure, it is possible to analyze and determine a specific time period for calibrating the estimated conversion rate of a target advertisement, and for that specific time period, calibrate the estimated conversion rate of the target advertisement using a set of reference advertisements associated with the target advertisement, thereby obtaining a more accurate estimated conversion rate of the target advertisement; furthermore, by using the calibration apparatus according to the embodiments of the present disclosure, targeted calibration can be performed by analyzing the characteristics of the estimated conversion rates of advertisements for specific industries, specific audiences, specific content, etc., so as to more accurately place advertisements, control advertising costs, predict advertising effects, etc.

[0142] The following reference Figure 15 This disclosure describes an advertising sorting apparatus according to embodiments thereof. Figure 15 A schematic diagram of the structure of an advertising sorting device 1500 according to an embodiment of the present disclosure is shown. Figure 15 As shown, the advertising sorting device 1500 includes an acquisition unit 1510, a revenue estimation unit 1520, and a sorting unit 1530. Besides these three units, the advertising sorting device 1500 may also include other components; however, since these components are not relevant to the content of this disclosure embodiment, their illustrations and descriptions are omitted here. Furthermore, since the function of the advertising sorting device 1500 is the same as described above... Figure 12 The details of the steps in the described ad ranking method 1200 are similar, so for simplicity, repeated descriptions of some parts are omitted here.

[0143] The acquisition unit 1510 is configured to acquire a calibrated estimated conversion rate for each of a plurality of ads to be delivered. The plurality of ads to be delivered can be any number of ads to be delivered; this embodiment does not impose specific limitations on this. In this embodiment, for example, the acquisition unit 1510 can utilize the above-mentioned reference... Figure 1 The described calibration method for advertising conversion rate calibration calibrates the estimated conversion rate of each of a plurality of ads to be delivered, thereby obtaining a calibrated estimated conversion rate for each of the plurality of ads to be delivered. However, the embodiments of this disclosure are not limited to this, and the calibrated estimated conversion rate of each ad to be delivered can also be obtained by other means. The estimated conversion rate of each ad to be delivered can be obtained, for example, through conversion rate prediction models such as PNN and FM as described above, or through any other method; the embodiments of this disclosure do not impose specific limitations on this. Since the above description has already detailed the use of... Figure 1 The calibration method shown here involves steps to calibrate the estimated conversion rate of the advertisement; therefore, for simplicity, repeated descriptions of the same content are omitted here.

[0144] The revenue estimation unit 1520 is configured to calculate an estimated revenue of each of the plurality of to-be-delivered advertisements according to the calibrated estimated conversion rate of each of the plurality of to-be-delivered advertisements. For example, the estimated revenue of a to-be-delivered advertisement can be measured by effective Cost Per Mile (eCPM), but the embodiments of the present disclosure are not limited thereto, and can also be measured by any other index such as Return on Investment (ROI). Generally, the eCPM of an advertisement can depend on the estimated conversion rate (PCVR), the estimated click-through rate (PCTR), and the bid of the advertiser. Among them, the bid of the advertiser can refer to the fee paid by the advertiser for one advertisement conversion; the estimated click-through rate refers to the estimated click-through rate of the advertisement after the advertisement is delivered. Therefore, the eCPM of the advertisement can be expressed as:

[0145] eCPM = PCVR x PCTR x bid

[0146] As can be seen from the above formula, the higher the estimated conversion rate PCVR of an advertisement, the higher the estimated revenue eCPM thereof, thereby bringing higher revenue to the advertisement platform. After the calibrated estimated conversion rate of each to-be-delivered advertisement is obtained in step S1210, a more accurate estimated revenue eCPM of each to-be-delivered advertisement can be calculated by using the above formula. Among them, the estimated click-through rate PCTR can be obtained by using the estimation method known in the art, and the bid of the advertiser depends on the real-time bid of the advertiser.

[0147] After the revenue estimation unit 1520 obtains the estimated revenue such as eCPM of each to-be-delivered advertisement, the ranking unit 1530 can rank the plurality of to-be-delivered advertisements based on the estimated revenue of each to-be-delivered advertisement. For example, the to-be-delivered advertisement with higher estimated revenue can be ranked in a more forward position, and the to-be-delivered advertisement with lower estimated revenue can be ranked in a more backward position. After the ranking is completed, the ranking unit 1530 can deliver the plurality of to-be-delivered advertisements in sequence according to the ranking result in a new delivery period, wherein the new delivery period can refer to a new delivery time period, a new delivery day, and the like. For example, one or more to-be-delivered advertisements ranked in the most forward position can be selected for delivery; or the to-be-delivered advertisement ranked in the most forward position can be selected for delivery first, and then the to-be-delivered advertisement ranked in the second position can be selected for delivery, and so on.

[0148] By utilizing the advertising sorting apparatus according to the above embodiments of the present disclosure, the estimated revenue of the advertisement to be placed can be calculated more accurately based on the calibrated estimated conversion rate of the advertisement to be placed, so as to sort multiple advertisements to be placed based on the estimated revenue, thereby prioritizing the placement of advertisements with higher estimated revenue, so as to maximize the revenue of the advertising platform.

[0149] Furthermore, the devices according to embodiments of this disclosure (e.g., calibration devices for predicting advertising conversion rates, advertising sorting devices, etc.) can also be used by means of Figure 16 The architecture of the exemplary computing device shown is used to implement this. Figure 16 A schematic diagram of the architecture of an exemplary computing device according to an embodiment of the present disclosure is shown. Figure 16 As shown, computing device 1600 may include a bus 1610, one or more CPUs 1620, read-only memory (ROM) 1630, random access memory (RAM) 1640, a communication port 1650 connected to a network, input / output components 1660, a hard disk 1670, etc. Storage devices in computing device 1600, such as ROM 1630 or hard disk 1670, may store various data or files used for computer processing and / or communication, as well as program instructions executed by the CPU. Computing device 1600 may also include a user interface 1680. Of course, Figure 16 The architecture shown is merely exemplary and can be omitted as needed when implementing different devices. Figure 16 One or more components in the computing device shown. The device according to embodiments of this disclosure can be configured to perform a calibration method for estimated advertising conversion rates and an advertising sorting method according to the various embodiments of this disclosure above, or to implement a calibration apparatus for estimated advertising conversion rates and an advertising sorting apparatus according to the various embodiments of this disclosure above.

[0150] The embodiments of this disclosure can also be implemented as a computer-readable storage medium. A computer-readable storage medium according to embodiments of this disclosure stores computer-readable instructions. When the computer-readable instructions are executed by a processor, the calibration method for estimated conversion rates of advertisements and the advertisement ranking method according to embodiments of this disclosure, as described with reference to the above figures, can be performed. The computer-readable storage medium includes, but is not limited to, volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc.

[0151] According to an embodiment of the present disclosure, a computer program product or computer program is also provided, which comprises computer readable instructions stored in a computer readable storage medium. A processor of a computer device can read the computer readable instructions from the computer readable storage medium, and the processor executes the computer readable instructions, so that the computer device performs the calibration method of advertisement estimated conversion rate and the advertisement ranking method described in various embodiments above.

[0152] Those skilled in the art can understand that the disclosed content of the present disclosure can have various modifications and improvements. For example, the various devices or components described above can be implemented by hardware, or by software, firmware, or a combination of some or all of the three.

[0153] In addition, as shown in the present disclosure and claims, unless the context clearly indicates otherwise, "a", "an", "one", and / or "the" do not specify a singular number, but also include a plural number. The "first", "second", and similar words used in the present disclosure do not indicate any order, number, or importance, but are only used to distinguish different components. Similarly, "include" or "contain" and similar words mean that the elements or objects before the word cover the elements or objects listed after the word and their equivalents, and do not exclude other elements or objects. "Connected" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect.

[0154] In addition, flowcharts are used in the present disclosure to illustrate the operations performed by the system according to the embodiments of the present disclosure. It should be understood that the preceding or following operations are not necessarily performed in sequence. On the contrary, various steps can be processed in reverse order or simultaneously. At the same time, other operations can be added to these processes, or one or more steps of the operation can be removed from these processes.

[0155] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. It should also be understood that terms such as those defined in generally dictionaries should be interpreted as having meanings consistent with their meanings in the context of the relevant art, and should not be interpreted in an idealized or overly formalized sense, unless explicitly defined herein.

[0156] The present disclosure has been described in detail, but it will be obvious to those skilled in the art that the present disclosure is not limited to the embodiments described in the specification. The present disclosure can be implemented as modifications and changes without departing from the spirit and scope of the present disclosure defined by the recitations of the claims. Therefore, the recitations of the specification are intended to serve as illustrative purposes only and are not intended to have any limiting meaning on the present disclosure.

Claims

1. A calibration method for advertising conversion rate estimation, comprising: Determine the calibration period for calibrating the estimated conversion rate of the target advertisement; Based on at least a portion of multiple ad attributes of a target ad, a plurality of ad sets associated with the target ad are determined, wherein ads in each of the plurality of ad sets have at least one common attribute, and at least a portion of the at least one common attribute is included in the plurality of ad attributes; Obtain the estimated conversion rate and actual conversion rate for each of the multiple ad sets; A reference ad set is determined from the plurality of ad sets to be used as a calibration reference, and calibration parameters are determined using the estimated conversion rate and actual conversion rate of the reference ad set; as well as The estimated conversion rate of the target advertisement is calibrated using the calibration parameters within the calibration time period.

2. The calibration method according to claim 1, wherein, Determining a reference ad set from the plurality of ad sets to be used as a calibration reference includes: The reference ad set is determined based at least on the data validity of each of the plurality of ad sets. The data validity indicates whether the total ad consumption of the ad set is greater than a predetermined threshold. The total ad consumption is the fee charged to the advertiser for a certain amount or time of ad delivery in the ad set.

3. The calibration method according to claim 2, wherein, Determining a reference ad set from the plurality of ad sets for use as a calibration reference further includes: The set of ads with valid data and the highest priority among the multiple ad sets is determined as the reference ad set.

4. The calibration method according to claim 1, wherein, Obtaining the estimated conversion rate and actual conversion rate for each of the plurality of ad sets includes, for each of the plurality of ad sets: Obtain the estimated conversion rate and actual conversion rate of each ad in the ad set for each time period across multiple time periods; The total estimated conversion rate and total actual conversion rate of the ads in the ad set over the multiple time periods are calculated by summing the estimated conversion rates and actual conversion rates of each ad. These sums are used as the estimated conversion rate and actual conversion rate of the ad set.

5. The calibration method according to claim 1, wherein, Determining calibration parameters using the estimated and actual conversion rates of the reference ad set includes: The quotient of the actual conversion rate and the estimated conversion rate of the reference ad set is determined as the calibration parameter.

6. The calibration method according to claim 1, wherein, The multiple advertising attributes include at least one of the following: advertiser, advertised product, advertised product brand, advertising targeting, and advertising region.

7. The calibration method according to claim 1, wherein, The calibration period for determining the estimated conversion rate of the target ad includes: Obtain the distribution of the predicted conversion rate deviation of sample ads in the sample ad set over multiple time periods; and Based on the distribution, a calibration period is determined for calibrating the estimated conversion rate of the target advertisement. The sample ads in the sample ad set have at least one common attribute with the target ad, and the sample ad set may be different from or the same as the ad set in the plurality of ad sets.

8. The calibration method according to claim 7, wherein, The distribution of predicted conversion rate deviations for sample ads within the sample ad set across multiple time periods includes: Calculate the statistical mean of the predicted conversion rate deviation of the sample ads in the sample ad set for each of the multiple time periods, and The calibration time period determined based on the distribution for calibrating the estimated conversion rate of the target advertisement includes: The time period with the largest statistical mean of the predicted conversion rate deviation of the sample ad set is determined as the calibration time period.

9. The calibration method according to claim 8, wherein, The statistical mean of the predicted conversion rate deviation of the sample ads in the sample ad set for each of the multiple time periods includes: For each of the multiple time periods: Obtain the actual conversion rate and estimated conversion rate of each sample ad in the sample ad set during the specified time period; Based on the actual conversion rate and the estimated conversion rate, calculate the deviation of the estimated conversion rate for each sample advertisement within the stated time period; and Calculate the statistical mean of the predicted conversion rate deviations of the sample ads in the sample ad set during the time period.

10. The calibration method according to claim 9, wherein, The calculation of the statistical mean of the predicted conversion rate deviation of sample ads in the sample ad set during the said time period includes: Obtain the ad consumption of each sample ad in the sample ad set during the specified time period; Based on the ad consumption, determine the weight of each sample ad in the sample ad set; and The statistical mean over the time period is calculated by multiplying the absolute value of the estimated conversion rate deviation of each sample ad in the sample ad set by its corresponding weight, and then summing the products.

11. The calibration method according to claim 7, wherein, The distribution of predicted conversion rate deviations for sample ads within the sample ad set across multiple time periods includes: Calculate the distribution ratio of the estimated conversion rate deviation of the sample ads in the sample ad set in each of the multiple time periods across different evaluation intervals, wherein the evaluation intervals include at least one overestimation interval and at least one underestimation interval, and The calibration time period determined based on the distribution for calibrating the estimated conversion rate of the target advertisement includes: The calibration period is defined as the time period in which the proportion of the estimated conversion rate deviation of the sample advertisement falls into the overestimated range or the proportion of the underestimated range.

12. A calibration device for predicting advertising conversion rates, comprising: The first determining unit is configured to determine a calibration time period for calibrating the estimated conversion rate of a target advertisement, and to determine a plurality of advertisement sets associated with the target advertisement based on at least a portion of a plurality of advertisement attributes of the target advertisement, wherein the advertisements in each of the plurality of advertisement sets have at least one common attribute, and at least a portion of the at least one common attribute is included in the plurality of advertisement attributes. The acquisition unit is configured to acquire the estimated conversion rate and the actual conversion rate of each of the plurality of ad sets; The second determining unit is configured to determine a reference ad set from the plurality of ad sets for use as a calibration reference, and to determine calibration parameters using the estimated conversion rate and actual conversion rate of the reference ad set; as well as The calibration unit is configured to calibrate the estimated conversion rate of the target advertisement within the calibration time period using the calibration parameters.

13. A calibration device for predicting advertising conversion rates, comprising: One or more processors; as well as One or more memories, wherein computer-readable code is stored in the memories, and when executed by the one or more processors, the one or more processors cause the one or more processors to perform the method as described in any one of claims 1-11.

14. A computer-readable storage medium having stored thereon computer-readable instructions, which, when executed by a processor, cause the processor to perform the method as described in any one of claims 1-11.

Images