Predicted conversion rate calibration method, apparatus, computing device, and storage medium

Abstract

The present disclosure provides a method, device, computing equipment and storage medium for calibrating the estimated conversion rate of target recommendation information. The method comprises: obtaining the estimated conversion rate of the target recommendation information; determining an initial calibration coefficient for the estimated conversion rate of the target recommendation information according to the historical conversion data of at least one related recommendation information associated with the target recommendation information; determining an adjustment coefficient for adjusting the initial calibration coefficient according to the current consumption, the current conversion quantity and the target conversion cost of the target recommendation information; determining a final calibration coefficient for the estimated conversion rate of the target recommendation information based on the initial calibration coefficient and the adjustment coefficient; and calibrating the estimated conversion rate of the target recommendation information by using the final calibration coefficient. The method helps to improve the accuracy of the calibrated estimated conversion rate while not negatively affecting the consumption of the target recommendation information.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of computer, and in particular, to a method and apparatus for calibrating estimated conversion rate of target recommendation information, a computing device, and a computer readable storage medium. BACKGROUND

[0002] With the development of computer technology, people increasingly use terminal devices to process various transactions in life, work, and study via computer networks. Accordingly, more and more organizations or individuals begin to put recommendation information on network platforms to recommend various objects such as goods, applications, articles, and videos to terminal users, so as to attract terminal users to perform corresponding conversion behaviors such as purchase, download, browsing, and tipping.

[0003] In some scenarios, a network platform can predict the probability of a terminal user performing a corresponding conversion behavior when clicking on a recommendation information, for example, using a machine learning-based conversion rate prediction model to perform such prediction, and for example, further optionally determining a corresponding consumption for an organization or individual putting the recommendation information according to the predicted probability. However, such prediction is often not accurate enough, resulting in the consumption determined by the network platform being too high or too low, which may in turn cause the resources actually delivered to the organization or individual putting the recommendation information to be higher or lower than the resources that should have been delivered. SUMMARY

[0004] Therefore, the present disclosure provides a method, apparatus, computing device, and storage medium for calibrating the estimated conversion rate of target recommendation information, which can alleviate, mitigate, or even eliminate the above problems.

[0005] According to an aspect of the present disclosure, a method for calibrating the estimated conversion rate of target recommendation information is provided, comprising: obtaining the estimated conversion rate of target recommendation information; determining an initial calibration coefficient for the estimated conversion rate of the target recommendation information according to historical conversion data of at least one related recommendation information associated with the target recommendation information; determining an adjustment coefficient for adjusting the initial calibration coefficient according to the current consumption, the current conversion quantity, and the target conversion cost of the target recommendation information, wherein the current consumption is the total consumption of the target recommendation information before the current time, and the target conversion cost is the expected cost of a single conversion of the target recommendation information; determining a final calibration coefficient for the estimated conversion rate of the target recommendation information based on the initial calibration coefficient and the adjustment coefficient; and calibrating the estimated conversion rate of the target recommendation information using the final calibration coefficient.

[0006] In some embodiments, the determining the adjustment coefficient for adjusting the initial calibration coefficient according to the current consumption, the current conversion quantity, and the target conversion cost of the target recommendation information comprises: determining a current cost of the target recommendation information based on the current conversion quantity and the target conversion cost of the target recommendation information; determining a current consumption cost deviation based on the current consumption and the current cost of the target recommendation information; and determining the adjustment coefficient according to the current consumption and the current consumption cost deviation.

[0007] In some embodiments, the determining the adjustment coefficient based on the current consumption and the current consumption cost deviation comprises: determining a threshold interval in which the current consumption and the current consumption cost deviation are located; and determining the adjustment coefficient according to the determined threshold interval.

[0008] In some embodiments, the determining the adjustment coefficient according to the determined threshold interval comprises: determining the adjustment coefficient as a preset value when the determined threshold interval belongs to an interval range in which the current consumption is less than or equal to a first consumption threshold or the current consumption cost deviation is greater than or equal to a first deviation threshold.

[0009] In some embodiments, the determining the adjustment coefficient according to the determined threshold interval comprises: determining the adjustment coefficient negatively related to the current consumption cost deviation when the determined threshold interval belongs to an interval range in which the current consumption is greater than or equal to the first consumption threshold and the current consumption cost deviation is less than or equal to the first deviation threshold.

[0010] In some embodiments, the determining the initial calibration coefficient for the estimated conversion rate of the target recommendation information according to the historical conversion data of at least one related recommendation information associated with the target recommendation information comprises: obtaining the historical conversion data of the at least one related recommendation information associated with the target recommendation information, the historical conversion data comprising an estimated conversion rate corresponding to each click of the at least one related recommendation information and an actual conversion situation indicating whether the click is successfully converted; determining a cumulative conversion quantity of the at least one related recommendation information and a weighted sum of the estimated conversion rates corresponding to each click based on the estimated conversion rate corresponding to each click of the at least one related recommendation information and the actual conversion situation; and determining the initial calibration coefficient based on the determined cumulative conversion quantity and the weighted sum of the estimated conversion rates.

[0011] In some embodiments, the obtaining the historical conversion data of the at least one related recommendation information associated with the target recommendation information comprises: when consumption in a latest statistical period is greater than a preset threshold, obtaining the historical conversion data in the latest statistical period of the at least one related recommendation information, wherein the consumption in the latest statistical period is total consumption in the latest statistical period for the at least one related recommendation information.

[0012] In some embodiments, the determining the cumulative conversion quantity of the at least one related recommendation information and the weighted sum of the estimated conversion rates corresponding to each click comprises: determining the cumulative conversion quantity in the latest statistical period of the at least one related recommendation information based on the historical conversion data in the latest statistical period, and determining the weighted sum of the estimated conversion rates corresponding to each click with equal weights.

[0013] In some embodiments, the obtaining the historical conversion data of the at least one related recommendation information associated with the target recommendation information comprises: when consumption in a latest statistical period is less than a preset threshold, obtaining the historical conversion data in a plurality of statistical periods of the at least one related recommendation information.

[0014] In some embodiments, the determining the cumulative conversion quantity of the at least one related recommendation information and the weighted sum of the estimated conversion rates corresponding to each click comprises: determining the conversion quantity in each statistical period and the sum of the estimated conversion rates corresponding to each click based on the historical conversion data in the plurality of statistical periods; determining the weighted sum of the conversion quantities corresponding to the plurality of statistical periods as the cumulative conversion quantity of the at least one related recommendation information, and determining the weighted sum of the sums of the estimated conversion rates corresponding to the plurality of statistical periods as the weighted sum of the estimated conversion rates of the at least one related recommendation information, wherein the weight corresponding to a statistical period farther away from the current time is smaller.

[0015] In some embodiments, the obtaining the historical conversion data of the at least one related recommendation information associated with the target recommendation information comprises: obtaining historical conversion data of a plurality of groups of related recommendation information associated with the target recommendation information according to a plurality of preset attribute dimensions, wherein the plurality of preset attribute dimensions correspond to the plurality of groups of related recommendation information one by one, and the plurality of preset attribute dimensions have respective preset priorities; judging the validity of the corresponding historical conversion data in turn according to the descending order of the preset priorities of the plurality of preset attribute dimensions; and taking the historical conversion data first judged as valid as the obtained historical conversion data of the at least one related recommendation information associated with the target recommendation information.

[0016] In some embodiments, the obtaining, according to the plurality of preset attribute dimensions, the historical conversion data of the plurality of groups of related recommendation information associated with the target recommendation information comprises: obtaining, according to the plurality of preset attribute dimensions and at least one additional attribute dimension, the historical conversion data of the plurality of groups of related recommendation information associated with the target recommendation information, wherein each related recommendation information in the plurality of groups of related recommendation information is consistent with the target recommendation information in all additional attribute dimensions in the at least one additional attribute dimension, and is consistent with the target recommendation information in at least one preset attribute dimension in the plurality of preset attribute dimensions.

[0017] According to another aspect of the present disclosure, a calibration device for calibrating an estimated conversion rate of target recommendation information is provided, comprising: an obtaining module configured to obtain an estimated conversion rate of target recommendation information; a first determining module configured to determine an initial calibration coefficient for the estimated conversion rate of the target recommendation information according to historical conversion data of at least one related recommendation information associated with the target recommendation information; a second determining module configured to determine an adjustment coefficient for adjusting the initial calibration coefficient according to a current consumption, a current conversion quantity and a target conversion cost of the target recommendation information, wherein the current consumption is a total consumption before the current time for the target recommendation information, and the target conversion cost is an expected cost for a single conversion of the target recommendation information; a third determining module configured to determine a final calibration coefficient for the estimated conversion rate of the target recommendation information based on the initial calibration coefficient and the adjustment coefficient; and a calibration module configured to calibrate the estimated conversion rate of the target recommendation information using the final calibration coefficient.

[0018] According to still another aspect of the present disclosure, a computing device is provided, comprising: a memory configured to store computer executable instructions; and a processor configured to execute the method provided by the preceding aspects of the present disclosure when the computer executable instructions are executed by the processor.

[0019] According to still another aspect of the present disclosure, a computer readable storage medium is provided, which stores computer executable instructions, when the computer executable instructions are executed, the method provided by the preceding aspects of the present disclosure is executed.

[0020] By the method provided in the present disclosure, the initial calibration coefficient can be determined according to the historical conversion data of at least one related recommendation information associated with the target recommendation information, and the adjustment coefficient can be determined according to the current consumption, the current conversion quantity and the target conversion cost of the target recommendation information, and then the final calibration coefficient can be determined based on the determined initial calibration coefficient and adjustment coefficient, which can be used to calibrate the estimated conversion rate of the target recommendation information. Thus, the estimated conversion rate of the target recommendation information can be calibrated by means of the historical conversion data of the related recommendation information and the self data of the target recommendation information, so that the final calibration coefficient suitable for the target recommendation information can be determined efficiently and accurately, thereby helping to improve the accuracy of the calibrated estimated conversion rate without negatively affecting the consumption and other indicators of the target recommendation information. Further, this helps to improve the accuracy of the consumption determined based on the estimated conversion rate.

[0021] These and other aspects of the present disclosure will be apparent from and elucidated with reference to the embodiments described hereinafter. BRIEF DESCRIPTION OF DRAWINGS

[0022] In the following description of example embodiments in conjunction with the attached drawings, further details, features and advantages of the present disclosure are disclosed, in which:

[0023] Figure 1 An example scenario to which the technical solutions provided by some embodiments of the present disclosure can be applied is schematically shown;

[0024] Figure 2 A charging and bidding scheme for recommendation information in the related art is schematically shown;

[0025] Figure 3 An example flowchart of the calibration method according to some embodiments of the present disclosure is schematically shown;

[0026] Figure 4A An example flowchart of the sub-step in Figure 3 according to some embodiments of the present disclosure is schematically shown;

[0027] Figure 4B An example flowchart of the sub-step in Figure 3 according to some embodiments of the present disclosure is schematically shown;

[0028] Figure 5A An example flowchart of another sub-step in Figure 3 according to some embodiments of the present disclosure is schematically shown;

[0029] Figure 5B An example flowchart of the sub-step in Figure 3an example flowchart of another sub-step in the method of FIG. 1 1 ;

[0030] Figure 6 An example block diagram of a calibration device is illustratively shown, in accordance with some embodiments of the present disclosure;

[0031] Figure 7 An example block diagram of a computing device is illustratively shown, in accordance with some embodiments of the present disclosure. DETAILED DESCRIPTION

[0032] Before the embodiments of the present disclosure are described in detail, some related concepts are first explained.

[0033] 1. Recommended information: information used to recommend one or more objects, such as an advertisement. The recommended information can be placed by an organization or an individual through one or more network platforms, and the network platform can present the placed recommended information on the terminal device of one or more users according to certain rules. When the user clicks on the recommended information presented on the terminal device, the terminal device can display a new page or window to present the recommended object, such as a commodity purchase page, an application download link, an article browsing page, a video playing page, etc.

[0034] 2. Click: a human-computer interaction behavior that can be used to trigger the opening of a window, a page, an application, etc. related to the clicked recommended information. The click can be implemented in various possible ways, such as using a mouse, a touchpad, a touch screen, etc. input device or through gesture control, voice control, motion control, light signal, etc. to realize the click behavior.

[0035] 3. Conversion: refers to the user completing a preset user behavior for the recommended object after clicking on the recommended information, such as a purchase behavior for a commodity purchase page, a download behavior for an application download link, a browsing (such as browsing time exceeding a predetermined threshold), a like, a forward, or a reward behavior for an article browsing page or a video playing page, etc.

[0036] 4. Predicted Conversion Rate (PCVR): generally refers to the probability of the user performing a corresponding conversion behavior after clicking on the recommended information. Generally speaking, the PCVR can be predicted using a conversion rate prediction model, such as using the user portrait of the user and various attributes of the recommended information as model inputs to make the prediction.

[0037] 5. PCVR calibration: a kind of post-adjustment of the prediction result of the conversion rate prediction model, for example, the PCVR can be adjusted in a targeted manner based on a specific industry.

[0038] 6、Industry Factor: a factor used to re-adjust the PCVR of recommended information such as advertisements for a certain industry. For example, such re-adjustment can be a specific industry adjustment to the PCVR estimate based on a certain industry, or a specific demographic effect enhancement to the PCVR based on a certain industry-specific demographic. Exemplarily, in the context of direct e-commerce, industry factors can include an industry-based direct e-commerce PCVR make-up factor and a specific demographic-based direct e-commerce high-conversion demographic enhancement factor.

[0039] 7、COST: refers to the amount of resources actually delivered or to be delivered by an organization or individual placing recommended information on a network platform to the network platform. Exemplarily, resources can refer to any deliverable entity or non-entity resources such as currency, electronic currency, virtual currency, digital currency, virtual props, platform points, etc. In some embodiments, COST can reflect the revenue of the network platform, and the greater the COST, the higher the revenue of the network platform can be.

[0040] 8、Target Conversion Cost: refers to the expected cost for a single conversion of recommended information, more specifically, the amount of resources that an organization or individual placing recommended information expects to deliver to the network platform for a single conversion of recommended information. The target conversion cost is usually set in advance by the organization or individual placing recommended information. Exemplarily, in the context of advertisements, it can also refer to the target bid for conversion, which can be set in advance by the advertiser.

[0041] 9、GMV (Gross Merchandise Volume): refers to the total amount of resources that an organization or individual placing recommended information on a network platform should deliver to the network platform, which can generally be determined by the product of the target conversion cost of recommended information and the actual number of conversions. In some embodiments, GMV can also be understood as the cost of the network platform. Exemplarily, if GMV is greater than COST, it can be considered that the platform is under-received; if GMV is less than COST, it can be considered that the platform is over-received.

[0042] 10、Achievement Rate: for a certain recommended information or multiple recommended information, if its COST is equal to its GMV, it can be considered to be strictly achieved; if the ratio of its COST to its GMV meets a certain interval range around 1, it can be considered to be achieved. Exemplarily, in the context of advertisements, generally, when its COST is equal to its GMV, it can be considered to be strictly achieved; when its COST meets 0.8 <= COST / GMV <= 1.2, i.e., the ratio of its COST to its GMV is between 0.8 and 1.2, it can be considered to be achieved.

[0043] 11. Non-cost overrun rate: for a certain recommended information or multiple recommended information, if the ratio of its consumption and GMV is less than or equal to a certain preset threshold, it can be considered as non-cost overrun. Exemplarily, for an advertisement, generally, when its consumption and GMV satisfy COST / GMV <= 1.2, i.e., the ratio of consumption and GMV is less than 1.2, it can be considered as non-cost overrun. In other words, the non-cost overrun rate only considers the non-cost overrun of the recommended information, and does not consider the under-recovery of the platform.

[0044] Figure 1 An example scenario 100 to which the technical solutions provided by some embodiments of the present disclosure can be applied is schematically shown.

[0045] As shown in Figure 1 , the scenario 100 can include a network platform server 110, a publisher device 130 for putting recommended information on the network platform, and a user terminal device 150, and optionally further include a database device 120 for storing publishing data, and these servers or devices can communicate via a network 170.

[0046] Exemplarily, a publisher manager 140 can put recommended information on the network platform through the publisher device 130, and set a target conversion cost for the published recommended information, for example, set a target bid for the published advertisement. The network platform server 110 can publish recommended information based on a pre-written publishing algorithm. Exemplarily, for a certain publishing position, the network platform server 110 can compare the click probability, conversion probability or possible consumption of several recommended information, and then publish the recommended information with the highest click probability, conversion probability or possible consumption in the publishing position, so as to realize higher interaction probability of users and published recommended information, and thus possibly help to realize higher platform revenue and higher cost benefit rate of the publisher. Exemplarily, a recommended information publishing system (such as an advertisement publishing system) which can be at least partially deployed on the network platform server 110 can sort each recommended information (such as an advertisement) according to the product of the estimated click rate, the estimated conversion rate and the target conversion cost (such as the target bid of the advertisement), and select the recommended information with the highest ranking to publish recommended information on the terminal device. Optionally, the network platform server 110 can obtain historical data related to recommended information from a separate database device 120 or from a local database to realize the above publishing process. The user 160 can browse the recommended information published by the network platform server 110 on the terminal device 150, and optionally obtain other services provided thereby. The user 160 can click the recommended information of interest on the terminal device 150, and optionally perform a conversion behavior. Generally, such conversion behavior can be recorded by the publisher of the recommended information.

[0047] In the present disclosure, the server 110 can be a single server or a cluster of servers, on which an application for performing the calibration method of the estimated conversion rate of the recommended information and storing related data can be run. Alternatively, the server 110 can also run other applications and store other data. For example, the server 110 can include multiple virtual hosts to run different applications and provide different services.

[0048] In the present disclosure, the terminal device 150 and the publisher device 130 can be various types of devices, such as mobile phones, tablets, laptops, wearable devices such as smartwatches, in-vehicle devices, etc. The terminal device 150 can be deployed with a client, which can be used to present the recommended information and optionally provide other services, and can take any of the following forms: a locally installed application, a mini-program accessed via other applications, a web program accessed via a browser, etc. The user 160 can view the recommended information presented by the client through the input / output interface of the terminal device 150 and perform click behavior, achieve conversion behavior, etc. Similarly, the terminal device 130 can be deployed with a client for publishing the recommended information, and the publisher administrator 140 can input data related to the published recommended information and view the publishing results, etc. through the input / output interface of the publisher device 130. Alternatively, the publisher device 130 can be integrated with the network platform server 110, in which case the network platform party and the recommended information publisher can be the same subject.

[0049] In the present disclosure, the database device 120 can be regarded as an electronic file cabinet, i.e. a place for storing electronic files, in which users can add, query, update, delete, etc. data in the files. The so-called "database" is a collection of data stored together in a certain way, shared by multiple users, with as little redundancy as possible, and independent of each other from the application.

[0050] In addition, in the present disclosure, the network 170 can be a wired network connected via cables, optical fibers, etc., or a wireless network such as 2G, 3G, 4G, 5G, Wi-Fi, Bluetooth, ZigBee, Li-Fi, etc.

[0051] Figure 2 A charging and bidding scheme 200 for recommended information in the related art is schematically shown.

[0052] As Figure 2As shown, in some embodiments, the process of delivering a recommended information such as an advertisement can include four stages: exposure, click, conversion, and payment. Exposure of a recommended information can refer to the recommended information being presented in a client, thereby making the recommended information have a chance of being clicked by a user. Payment for a recommended information can refer to the network platform charging a delivery party according to the delivery effect, and the charge can be measured in any type of resource agreed by the two parties. In the related art, the charge required by a delivery party of a recommended information can be determined in multiple ways.

[0053] Exemplarily, the network platform party and the delivery party can agree on a billing point and a bidding point in advance, and how to charge according to the billing point and the bidding point. The billing point refers to what way the network platform party charges. If the billing point is exposure, the network platform party charges the delivery party according to the exposure amount of the recommended information; if the billing point is click, the network platform party charges the delivery party according to the click amount of the recommended information; and so on. The bidding point refers to what way the delivery party bids. If the bidding point is exposure, the delivery party bids according to the exposure amount of the recommended information; if the bidding point is click, the delivery party bids according to the click amount of the recommended information.

[0054] Specifically, in the field of advertisements, if both the billing point and the bidding point are focused on exposure, such an advertisement is a classic CPM (Cost Per Mille, cost per thousand impressions) advertisement; if both the billing point and the bidding point are focused on click, such an advertisement is a classic CPC (Cost Per Click, cost per click) advertisement; if both the billing point and the bidding point are focused on conversion, such an advertisement is a CP A (Cost Per Action, cost per action) advertisement.

[0055] By analyzing the charging and bidding schemes in the related art, the applicant finds that the mode of CPA advertisement that charges and bids by conversion is more helpful to measure the actual delivery effect of recommended information such as advertisements. However, this scheme has many difficulties in implementation. For example, usually, the conversion data of recommended information is recorded by the delivery party of recommended information, and the network platform often can only rely on the active return of the delivery party to obtain these conversion data. In the case of adopting the CPA mode, in order to reduce consumption (i.e., save expenses), the delivery party may choose not to return or return less conversion data, and the network platform party is difficult to supervise this situation, thereby making it difficult to sustain this mode. In this case, the CPA mode needs to be further improved to solve the above problems. For example, modes such as OCPA (Optimized CPA) can be adopted. In this type of mode, the charging point and the bidding point are separated, the charging point is at the click, and the bidding point is at the conversion. Specifically, the network platform party can predict the estimated conversion rate (PCVR) for each click of recommended information, and determine the charge for this click based on the target conversion cost set by the delivery party and the estimated conversion rate. This way can advantageously solve the problems existing in the CPA mode, but its charging accuracy depends heavily on the accuracy of the predicted estimated conversion rate, therefore, how to determine a more accurate estimated conversion rate becomes a key problem in this type of mode.

[0056] Figure 3 An example flowchart of a method 300 (for brevity, hereinafter referred to as calibration method 300) for calibrating the estimated conversion rate of target recommended information according to some embodiments of the present disclosure is schematically shown. Exemplarily, the calibration method 300 can be deployed on the server 110 or a virtual host therein in the form of an application program or the like, to calibrate the estimated conversion rate predicted for the target recommended information. Figure 1 The server 110 or the virtual host therein shown can be used to calibrate the estimated conversion rate predicted for the target recommended information.

[0057] Specifically, in step 310, the estimated conversion rate of the target recommended information can be obtained. Exemplarily, the server or host used to execute the calibration method 300 can determine the estimated conversion rate of the target recommended information by itself, or receive such estimated conversion rate from other servers or hosts. In some embodiments, a machine learning-based model or other algorithmic model can be used to determine the estimated conversion rate of the target recommended information. Exemplarily, one or more of various information data of the user who clicks on the target recommended information (such as age, gender, occupation, interest, etc.), related data of the target recommended information (such as the name, category, historical conversion of the recommended object, etc.), and current environmental data (such as time, location, etc.), etc. can be encoded and input into the model for estimating the conversion rate, and then the above-mentioned estimated conversion rate is determined based on the output of the model.

[0058] At step 320, an initial calibration coefficient of the estimated conversion rate of the target recommendation information can be determined according to historical conversion data of at least one related recommendation information associated with the target recommendation information. In some embodiments, the at least one related recommendation information can include recommendation information having the same or similar attributes as the target recommendation information in at least one attribute dimension, such as one or more of recommendation information having the same or similar audience as the target recommendation information, recommendation information having the same or similar delivery party as the target recommendation information, recommendation information having the same or similar recommendation object as the target recommendation information, recommendation information involving the same or similar region as the target recommendation information, and the like. Optionally, the at least one related recommendation information can or can not include the target recommendation information itself.

[0059] Specifically, in some embodiments, a recommendation information can have a delivery party, an audience, and a recommendation object. The delivery party of the target recommendation information creates or delivers the target recommendation information, and recommendation information created or delivered by the same delivery party can have certain similarity. The audience of the recommendation information can refer to the population targeted by the recommendation information, and historical conversion data of recommendation information having the same or similar audience as the target recommendation information can have higher reference value for the calibration of the estimated conversion rate of the target recommendation information. The audience of the recommendation information can be characterized by a series of targeting, such as basic targeting such as the region, age, or occupation of the user, user device, user use scenario of the recommendation object, and the like, behavioral targeting such as user search behavior, social behavior, browsing behavior, and the like, and interest targeting such as interests of the user, and the like. Alternatively, in the case of too fine-grained targeting and difficulty in obtaining sufficient data, the audience of the recommendation information can be implicitly expressed by information in other dimensions, such as the brand of the recommendation object involved in the recommendation information, and the like, it can be understood that the audience population of the same or similar brand has certain similarity. The recommendation object involved in the recommendation information is the subject content of the recommendation information, and the population interested in the recommendation object involved in the target recommendation information is more likely to complete conversion behavior for the recommendation object. Alternatively or additionally, related recommendation information associated with the target recommendation information can be determined based on attributes of applicable other attribute dimensions.

[0060] In step 330, an adjustment coefficient for adjusting the initial calibration coefficient can be determined based on the current consumption, current conversion count, and target conversion cost of the target recommendation information. The current consumption can be the total consumption for the target recommendation information up to the current time, and the target conversion cost can be the expected cost of a single conversion for the target recommendation information. For example, the server or host can obtain the current consumption, current conversion count, and target conversion cost of the target recommendation information from a separate database or a local database. Optionally, the current consumption and current conversion count of the target recommendation information can refer to the cumulative consumption or cumulative conversion count from midnight of the day to the current time or a time before the current time. Alternatively, the current consumption and current conversion count of the target recommendation information can refer to the cumulative consumption or cumulative conversion count from the first day of the month to the current time or a time before the current time, or the cumulative consumption or cumulative conversion count from the first time the target recommendation information was delivered to the current time or a time before the current time, etc. In other words, the statistical range of the current consumption and current conversion count of the target recommendation information can be limited according to actual needs.

[0061] In step 340, a final calibration coefficient for the estimated conversion rate of the target recommendation information can be determined based on the initial calibration coefficient and the adjustment coefficient. In some embodiments, the product of the initial calibration coefficient and the adjustment coefficient can be determined as the final calibration coefficient. Alternatively, further calculations can be performed based on the product of the initial calibration coefficient and the adjustment coefficient, and the result of the calculations can be determined as the final calibration coefficient. Alternatively, in other embodiments, the ratio, sum, etc., of the initial calibration coefficient and the adjustment coefficient can also be used as the final calibration coefficient.

[0062] In step 350, the estimated conversion rate of the target recommendation information can be calibrated using the final calibration coefficient. In some embodiments, the final calibration coefficient can be multiplied by the estimated conversion rate obtained in step 310, and the product can be used as the calibrated estimated conversion rate. Alternatively, the final calibration coefficient can be used to calibrate the estimated conversion rate of the target recommendation information in other ways, depending on the actual situation.

[0063] pass Figure 3 The calibration method 300 shown, after obtaining the estimated conversion rate of the target recommendation information, can first use historical conversion data of at least one related recommendation information associated with the target recommendation information, as well as data such as the current consumption, current conversion quantity, and target conversion cost of the target recommendation information itself, to calibrate the estimated conversion rate. Optionally, it can determine the consumption based on the calibrated estimated conversion rate. This approach allows for the efficient and accurate determination of the final calibration coefficient applicable to the target recommendation information based on its own attributes and historical data, thereby improving the accuracy of the calibrated estimated conversion rate and thus helping to improve the achievement rate and cost-free rate of the target recommendation information.

[0064] As an example, Figure 4A An example flowchart of step 320 in Figure 3 is shown in FIG. 3B. It should be understood that although the sub-steps of step 320 are shown in Figure 4A in FIG. 3B, this is merely exemplary, and in practice, step 320 can also be performed according to other flows.

[0065] At step 321, historical conversion data of at least one related recommendation information associated with the target recommendation information can be obtained, which can include an estimated conversion rate corresponding to each click of the at least one related recommendation information and an actual conversion condition indicating whether the click is successfully converted. Exemplarily, the server or host can obtain the historical conversion data of the at least one related recommendation information associated with the target recommendation information from a separate database or a local database. In some embodiments, the obtained historical conversion data can only include the estimated conversion rate corresponding to each click of the at least one related recommendation information and the actual conversion condition indicating whether the click is successfully converted; or the obtained historical conversion data can include more abundant data, such as exposure time of the at least one related recommendation information, whether each exposure obtains a click, time of the click, etc. in addition to the above-mentioned estimated conversion rate and actual conversion condition.

[0066] In some embodiments, the historical conversion data of the at least one related recommendation information associated with the target recommendation information can be obtained according to a plurality of preset attribute dimensions. Specifically, first, the historical conversion data of a plurality of groups of related recommendation information associated with the target recommendation information can be obtained according to a plurality of preset attribute dimensions, wherein the plurality of preset attribute dimensions correspond to the plurality of groups of related recommendation information one by one, and the plurality of preset attribute dimensions have respective preset priorities; then, the validity of the corresponding historical conversion data can be judged in turn according to the descending order of the preset priorities of the plurality of preset attribute dimensions; finally, the historical conversion data that is first judged to be valid can be obtained as the historical conversion data of the at least one related recommendation information associated with the target recommendation information. Exemplarily, the preset attribute dimensions can include one or more of various attribute dimensions such as the publisher of the recommendation information, the type of the recommended object corresponding to the recommendation information, the name, the audience of the recommendation information, etc. By selecting the highest priority and valid historical conversion data from the historical conversion data of the plurality of groups of related recommendation information associated with the target recommendation information, it is helpful to ensure that sufficient historical conversion data can be obtained to perform the calibration process of the estimated conversion rate of the target recommendation information, thereby helping to further improve the accuracy of the calibrated estimated conversion rate.

[0067] Exemplarily, the server or host can read its attributes in multiple preset attribute dimensions from the attribute table of the target recommendation information, and then find the recommendation information with the same or similar attributes as each attribute in the local database or separate database according to the history conversion data thereof. For example, the attributes of the recommendation information can be encoded as vectors, the attributes with the same vectors can be regarded as the same attributes, or the attributes with the distance (such as cosine distance, etc.) between the vectors less than a preset threshold can be regarded as similar attributes. Alternatively, other judgment mechanisms can also be adopted.

[0068] Specifically, taking the commodity advertisement as an example, the advertiser ID (Identity, identity), the advertisement commodity brand, and the advertisement commodity can be selected as the preset attribute dimensions. In this case, the advertiser ID, the advertisement commodity brand, and the advertisement commodity corresponding to the target advertisement can be obtained from the information table of the target advertisement, and then three groups of related advertisements corresponding to the related advertisements with the same (or similar) advertiser ID, the same (or similar) advertisement commodity brand, and the same (or similar) advertisement commodity can be obtained. Then, the effectiveness of the history conversion data of the three groups of related advertisements can be judged according to the priority. Optionally, the priority descending order can be set as: the advertiser ID, the advertisement commodity, and the advertisement commodity brand. In addition, the way of judging the effectiveness of the data can be set as: when the consumption is greater than 4 times the target conversion cost, the data is effective; otherwise, the data is invalid. It should be understood that the basis for judging the effectiveness can be set according to the needs, and is not limited to the consumption greater than 4 times the target conversion cost as recorded herein. It can also be set as the consumption greater than 5 times, 6 times, or a certain fixed value of the target conversion cost, or it can also be set as the conversion quantity greater than 4, 5, 6, etc., or it can also be based on other quantities to judge the effectiveness of the data. In addition, the target conversion cost used for judging the effectiveness herein can be the target conversion cost of the target recommendation information, or it can also be the average, maximum, median, etc. of the target conversion cost of at least one related recommendation information. Thus, the effectiveness of the history conversion data of each group of related advertisements can be judged in turn according to the priority descending order, and finally the history conversion data with the highest priority and effectiveness is retained.

[0069] In some embodiments, the historical conversion data of the multiple sets of relevant recommendation information associated with the target recommendation information can be obtained according to multiple preset attribute dimensions and at least one additional attribute dimension. Specifically, the historical conversion data of the multiple sets of relevant recommendation information associated with the target recommendation information can be obtained according to multiple preset attribute dimensions and at least one additional attribute dimension, where each relevant recommendation information in the multiple sets of relevant recommendation information has the same attribute as the target recommendation information in all the additional attribute dimensions in the at least one additional attribute dimension, and has the same attribute as the target recommendation information in at least one preset attribute dimension in the multiple preset attribute dimensions. For example, the additional attribute dimensions can include attribute dimensions related to the time, manner, range, etc. of the delivery of the recommendation information. By adding at least one additional attribute dimension as a screening condition to screen the relevant recommendation information, it is helpful to more accurately obtain the historical conversion data of the relevant recommendation information that is more referential to the target recommendation information, thereby helping to further improve the accuracy of the calibrated estimated conversion rate.

[0070] Specifically, still taking the commodity advertisement as an example, the advertiser ID, the advertisement commodity brand, and the advertisement commodity can still be selected as the preset attribute dimensions, and the site set to which the advertisement belongs and whether the advertisement is new or old can be selected as the additional attribute dimensions. In this case, the historical conversion data of the relevant advertisements that belong to the same site set as the target advertisement and are new advertisements or old advertisements can be respectively counted under the advertiser ID dimension, the advertisement commodity brand dimension, and the advertisement commodity dimension. For example, assuming that there are two site sets, site set 27 and site set 28, and the advertisements exposed on the same day and not exposed before are regarded as new advertisements, and the advertisements exposed before are regarded as old advertisements (or other mechanisms for distinguishing new / old advertisements can also be selected), the site set attribute dimension and the new / old advertisement attribute dimension can be combined into four combined attribute dimensions: [site set 27, new advertisement], [site set 27, old advertisement], [site set 28, new advertisement], and [site set 28, old advertisement]. If the target advertisement belongs to [site set 27, new advertisement], the relevant advertisements that have the same advertiser ID as the target advertisement and belong to the same dimension, the relevant advertisements that have the same advertisement commodity brand as the target advertisement and belong to the same dimension, and the relevant advertisements that have the same advertisement commodity as the target advertisement and belong to the same dimension can be respectively obtained, and then the effectiveness of the historical conversion data of each set of relevant advertisements is judged according to the preset priority order, and the historical conversion data with the highest priority and effectiveness is retained.

[0071] In some embodiments, step 321 can include steps 3211, 3212, and 3213 as shown in Figure 4B Similarly, it should be understood that, although steps 3211, 3212, and 3213 are shown in Figure 4BThe sub-steps of step 321 are shown in FIG. 3, but this is merely exemplary, and in practice, step 321 can also be performed according to other procedures.

[0072] Specifically, in step 3211, it can be determined whether the consumption in the most recent statistical period is greater than a preset threshold, where the consumption in the most recent statistical period is the total consumption of the at least one related recommendation information in the most recent statistical period. The preset threshold can be selected as a suitable value according to experience and actual needs, for example, 4 times, 5 times, 6 times, etc. of the target conversion cost. The target conversion cost involved in the preset threshold can be the target conversion cost of the target recommendation information, or can also be the average, maximum, median, etc. of the target conversion cost of the at least one related recommendation information. The statistical period can be set to 1 hour, 2 hours, 3 hours, 1 day, etc. according to actual needs.

[0073] When the consumption in the most recent statistical period is greater than the preset threshold, step 3212 can be performed to obtain the historical conversion data of the at least one related recommendation information in the most recent statistical period.

[0074] When the consumption in the most recent statistical period is less than or equal to the preset threshold, step 3213 can be performed to obtain the historical conversion data of the at least one related recommendation information in the most recent multiple statistical periods. The multiple statistical periods can be a preset number of statistical periods, for example, 6 statistical periods, 8 statistical periods, 12 statistical periods, etc., or can be multiple statistical periods within a certain range, for example, multiple statistical periods within the range from the current zero point to the current time, such as 6 statistical periods from 0 o'clock to 6 o'clock of the day (assuming each statistical period is 1 hour) if the current time is 6 o'clock in the morning.

[0075] At step 322, the accumulated conversion quantity of the at least one relevant recommendation information and the weighted sum of the estimated conversion rate corresponding to each click can be determined based on the estimated conversion rate corresponding to each click of the at least one relevant recommendation information and the actual conversion condition. For example, the accumulated conversion quantity and the weighted sum of the estimated conversion rate can be determined according to actual conditions. For example, the conversion quantity in the last one hour, the last two hours, the last one day, or other time periods can be counted as the accumulated conversion quantity, or the conversion quantities in different time periods can be counted and weighted to determine the accumulated conversion quantity. Similarly, for example, the equal-weighted sum of the estimated conversion rate corresponding to each click in the last one hour, the last two hours, the last one day, or other time periods can be counted as the weighted sum of the estimated conversion rate, or the conversion quantities in different time periods can be counted and weighted to determine the weighted sum of the estimated conversion rate. For example, the weight corresponding to each statistical period can be determined according to different mechanisms, for example, the weight can be determined according to the time distance between the statistical period and the current time, the greater the time distance, the smaller the weight; or the weight can be determined according to the consumption in the statistical period, the greater the consumption, the greater the weight; or the weight of all statistical periods or part of the statistical periods can be set to be equal weight; and the like.

[0076] In some embodiments, step 322 can include steps 3221, 3222 and 3223 as shown in Figure 4B For example, step 3221 can be performed when the consumption in the last statistical period is greater than a preset threshold, and steps 3222 and 3223 can be performed when the consumption in the last statistical period is less than (or equal to) the preset threshold. Similarly, it should be understood that although the sub-steps of step 322 are shown in Figure 4B However, in fact, step 322 can also be performed according to other processes.

[0077] At step 3221, the accumulated conversion quantity of the at least one relevant recommendation information in the last statistical period and the weighted sum of the estimated conversion rate corresponding to each click can be determined based on the historical conversion data in the last statistical period. For example, the accumulated conversion quantity in the last period can be counted based on the actual conversion condition corresponding to each click of the at least one relevant recommendation information in the historical conversion data, and the actual conversion condition can indicate whether each click is successfully converted. Moreover, the sum of the estimated conversion rate corresponding to each click in the last period can be determined by summing operation based on the estimated conversion rate corresponding to each click of the at least one relevant recommendation information in the historical conversion data, in other words, the weighted sum of the estimated conversion rate corresponding to each click can be determined with a weight of 1.

[0078] At step 3222, the number of conversions in each statistical period and the sum of the estimated conversion rates corresponding to each click can be determined based on historical conversion data in the plurality of statistical periods.

[0079] At step 3223, a weighted sum of the number of conversions corresponding to the plurality of statistical periods can be determined as the cumulative number of conversions of the at least one related recommendation information, and a weighted sum of the sum of the estimated conversion rates corresponding to the plurality of statistical periods can be determined as the weighted sum of the estimated conversion rates of the at least one related recommendation information, wherein the weight corresponding to the statistical period farther away from the current time is smaller. Alternatively, the weight corresponding to the number of conversions and the sum of the estimated conversion rates for each statistical period can be equal, so that the final weighted sum can better reflect the relationship between the two. Or, slightly different weights can also be set as needed. Exemplarily, the weight corresponding to each statistical period can be a pre-set constant value, or can also be calculated according to a pre-set rule, for example, a weighting rule based on time decay strategy, a weighting rule based on consumption size, etc.

[0080] At step 323, an initial calibration coefficient can be determined based on the determined cumulative number of conversions and the weighted sum of the estimated conversion rates. In some embodiments, the ratio of the determined cumulative number of conversions to the weighted sum of the estimated conversion rates can be determined as the initial calibration coefficient. Alternatively, the ratio of the determined cumulative number of conversions to the weighted sum of the estimated conversion rates can be further processed to determine the initial calibration coefficient. Alternatively, the determined cumulative number of conversions and / or the weighted sum of the estimated conversion rates can also be further processed, and then the initial calibration coefficient is determined based on the processed cumulative number of conversions and / or the weighted sum of the estimated conversion rates.

[0081] Specifically, the following still takes the commodity advertisement as an example, and more details of the process shown in the figure are described. It is assumed that the advertiser ID, the advertisement commodity brand, and the advertisement commodity are selected as the pre-set attribute dimensions, one statistical period is determined as 1 hour, and the pre-determined threshold related to the consumption in the nearest statistical period is determined as 4 times of the target conversion cost. Figure 4B

[0082] Exemplarily, for the advertiser ID attribute dimension, the historical conversion data of the at least one related recommendation information can be obtained. When the consumption in the nearest hour in the obtained historical conversion data is greater than 4 times of the target conversion cost, the cumulative number of conversions of the at least one related recommendation information in the nearest hour and the sum of the estimated conversion rates can be counted. Similarly, the cumulative number of conversions of the at least one related recommendation information in the nearest hour and the sum of the estimated conversion rates ​and the cumulative conversion number of the at least one related recommendation information in the last one hour and the sum of the estimated conversion rates If the historical conversion data of the at least one related recommendation information obtained for the advertiser ID dimension attribute is the highest priority and valid data, the cumulative conversion number and the sum of the estimated conversion rates are retained as the valid cumulative conversion number and the sum of the estimated conversion rates and the initial calibration coefficient is determined based on the two. For example, the initial calibration coefficient can be determined as .

[0083] In addition, exemplarily, for the advertiser ID attribute dimension, when the consumption of the last one hour in the obtained historical conversion data is less than or equal to 4 times of the target conversion cost, the cumulative conversion number and the weighted sum of the estimated conversion rates can be determined based on the conversion data in all statistical periods from zero o'clock of the day to the current time. First, the sum of the conversion number and the estimated conversion rate in each hour can be counted. Then, the weighted sum of the conversion number in each hour can be determined as the cumulative conversion number based on a time decay strategy (such as linear decay, Gaussian decay, etc.), and the weighted sum of the sum of the estimated conversion rate in each hour can be determined as the weighted sum of the estimated conversion rate based on the same time decay strategy. Specifically, the cumulative conversion number of the day and the weighted sum of the estimated conversion rate can be determined according to the following formula and the weighted sum of the estimated conversion rates :

[0084]

[0085]

[0086] wherein lamda is a time decay coefficient, is the conversion number in the i-th hour, is the sum of the estimated conversion rate in the i-th hour. Exemplarily, the specific value of lamda can be determined according to actual needs, and the larger the value, the faster the decay rate. Alternatively, lamda can be set to 0.05. Similarly, the cumulative conversion number of the at least one related recommendation information in the day and the weighted sum of the estimated conversion rates can be determined for the advertiser commodity brand dimension attribute and the weighted sum of the estimated conversion rates , and the cumulative conversion number of the at least one related recommendation information in the day and the weighted sum of the estimated conversion rates can be determined for the advertiser commodity dimension attribute and the weighted sum of the estimated conversion rates If the historical conversion data of at least one relevant recommendation information obtained for the advertiser ID dimension attribute is the highest priority and effective data, the cumulative conversion number and the estimated conversion rate is retained as the effective cumulative conversion number and the sum of the estimated conversion rate , and determines the initial calibration coefficient based on both. For example, the initial calibration coefficient can be determined as .

[0087] It is found through experiments that when the estimated conversion rate is calibrated using the initial calibration coefficient determined according to the above method, it can be found that the achievement rate and the non-exceeding cost rate of the target recommendation information are obviously improved, but the consumption and GMV exist to a certain extent. Decrease. Specifically, a set of experimental data for commodity advertisements is shown below, which shows the mean and distribution of PCVR bias (PCVR_bias) of several advertisements in four time periods throughout the day.

[0088]

[0089] Table 1: Mean of PCVR_bias of advertisements in four time periods throughout the day

[0090]

[0091] Table 2: Distribution of PCVR_bias of advertisements in four time periods throughout the day.

[0092] Specifically, the PCVR_bias can be calculated by the following process. First, the number of conversions, the sum of PCVR of valid clicks, and the spend of a single ad in each hour can be counted. For example, the historical conversion data of each hour can be sorted in chronological order, and the number of conversions, the sum of PCVR of valid clicks, and the spend can be calculated by the cumulative sum function (such as cumsum). The valid clicks can refer to the clicks after removing the cheating clicks, and the spend can be obtained by accumulating the actual delivery resources of the advertiser corresponding to each click. Then, the bias value in each hour PCVR_bias = sum of PCVR in the hour / number of conversions in the hour can be calculated. Alternatively, the bias value in a certain period PCVR_bias = sum of PCVR in the period / number of conversions in the period can be calculated according to the above process, such as the 0-6 hours, 6-12 hours, 12-18 hours, and 18-24 hours in Table 1. In order to more intuitively represent the difference between the sum of PCVR and the number of conversions, abs(bias-1) = |PCVR_bias-1| can be further calculated. Subsequently, the abs(bias-1) of each ad can be weighted and summed based on the spend of each ad to obtain the final abs(bias-1) of several ads in each period. Further, Table 2 shows the probability of PCVR_bias of several ads falling into different intervals in each period, and through these probabilities, the proportion of overestimation and underestimation of PCVR can be intuitively seen.

[0093] From the results of Table 1, it can be seen that after calibration using the initial calibration coefficient, the PCVR in each period becomes more accurate, which helps to improve the achievement rate and the non-over-cost rate of the ad. However, from the results of Table 2, it can be seen that after calibration using the initial calibration coefficient, the proportion of overestimation of PCVR decreases, especially in the earlier period, and this decrease is more obvious, which will lead to the value of the estimated ECPM (Effective Cost Per Mile) being too low, and further lead to the spend and GMV being too low. Therefore, it is necessary to further adjust the initial calibration coefficient by steps 330 and 340 shown in the figure, which allows the PCVR to be adjusted according to the current spend, the current number of conversions, and the target conversion cost of the target recommendation information, thereby reducing the proportion of underestimation of PCVR, and further allowing the spend and GMV to be increased possibly without significantly affecting the overall accuracy of PCVR. Figure 3

[0094] As an example, Figure 5A An example flowchart of step 330 in Figure 3 according to some embodiments of the present disclosure is schematically shown. It should be understood that although the sub-steps of step 330 are shown in Figure 5A , this is only exemplary, and in fact, step 330 can also be performed according to other flows.​

[0095] At step 331, the current cost of the target recommendation information can be determined based on the current conversion quantity of the target recommendation information and the target conversion cost. Illustratively, the current cost of the target recommendation information can be determined by multiplying the current conversion quantity of the target recommendation information and the target conversion cost; or, the current cost of the target recommendation information can be determined by further scaling or other processing based on the product of the current conversion quantity of the target recommendation information and the target conversion cost. In some embodiments, the current cost of the target recommendation information can reflect the amount of resources that the publisher should deliver to the network platform for the target recommendation information before the current time.

[0096] At step 332, the current consumption cost deviation can be determined based on the current consumption and the current cost of the target recommendation information. Illustratively, the current consumption cost deviation can be determined by the ratio of the current consumption and the current cost (i.e., current consumption / current cost deviation); or, the current consumption cost deviation can be determined by pre-processing the current consumption and / or the current cost, and then determining the ratio thereof; or, the current consumption cost deviation can be determined by further processing the ratio, such as subtracting 1 from the ratio, etc.

[0097] At step 333, the adjustment coefficient can be determined according to the current consumption and the current consumption cost deviation. Intuitively, when the current consumption is low, the overall data is more susceptible to interference, and thus a lower adjustment coefficient should be determined for the initial calibration coefficient of the estimated conversion rate at this stage; when the current consumption cost deviation is high, it indicates that the overestimation proportion in the estimated conversion rate before the current time can have been too large, and thus a higher adjustment coefficient should not be determined for the initial calibration coefficient thereof. In view of the above analysis, the adjustment coefficient for the initial calibration coefficient of the estimated conversion rate can be determined according to the threshold interval in which the current consumption and the current consumption cost deviation are located.

[0098] In some embodiments, as shown in FIG. 3B, step 333 can include step 3331 and step 3332. At step 3331, the threshold interval in which the current consumption and the current consumption cost deviation are located can be determined; at step 3332, the adjustment coefficient can be determined according to the determined threshold interval. Similarly, it should be understood that although the sub-steps of step 333 are shown in FIG. 3B, this is merely exemplary, and in practice, step 333 can also be performed according to other procedures. Figure 5B Figure 5B

[0099] ​​In some embodiments, when the determined threshold interval belongs to an interval range in which the current consumption is less than or equal to a first consumption threshold or the current consumption cost deviation is greater than or equal to a first deviation threshold, the adjustment coefficient can be determined as a preset value; when the determined threshold interval belongs to an interval range in which the current consumption is greater than or equal to the first consumption threshold and the current consumption cost deviation is less than or equal to the first deviation threshold, the adjustment coefficient can be determined in negative correlation with the current consumption cost deviation.

[0100] Specifically, the following still takes the commodity advertisement as an example to more specifically describe the flow shown in the figure. Figure 5B

[0101] The advertisement can be divided into three stages according to the size of the current consumption: the current consumption is less than 8 times the target conversion cost target_cpa, the current consumption is between 8 times and 25 times the target conversion cost target_cpa, and the current consumption is greater than 25 times the target conversion cost target_cpa.

[0102] When the advertisement consumption is less than 8 times target_cpa (which can be considered as the first consumption threshold mentioned in the foregoing), the adjustment coefficient coef can be set to 1, that is, the initial calibration coefficient is not adjusted.

[0103] When the advertisement consumption is between 8 times and 25 times target_cpa, the adjustment coefficient coef can be determined by the following formula:

[0104]

[0105] When the advertisement consumption is greater than 25 times target_cpa, the adjustment coefficient coef can be determined by the following formula:

[0106]

[0107] In the above formula, target_cpa is the target bid of the advertisement, that is, the target conversion cost; current_cpa_bias is the current consumption cost deviation of the advertisement in the CPA mode, which can be calculated by current_cpa_bias = cost / (conv_num*target_cpa); cost is the current consumption of the advertisement, and conv_num is the current conversion number of the advertisement. Exemplarily, the value of 1.1 of current_cpa_bias can be considered as the first deviation threshold mentioned in the foregoing.

[0108] ​Subsequently, the initial calibration coefficient determined in step 320 can be adjusted using the adjustment coefficient coef, for example, the final calibration coefficient can be determined as final_cali_rate = cali_rate * coef. Finally, the PCVR acquired in step 310 can be calibrated using the final calibration coefficient, for example, by multiplying the PCVR with the final calibration coefficient to acquire the calibrated PCVR.

[0109] Through experiments on commodity advertisements, it is found that after calibrating the PCVR using the adjusted final calibration coefficient, the achievement rate of the advertisement is increased by about 8%, the non-exceeding cost rate of the advertisement is increased by 10%, and the consumption and GMV of the advertisement are increased by about 2%. Therefore, by calibrating the PCVR using the final calibration coefficient, the achievement rate and the non-exceeding cost rate of the recommended information can be greatly improved while ensuring the stability of the consumption and GMV.

[0110] It should be understood that the above description of the calibration method 300 is merely exemplary and is not intended to limit the scope of the present disclosure. Figures 3-5B The calibration method 300 described can be applied to PCVR calibration of recommended information of various types and various industries, and can be applied to PCVR calibration of recommended information at various stages, such as early, middle and late stages of delivery. It should also be understood that the various steps shown in the figure do not necessarily need to be performed in the order shown. For example, step 320 of determining the initial calibration coefficient and step 330 of determining the adjustment coefficient can be performed in reverse order or in parallel, etc.

[0111] Figure 6 An example block diagram of a calibration device 600 for calibrating the estimated conversion rate of target recommended information according to some embodiments of the present disclosure is schematically shown. For example, the calibration device can be deployed in the network platform server 110 in Figure 1 and used to perform the calibration method 300 described according to Figures 3-5B As shown in Figure 6 , the calibration device 600 includes an acquisition module 610, a first determination module 620, a second determination module 630, a third determination module 640, and a calibration module 650.

[0112] In particular, the obtaining module 610 can be configured to obtain an estimated conversion rate of target recommendation information; the first determining module 620 can be configured to determine an initial calibration coefficient for the estimated conversion rate of the target recommendation information according to historical conversion data of at least one related recommendation information associated with the target recommendation information; the second determining module 630 can be configured to determine an adjustment coefficient for adjusting the initial calibration coefficient according to a current consumption, a current conversion quantity and a target conversion cost of the target recommendation information, wherein the current consumption is a total consumption before the current time for the target recommendation information, and the target conversion cost is an expected cost for a single conversion of the target recommendation information; the third determining module 640 can be configured to determine a final calibration coefficient for the estimated conversion rate of the target recommendation information based on the initial calibration coefficient and the adjustment coefficient; and the calibration module 650 can be configured to calibrate the estimated conversion rate of the target recommendation information by using the final calibration coefficient.

[0113] It should be understood that the calibration apparatus 600 can be implemented in software, hardware or a combination of software and hardware. Multiple different modules can be implemented in the same software or hardware structure, or one module can be implemented by multiple different software or hardware structures.

[0114] In addition, the calibration apparatus 600 can be used to implement the calibration method described in the foregoing, and the related details have been described in the foregoing in detail, and for the sake of brevity, will not be repeated here. The calibration apparatus 600 can have the same features and advantages as described with respect to the foregoing calibration method.

[0115] Figure 7 An example block diagram of a computing device 700 according to some embodiments of the present disclosure is schematically shown. For example, it can represent a server 110 in Figure 1 or other types of computing devices that can be used to deploy the calibration apparatus provided by the present disclosure.

[0116] As shown, the example computing device 700 includes a processing system 701, one or more computer readable medium 702, and one or more I / O interface 703 communicatively coupled with one another. Although not shown, the computing device 700 can also include a system bus or other data and command transfer system that couples the various components to one another in communication. The system bus can include any one or combination of different bus structures, e.g., a memory bus or memory controller, a peripheral bus, a serial bus, a parallel bus, and / or a local bus, using any of a variety of bus architectures, or can also include special purpose buses, such as control and data lines.

[0117] The processing system 701 is representative of the functionality performed by a hardware-based processing unit or device, such as a microprocessor or microcontroller. Alternatively, the processing system 701 can be implemented with a combination of hardware and software. The processing system 701 is capable of processing instructions for execution on the processing system 701 and can include various suitable elements, such as a processor 704. Suitable elements can include one or more of a cache, bus interface, arithmetic logic unit, and control unit. The processing system 701 can further include a microcode read only memory containing microcode. The processor 704 can represent a single processing unit or hardware logic component that is in electrical communication with bus 702. The processor 704 can include a number of physical and functional components including, but not limited to a processor core, a bus interface unit, a low- latency cache, and a cache controller. The processor 704 can be a single processing unit, or it can be a plurality of processing units, functioning together as a system, as described below. The processing system 701 can be a component of a larger system, such as a desktop computer, a laptop computer, a handheld computer, a network router, switch or bridge, or any of various other types of systems. Suitable implementations for the processing system 701 can depend on the type of system in which the processing system 701 is incorporated. The processing system 701 can be a single processing unit, or it can be a plurality of processing units, functioning together as a system, as described below. The processing system 701 can be a component of a larger system, such as a desktop computer, a laptop computer, a handheld computer, a network router, switch or bridge, or any of various other types of systems. Suitable implementations for the processing system 701 can depend on the type of system in which the processing system 701 is incorporated.

[0118] The computer-readable medium 702 is illustrated as including memory / storage 705. The memory / storage 705 represents memory / storage associated with a processing system 701 that is capable of executing a program of instructions. The memory / storage 705 can include volatile memory or non-volatile memory, or a combination of both. The memory / storage 705 can also be used for storing data or other information as described herein. The memory / storage 705 can include memory such as a semiconductor-based memory device (e.g., a random access memory, or RAM), a flash memory or other non-volatile memory or storage, a hard disk drive, etc. The memory / storage 705 can store data including historical conversion data, current consumption, current number of conversions, target conversion cost, etc. as described in the above embodiments. The computer-readable medium 702 can be configured in a computer program product. The computer-readable medium 702 can include a computer-readable medium 702 and can be a computer-readable storage medium or computer-readable storage device, a computer-readable signal medium, or a combination of computer-readable storage and computer-readable signal media.

[0119] The input / output interface(s) 703 are representative of a functional unit allowing a user to enter commands and information to computing device 700, and further allows for information to be presented to the user and / or other components or devices using various input / output devices. Examples of input devices include a keyboard, a cursor control device (e.g., a mouse), a microphone (e.g., for voice inputs), a scanner, touch functionality (e.g., capacitive or other sensors that are configured to detect physical touch), a camera (e.g., which can employ visible or non-visible wavelengths such as infrared frequencies to detect movement that does not involve touch as gestures), a card reader, a receiver, etc. Examples of output devices include a display device (e.g., a monitor or projector), speakers, a printer, a tactile- response device, a network card, a transmitter, etc. Illustratively, in the embodiments described above, a manager can be allowed to input various preset thresholds, etc. through the input devices, a manager can be allowed to monitor the execution process and / or results of the calibration method, etc. through the output devices.

[0120] The computing device 700 also includes a predicted conversion rate calibration application 706. The predicted conversion rate calibration application 706 can be stored in the memory / storage 705 as a computer program instructing. The predicted conversion rate calibration application 706 can be implemented together with the processing system 701, etc. to implement the methods described with respect toFigure 6 The overall functionality of each of the modules of the described calibration apparatus 600.

[0121] Various techniques can be described in the general context of software, hardware, elements or program modules. Generally, these modules include routines, programs, objects, elements, components, data structures, and the like that perform particular tasks or implement particular abstract data types. The terms "module," "functionality," and the like as used herein generally refer to software, firmware, hardware, or a combination thereof. Features of the described technology are platform-independent, meaning that the technology can be implemented on a variety of computing platforms having a variety of processors.

[0122] Implementations of the described modules and techniques can be stored or transmitted across some form of computer-readable media. Computer-readable media can include various media that can be accessed by the computing device 700. By way of example, and not limitation, computer-readable media can include "computer-readable storage media" and "computer-readable signal media."

[0123] In contrast to signal-bearing media, "computer-readable storage media" refers to media or means configured to hold information for a period of time. Accordingly, computer-readable storage media does not include signals per se. Computer-readable storage media includes volatile and non-volatile, removable and non-removable media implemented in a method or technology for storage of information such as computer readable instructions, data structures, program modules, logical elements / circuits, or other data. Examples of computer-readable storage media can include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical storage, hard disks, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or other storage devices, tangible media, or articles of manufacture that are appropriate for storage of desired information and that can be accessed by a computer.

[0124] "Computer-readable signal media" refers to a signal-bearing medium that is configured to transmit instructions to the hardware of the computing device 700, such as via network. Signal media typically can take the form of modulated data signals, such as carrier waves, data signals, or other transport mechanisms. Signal media additionally include any information delivery media. By way of example, and not limitation, the term "signal media" includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared, and other wireless media.

[0125] As previously described, hardware elements 701 and computer-readable media 702 are representative of instructions, modules, programmable device logic and / or fixed device logic implemented in hardware that can be employed in some embodiments to implement at least portions of the techniques described herein. Hardware elements can include components of an integrated circuit or

[0126] The foregoing combination of software and hardware modules also can be used to implement various techniques and modules described herein. Thus, software, hardware, or program modules and other program modules can be implemented as one or more instructions and / or logic embodied on some form of computer-readable storage media and / or by one or more hardware elements 701. The computing device 700 can be configured to implement particular instructions and / or functions corresponding to the software and / or hardware modules. Accordingly, implementation of a module as a software module and / or hardware module, either in whole or in part, can be determined as a matter of design choice by one skilled in the art. For example, software modules can be implemented as software stored in memory or storage and executed on general purpose processors (e.g., as described above), or can be implemented as hardware or as combinations of software and hardware, using any one or more hardware elements 701 and / or computer-readable storage media 702 described above. In a

[0127] The techniques described herein can be supported by these various configurations of the computing device 700 and are not limited to the specific examples of the techniques described herein.

[0128] It should be appreciated that for clarity's sake, the embodiments of the present disclosure have been described with reference to different functional elements. It will be apparent, however, that the functionality of each of the functional elements can be implemented in a single element, multiple elements or as part of other functional elements without departing from the present disclosure. For example, the functionality of a particular functional element can be performed by multiple different elements. Accordingly, references to a particular functional element are to be taken as references to suitable elements for providing the described functionality and not as indicative of a strict logical or physical structure or organization. Thus, the present disclosure can be implemented in a single element or can be physically and functionally distributed across different elements and circuits.

[0129] The present disclosure provides a computer-readable storage medium having stored thereon computer-readable instructions that, when executed, implement the above-described method for calibrating a predicted conversion rate of target recommendation information.

[0130] The present disclosure provides a computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computing device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the computing device to perform the method for calibrating the estimated conversion rate of target recommendation information provided in various optional implementations described above.

[0131] Variations to the disclosed embodiments can become apparent to those of ordinary skill in the art upon reading the foregoing description and understanding the associated drawings. In the claims, the word "comprising" does not exclude other elements or steps, and the word "a" or "an" does not exclude a plurality. The mere fact that measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

Claims

1. A method for calibrating an estimated conversion rate of target recommendation information, comprising: obtaining an estimated conversion rate of target recommendation information; determining an initial calibration coefficient for the estimated conversion rate of the target recommendation information according to historical conversion data of at least one related recommendation information associated with the target recommendation information; determining an adjustment coefficient for adjusting the initial calibration coefficient according to a current consumption, a current conversion quantity and a target conversion cost of the target recommendation information, wherein the current consumption is a total consumption before a current time for the target recommendation information, and the target conversion cost is an expected cost for a single conversion of the target recommendation information; determining a final calibration coefficient for the estimated conversion rate of the target recommendation information based on the initial calibration coefficient and the adjustment coefficient; calibrating the estimated conversion rate of the target recommendation information by using the final calibration coefficient; wherein the determining the initial calibration coefficient for the estimated conversion rate of the target recommendation information according to the historical conversion data of the at least one related recommendation information associated with the target recommendation information comprises: obtaining the historical conversion data of the at least one related recommendation information associated with the target recommendation information, the historical conversion data comprising an estimated conversion rate corresponding to each click of the at least one related recommendation information and an actual conversion indicating whether the click is successfully converted; determining a cumulative conversion quantity and a weighted sum of the estimated conversion rates corresponding to each click of the at least one related recommendation information based on the estimated conversion rate corresponding to each click of the at least one related recommendation information and the actual conversion; determining the initial calibration coefficient based on the determined cumulative conversion quantity and the weighted sum of the estimated conversion rates; wherein the obtaining the historical conversion data of the at least one related recommendation information associated with the target recommendation information comprises: obtaining historical conversion data of a plurality of groups of related recommendation information associated with the target recommendation information according to a plurality of preset attribute dimensions, wherein the plurality of preset attribute dimensions correspond to the plurality of groups of related recommendation information one by one, and the plurality of preset attribute dimensions have respective preset priorities; judging the effectiveness of the corresponding historical conversion data in turn according to a descending order of the preset priorities of the plurality of preset attribute dimensions; taking the historical conversion data that is first judged as valid as the obtained historical conversion data of the at least one related recommendation information associated with the target recommendation information.

2. The method of claim 1, wherein, The determining the adjustment coefficient for adjusting the initial calibration coefficient according to the current consumption, the current conversion quantity and the target conversion cost of the target recommendation information comprises: determining a current cost of the target recommendation information based on the current conversion quantity and the target conversion cost of the target recommendation information; determining a current consumption cost deviation based on the current consumption and the current cost of the target recommendation information; determining the adjustment coefficient according to the current consumption and the current consumption cost deviation.

3. The method of claim 2, wherein, The determining the adjustment coefficient according to the current consumption and the current consumption cost deviation comprises: determining a threshold interval in which the current consumption and the current consumption cost deviation are located. According to the determined threshold interval, the adjustment coefficient is determined.

4. The method of claim 3, wherein, The determination of the adjustment coefficient according to the determined threshold interval comprises: When the determined threshold interval belongs to an interval range in which the current consumption is less than or equal to a first consumption threshold or the current consumption cost deviation is greater than or equal to a first deviation threshold, the adjustment coefficient is determined as a preset value.

5. The method of claim 3, wherein, The determination of the adjustment coefficient according to the determined threshold interval comprises: When the determined threshold interval belongs to an interval range in which the current consumption is greater than or equal to a first consumption threshold and the current consumption cost deviation is less than or equal to a first deviation threshold, the adjustment coefficient is determined in negative correlation with the current consumption cost deviation.

6. The method of claim 1, wherein, The obtaining of the historical conversion data of at least one related recommendation information associated with the target recommendation information comprises: When the consumption in the most recent statistical period is greater than a preset threshold, the historical conversion data in the most recent statistical period of the at least one related recommendation information is obtained, wherein the consumption in the most recent statistical period is the total consumption of the at least one related recommendation information in the most recent statistical period.

7. The method of claim 6, wherein, The determination of the cumulative conversion quantity of the at least one related recommendation information and the weighted sum of the estimated conversion rates corresponding to each click comprises: Based on the historical conversion data in the most recent statistical period, the cumulative conversion quantity of the at least one related recommendation information in the most recent statistical period is determined, and the weighted sum of the estimated conversion rates corresponding to each click is determined with equal weights.

8. The method of claim 1 or 6, wherein, The obtaining of the historical conversion data of at least one related recommendation information associated with the target recommendation information comprises: When the consumption in the most recent statistical period is less than a preset threshold, the historical conversion data in the most recent statistical period of the at least one related recommendation information is obtained.

9. The method of claim 8, wherein, The determination of the cumulative conversion quantity of the at least one related recommendation information and the weighted sum of the estimated conversion rates corresponding to each click comprises: Based on the historical conversion data in the multiple statistical periods, the conversion quantity in each statistical period and the sum of the estimated conversion rates corresponding to each click are determined; The weighted sum of the conversion quantities corresponding to the multiple statistical periods is determined as the cumulative conversion quantity of the at least one related recommendation information, and the weighted sum of the sums of the estimated conversion rates corresponding to the multiple statistical periods is determined as the weighted sum of the estimated conversion rates of the at least one related recommendation information, wherein the weight corresponding to the statistical period farther away from the current time is smaller.

10. The method of claim 1, wherein, The obtaining of the historical conversion data of multiple groups of related recommendation information associated with the target recommendation information according to multiple preset attribute dimensions comprises: According to multiple preset attribute dimensions and at least one additional attribute dimension, the historical conversion data of multiple groups of related recommendation information associated with the target recommendation information is obtained, wherein each related recommendation information in the multiple groups of related recommendation information has the same attribute as the target recommendation information in all additional attribute dimensions in the at least one additional attribute dimension, and has the same attribute as the target recommendation information in at least one preset attribute dimension in the multiple preset attribute dimensions.

11. A calibration apparatus for calibrating an estimated conversion rate of target recommendation information, comprising: an obtaining module configured to obtain an estimated conversion rate of target recommendation information; a first determining module configured to determine an initial calibration coefficient for the estimated conversion rate of the target recommendation information according to historical conversion data of at least one related recommendation information associated with the target recommendation information; a second determining module configured to determine an adjustment coefficient for adjusting the initial calibration coefficient according to a current consumption, a current conversion quantity and a target conversion cost of the target recommendation information, wherein the current consumption is a total consumption before a current time for the target recommendation information, and the target conversion cost is an expected consumption for a single conversion of the target recommendation information; a third determining module configured to determine a final calibration coefficient for the estimated conversion rate of the target recommendation information based on the initial calibration coefficient and the adjustment coefficient; a calibration module configured to calibrate the estimated conversion rate of the target recommendation information by using the final calibration coefficient; wherein the first determining module is further configured to: obtain historical conversion data of at least one related recommendation information associated with the target recommendation information, the historical conversion data comprising an estimated conversion rate corresponding to each click of the at least one related recommendation information and an actual conversion indicating whether the click is successfully converted; determine a cumulative conversion quantity of the at least one related recommendation information and a weighted sum of the estimated conversion rates corresponding to each click based on the estimated conversion rate corresponding to each click of the at least one related recommendation information and the actual conversion; determine the initial calibration coefficient based on the determined cumulative conversion quantity and the weighted sum of the estimated conversion rates; wherein the first determining module is further configured to: obtain historical conversion data of a plurality of groups of related recommendation information associated with the target recommendation information according to a plurality of preset attribute dimensions, wherein the plurality of preset attribute dimensions correspond to the plurality of groups of related recommendation information one by one, and the plurality of preset attribute dimensions have respective preset priorities; determine the effectiveness of the corresponding historical conversion data in turn according to a descending order of the preset priorities of the plurality of preset attribute dimensions; determine the historical conversion data that is first determined to be effective as the obtained historical conversion data of at least one related recommendation information associated with the target recommendation information.

12. A computing device, comprising: a memory configured to store computer-executable instructions; a processor configured to perform the method according to any one of claims 1 to 10 when the computer-executable instructions are executed by the processor.

13. A computer-readable storage medium storing computer-executable instructions, which when executed perform the method according to any one of claims 1 to 10.

Images