Method, apparatus, device, and storage medium for evaluating conversion

Abstract

According to an embodiment of the present invention, there is provided a method, an apparatus, a device, and a storage medium for evaluating conversion. The method includes presenting a first content item associated with a first object on a terminal device corresponding to a content providing platform; presenting, in response to determining that a click operation on the first content item has been detected, an auxiliary content item associated with the first object on the terminal device, where the auxiliary content item at least indicates a portal for performing a conversion operation on the first object; and determining first predicted conversion information for the first content item based on click feedback on the portal, where the click feedback indicates whether the portal has been clicked on the terminal device, and the first predicted conversion information indicates a probability that a conversion operation on the first object is expected to be performed. According to this solution, it is possible to effectively estimate predicted conversion information for a content item without relying on conversion feedback transmission information outside the content providing platform.

Description

Technical Field

[0001] [Cross - reference to Related Applications] This application claims priority to a Chinese patent application for invention titled "Method, Apparatus, Device, and Computer - readable Storage Medium for Evaluating Conversion", with application number 202210693437.2, filed on June 17, 2022.

[0002] [Technical Field] Exemplary embodiments of the present invention generally relate to the field of computer technology, and in particular, to a method, apparatus, device, and computer - readable storage medium for evaluating conversion.

Background Art

[0003] The Internet provides access to a wide variety of content. For example, through the Internet, various images, audio, video, web pages, etc. can be accessed. Also, the accessible content further includes specific content items associated with various objects, such as advertisements. Such content items are usually provided through an agreement between an object provider and a content provider, and the object provider may further pay a fee to the content provider based on the access status of the content item. Therefore, usually, it is necessary to estimate an index related to the presentation of content items including the conversion status of content items, where "conversion" indicates a download, registration, purchase, or other information - request actions that occur under the influence of the presented content item.

Summary of the Invention

[0004] In a first aspect of the present invention, a method for evaluating conversion is provided. The method includes presenting a first content item associated with a first object on a terminal device corresponding to a content providing platform, and in response to determining that a click operation on the first content item has been detected, presenting an auxiliary content item associated with the first object on the terminal device, where the auxiliary content item at least indicates a portal for performing a conversion operation on the first object, and determining first predicted conversion information for the first content item based on click feedback on the portal, where the click feedback indicates whether the portal has been clicked on the terminal device, and the first predicted conversion information indicates the probability that a conversion operation on the first object is expected to be performed.

[0005] In a second aspect of the present invention, a method for evaluating conversion is provided. The method includes presenting a first content item associated with a first object on a terminal device corresponding to a content providing platform, and in response to determining that a click operation on the first content item has been detected, presenting an auxiliary content item associated with the first object on the terminal device, where the auxiliary content item at least indicates a portal for performing a conversion operation on the first object, and providing click feedback on the portal to a content management system, where the click feedback indicates whether the portal has been clicked on the terminal device.

[0006] In a third aspect of the present invention, there is provided an apparatus for evaluating conversion. The apparatus includes a first presentation control module for presenting a first content item associated with a first object on a terminal device corresponding to a content providing platform, and a second presentation control module for presenting an auxiliary content item associated with the first object on the terminal device in response to determining that a click operation on the first content item has been detected, wherein the auxiliary content item at least indicates a portal for performing a conversion operation on the first object, and a conversion determination module for determining first predicted conversion information for the first content item based on click feedback on the portal, wherein the click feedback indicates whether the portal has been clicked on the terminal device, and the first predicted conversion information indicates the probability that a conversion operation on the first object is expected to be performed.

[0007] In a fourth aspect of the present invention, there is provided an apparatus for evaluating conversion. The apparatus includes a first presentation module for presenting a first content item associated with a first object on a terminal device corresponding to a content providing platform, a second presentation module for presenting an auxiliary content item associated with the first object on the terminal device in response to detecting a click operation on the first content item, wherein the auxiliary content item at least indicates a portal for performing a conversion operation on the first object, and a feedback providing module for providing click feedback on the portal to a content management system, wherein the click feedback indicates whether the portal has been clicked on the terminal device.

[0008] In a fifth aspect of the present invention, an electronic device is provided. The device includes at least one processing unit and at least one memory coupled to the at least one processing unit and used for storing instructions to be executed by the at least one processing unit. When the instructions are executed by the at least one processing unit, the method of the first aspect or the second aspect is executed by the device.

[0009] In a sixth aspect of the present invention, a computer-readable storage medium is provided. A computer program is stored on the medium, and when the computer program is executed by a processor, the method of the first aspect or the second aspect is implemented.

[0010] It should be understood that the content described in the summary part of the present invention is not intended to limit the main features or important features of the embodiments of the present invention, nor is it intended to limit the scope of the present invention. Other features of the present invention will be easily understood from the following description.

Brief Description of the Drawings

[0011] With reference to the following detailed description in conjunction with the drawings, the above-described features and other features, advantages, and aspects of each embodiment of the present invention will become more apparent. In the drawings, the same or similar symbols indicate the same or similar elements, among which

Figure 1

Figure 2

Figure 3A

Figure 3B

Figure 3C

Figure 3D

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Mode for Carrying Out the Invention

[0012] Hereinafter, embodiments of the present invention will be described in more detail with reference to the drawings. Although specific embodiments of the present invention are shown in the drawings, the present invention can be implemented in various forms and should not be construed as being limited to the embodiments described herein. Rather, it should be understood that these embodiments are provided for a more thorough and complete understanding of the present invention. It should be understood that the drawings and embodiments of the present invention are for illustrative purposes only and are not used to limit the protection scope of the present invention.

[0013] In the description of the embodiments of the present invention, the term "including" and similar terms are open-ended inclusion meaning "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". There may be other explicit and implicit definitions hereinafter.

[0014] It is understood that the data related to the present technical solution (including but not limited to the data itself, data acquisition, or data use) should comply with the corresponding laws and regulations and related specified requirements.

[0015] Before using the technical solutions disclosed in each embodiment of the present invention, it should be understood that the types, scope of use, usage scenarios, etc. of the personal information related to the present invention should be notified to the user in an appropriate manner in accordance with relevant laws and regulations, and the user's permission should be obtained.

[0016] For example, when responding to receiving an uncommitted request from a user, by sending presentation information to the user, it is explicitly presented to the user that the requested operation requires the acquisition and use of the user's personal information. Thereby, based on the presentation information, the user can independently select whether to provide personal information to software or hardware such as an electronic device, an application, a server, or a storage medium that executes the operation of the technical solution of the present invention.

[0017] As a selective but non-limiting implementation form, the method of sending presentation information to a user in response to receiving an uncommitted request from the user may be, for example, a method using a pop-up window, and the presentation information can be displayed in the form of text within the pop-up window. Further, the pop-up window may further include a selection control for the user to select "agree" or "disagree" to provide personal information to the electronic device.

[0018] It is understood that the above-described notification and user authorization acquisition process are only schematic and do not limit the implementation forms of the present invention, and other means that comply with relevant laws and regulations can also be applied to the implementation forms of the present invention.

[0019] The term "model" as used in this specification can learn the relationship between corresponding inputs and outputs from training data, and thereby generate corresponding outputs for a given input after training is completed. The generation of the model can be based on machine learning techniques. Deep learning is a machine learning algorithm that processes inputs using multiple-layer processing units and provides corresponding outputs. A neural network model is an example based on a deep learning model. In this specification, "model" may be referred to as "machine learning model", "learning model", "machine learning network", or "learning network", and these terms are used interchangeably in this specification.

[0020] A "neural network" is a machine learning network based on deep learning. A neural network can process inputs and provide corresponding outputs, and usually includes an input layer, an output layer, and one or more hidden layers between the input layer and the output layer. Neural networks used in deep learning applications usually include many hidden layers, thereby increasing the depth of the network. Each layer of the neural network is sequentially connected so that the output of the previous layer is provided as the input to the subsequent layer. The input layer receives the input of the neural network, and the output of the output layer functions as the final output of the neural network. Each layer of the neural network includes one or more nodes (also called processing nodes or neurons), and each node processes the input from the previous layer.

[0021] Generally, machine learning is roughly composed of three stages: a training stage, a testing stage, and an application stage (also called an inference stage). In the training stage, a large amount of training data is used to train the given model, and the parameter values are continuously and repeatedly updated until the model can obtain consistent inferences that meet the desired goals from the training data. Through training, it can be considered that the model can learn the association from input to output (also called the mapping from input to output) from the training data. The parameter values of the model after training are determined. In the testing stage, the performance of the model is determined by applying test inputs to the model after training and testing whether the model can provide correct outputs. In the application stage, the model can be used to process actual inputs and determine the corresponding outputs based on the parameter values obtained in training.

[0022] FIG. 1 shows a schematic diagram of an exemplary environment 100 in which embodiments of the present invention can be implemented. One or more content providers manage content provided on content delivery platform 110 using content management system 120. One or more terminal devices 130-1, 130-2, 130-3, etc. (collectively or individually referred to as terminal device 130 for ease of explanation) are associated with content delivery platform 110 and can access various content provided on content delivery platform 110. As an example, content delivery platform 110 may be an application, website, web page, and other accessible resources. Terminal device 130 may have an application installed for accessing content delivery platform 110 or may access content delivery platform 110 in an appropriate manner.

[0023] The content provider can provide different content to different terminal devices 130 based on management requirements and based on user operations in terminal device 130. Content management system 120 can provide one or more specific content items associated with one or more objects, such as one or more content items 142-1, 142-2, …, 142-M (collectively or individually referred to as content item 142 for ease of explanation) within content database 140, to terminal device 130. These content items may include, for example, advertisements. Objects associated with the content items include, for example, objects targeted by the advertisements.

[0024] In some embodiments, the content management system 120 can determine content items 142 to provide to one or more terminal devices 130 based at least on requests from object providers, e.g., based on bidding requests from object providers. In an advertising delivery scenario, an object provider may also be referred to as an advertiser. In some embodiments, an object provider may further pay a fee to a content provider based on the submission of content items.

[0025] In environment 100, the terminal device 110 may be any type of mobile terminal, fixed terminal, or portable terminal, including a mobile phone, desktop computer, laptop computer, notebook computer, netbook computer, tablet computer, media computer, multimedia tablet, personal communication system (PCS) device, personal navigation device, personal digital assistant (PDA), audio / video player, digital camera / video camera, positioning device, television receiver, radio receiver, e-book device, game device, or any combination of the foregoing, including accessories and peripherals of these devices or any combination thereof. In some embodiments, the terminal device 110 can also support any type of interface for the user (such as a "wearable" circuit). The content management system 120 may be, for example, various types of computing systems / servers that can provide computing functions, including but not limited to mainframes, edge computing nodes, computing devices in a cloud environment, etc.

[0026] It should be understood that the structure and function of each element within environment 100 are described only for purposes of illustration and do not imply any limitation to the scope of the present invention.

[0027] In a content item provision scenario, it is usually necessary to estimate indicators related to the presentation of content items, including the conversion status of content items. Among them, "conversion" refers to downloads, registrations, purchases, or other information request actions that occur under the influence of the presented content items. These indicators can be used to determine the presentation effect of content items, to determine how the content provider charges the object provider, to determine the content provider's strategy for providing content items among terminal devices, etc. For example, in the case of advertising for a specific object, the advertiser can expect to pay the content provider according to the number of conversion actions that occur for the advertisement.

[0028] The conversion of content items usually occurs on platforms other than the content provision platform, such as a platform managed by the object provider or a third-party platform. For example, the download action of a multimedia file may occur on the source website of the multimedia file, the download action of a specific application may occur on an application download platform such as the application store of the terminal device or the application download website, and the registration action of an application may occur on an application platform, etc. In such cases, the content provider expects the platform managed by the object provider or the third-party platform to feedback the conversion status of the content item, such as the statistical data of the conversion actions performed on the object over a certain period of time. The fact that the platform managed by the object provider or the third-party platform feedbacks the conversion status regarding the content item is also called "conversion feedback transmission".

[0029] Depending on the scenario, conversion feedback transmission may be blocked or delayed due to privacy concerns. This means that the content provider cannot track in real time whether a conversion action has occurred on the content item after the content item is presented, and cannot accurately evaluate the conversion status of the content item. This is disadvantageous for operations such as estimating the presentation effect of the content item and adjusting subsequent payment and offering strategies.

[0030] In an exemplary embodiment of the present invention, an improved conversion evaluation method is provided. According to the solution, after presenting a content item associated with an object on a terminal device corresponding to a content providing platform, when a click operation on the content item is detected, instead of directly jumping to another platform where a conversion action on the object is performed, continue to provide an auxiliary content item associated with the object. The auxiliary content item is used to further show a portal for performing a conversion action on the object. Based on the click feedback on the portal, determine the predicted conversion information for the content item, and the information indicates the probability that a conversion action on the object is expected to be performed.

[0031] According to the solution, it is possible to effectively estimate the predicted conversion information for the content item without depending on the conversion feedback transmission information outside the content providing platform, and avoid the situation where the conversion evaluation cannot be realized due to the delay of the conversion feedback transmission for various reasons and the failure to obtain the conversion feedback transmission.

[0032] Hereinafter, some exemplary embodiments of the present invention will be described with continued reference to the drawings.

[0033] Figure 2 shows a flowchart of a signaling flow 200 for evaluating conversions according to some embodiments of the present invention. For ease of explanation, the signaling flow 200 will be described with reference to the environment 100 of FIG. 1. The signaling flow 200 includes a content management system 120 and a terminal device 130. Although one terminal device 130 is shown in FIG. 2, the content management system 120 can execute a signaling flow similar to that of a plurality of terminal devices 130.

[0034] In the signaling flow 200, the content management system 120 presents (205) a content item 142 associated with an object at a terminal device 130 corresponding to the content providing platform 110. The terminal device 130 presents (210) the corresponding content item 142, for example, on a user interface corresponding to the content providing platform.

[0035] The content management system 120 can determine when and how to present content items based on various strategies and trigger conditions of the content items 142. In some embodiments, the content management system 120 can also determine different content items 142 in the content database 140 to be presented to different terminal devices 130. In some embodiments, the content management system 120 can control to present one content item 142 at a plurality of terminal devices 132.

[0036] While content item 142 is being presented, terminal device 132 detects a click operation on content item 142. When a click operation on the content item 142 is detected, terminal device 132 provides feedback (215) to content management system 120, and the feedback indicates detection of the click operation on the content item 142. Thereby, content management system 120 can determine detection of the click operation on the content item 142 (220).

[0037] In an embodiment of the present invention, one or more content items 142 in content database 140 have associated auxiliary content items. As an example, content item 142-1 shown in FIG. 2 has associated auxiliary content item 242-1, and content item 142-M has associated auxiliary content item 242-M. For ease of explanation, these auxiliary content items are collectively or individually referred to as auxiliary content item 242 in this specification. In some embodiments, one or more content items 142 (e.g., content item 142-2) may not have auxiliary content items. Embodiments of the present invention are not limited in this regard.

[0038] In response to detection of a click operation on the content item 142 at a particular terminal device 120, content management system 120 presents (225) auxiliary content item 242 associated with the object at terminal device 130. Terminal device 130 presents (230) the corresponding auxiliary content item 142, for example, on a user interface corresponding to a content providing platform.

[0039] The auxiliary content item 242 and the associated content item 142 target the same object. The auxiliary content item 242 at least shows a portal for executing the conversion operation of the object. Clicking on the portal enables the terminal device to access the platform for executing the conversion operation. In an embodiment of the present invention, the presentation of the auxiliary content item is used to further confirm whether the user intends to perform a conversion operation on the object. Such auxiliary content items may also be referred to as generalized items for advertisements or promotional items, etc.

[0040] In some embodiments, the conversion operation evaluated based on the auxiliary content item may be the act of the next step that can be executed after the click operation on the content item. For example, if the object is a downloadable object, the conversion operation may include a download operation. If the object is an object that pays a fee, the conversion operation may include, for example, a purchase operation. In other embodiments, according to actual needs, other conversion operations such as a registration operation, an act of paying a fee, and other information request operations may be evaluated.

[0041] In some embodiments, the auxiliary content item can further show the explanatory information associated with the object in order to provide more information associated with the object, thereby helping the user to determine whether to execute the conversion operation. For example, if the object is a downloadable application, the explanatory information shown by the auxiliary content item may include the icon of the application, the ranking, the screenshot of the page, the basic explanation, the size of the installation package, the developer information, etc. It should be understood that the explanatory information here can be configured according to the actual scenario.

[0042] In some embodiments, the content management system 120 can determine whether an auxiliary content item should be presented on a particular terminal device 130 based at least on feedback transmission constraints for the conversion operation on the object. The feedback transmission constraints can indicate whether the conversion operation executed on a particular terminal device 130 delays the feedback transmission or blocks the feedback transmission to the content management system 120. The feedback transmission constraints are set due to, for example, the privacy protection strategy of the object provider's platform or the third-party platform on which the conversion operation is executed, or other reasons. The feedback transmission constraints may be related to, for example, the type of the terminal device 120, the platform of the object provider that the terminal device 120 attempts to access, the configuration of a third-party platform, and the like. In one example, if the feedback transmission constraints indicate that the feedback transmission of the conversion operation executed on a particular terminal device 130 is delayed or the feedback is transmitted to the content management system 120, the content management system 120 determines to present the auxiliary content item 242.

[0043] In addition to the feedback transmission constraints, the content management system 120 can determine whether the auxiliary content item 242 should be presented based on other factors, such as a request from the object provider regarding whether the auxiliary content item should be presented. Embodiments of the present invention are not limited in this regard.

[0044] While the auxiliary content item 242 is being presented, the terminal device 130 detects a click operation on the conversion operation portal and provides click feedback on the portal to the content management system 120 (235), and the click feedback indicates whether the portal has been clicked on the terminal device 130. The content management system 120 receives the click feedback (240). Since the presentation of the auxiliary content item 242 is controlled by the content management system 120 and presented within the content providing platform, the content management system 120 can receive and collect such feedback information without being restricted by other platforms.

[0045] In some embodiments, when a click operation on the portal is detected on the terminal device 130, click feedback indicating that the portal has been clicked can be provided to the content management system 120. In some embodiments, if no click operation on the portal is detected even after the presentation of the auxiliary content item 242 is ended by the terminal device 130, or if no click operation on the portal is detected within a certain period of time after the presentation is started, click feedback indicating that the portal has not been clicked can be provided to the content management system 120. In some embodiments, when the auxiliary content item 242 is presented on a plurality of terminal devices 130, the content management system 120 can receive feedback information on whether the portal in the auxiliary content item 142 has been clicked from the plurality of terminal devices 130.

[0046] For a better understanding of the exemplary embodiments, the following will be described with reference to exemplary pages.

[0047] Figures 3A to 3C show schematic views of a presentation interface related to content items according to some embodiments of the present invention. Figure 3A shows a user interface 301 for presenting a content item. In this example, the user interface 301 is shown as a page of an application executable on a terminal device, where a content item (i.e., an advertisement) associated with an object (in this example, "Application XX") is presented. The user interface 301 further includes a pop-up window 310 showing related information about "Application XX" and a download tag 312 for "Application XX".

[0048] The terminal device 130 monitors a click operation on the content item in the user interface 301. In some embodiments, the detection of the click operation can be determined based on the user's click operation or other selection operations (e.g., slide gesture, voice control command, etc.) at one or more positions in the presentation interface of the content item. In some embodiments, a click operation at one or more positions in the user interface 301 can trigger the presentation of auxiliary content items. As shown in Figure 3B, when the user clicks on the pop-up window 310 in the user interface 301, clicks on the download tag 312, or performs a slide gesture in a predetermined direction on the page, it can all be determined that a click operation on the content item has been executed.

[0049] In response to detecting a click operation on the content item, the terminal device 130 presents the user interface 303 shown in FIG. 3C and presents auxiliary content items associated with "Application XX" therein. Specifically, the user interface 303 shows download tags 322, 324, etc., which indicate the download portal of "Application XX". The user interface 303 further shows description information of "Application XX", such as the application icon of "Application XX", the development company, banner information, a simple description, and the size of the installation package.

[0050] When a click operation on the download tags 322, 324 or a click operation on the portal is detected, the terminal device 130 can provide click feedback indicating that the portal has been clicked to the content management system 120. In some examples, in addition to the download tags 322, 324, one or more other positions in the user interface 303 may be configured as portals for conversion operations on the object. This can be configured according to actual needs, and the embodiments of the present invention are not limited in this regard.

[0051] In some embodiments, when the portal indicated by the auxiliary content item 242 is clicked, the terminal device 130 can also access or jump to a platform for performing a conversion operation on the object. FIG. 3D shows a user interface 304 for performing a conversion operation on the object, and the user interface 304 is the download interface of Application XX. In this interface, the user can perform a download operation as needed.

[0052] It should be understood that the user interfaces shown in FIGS. 3A to 3D are merely exemplary, and in reality, various interface designs may exist. Each graphic element in the interface can have different configurations and different visual representations, one or more of which may be omitted or replaced, and one or more other elements may exist. The embodiments of the present invention are not limited in this regard.

[0053] Continuing to refer back to FIG. 2, by providing auxiliary content items, the content management system 120 can collect click feedback on the conversion operation portal from the terminal device 130. Further, based on the click feedback on the content management system 120 portal, predicted conversion information for the content item can be determined (245), and the predicted conversion information indicates the probability that a conversion operation on the object is expected to be executed.

[0054] The presentation of the content item 142 may be referred to as the display or impression of the content item, such as the user interface 301 presented in FIG. 3A. In the evaluation of the content item 142, the number of presentations of the content item 142, that is, the number of displays or impressions, is also counted. The number of presentations of the content item 142 may include one or more presentations on multiple terminal devices and / or one or more presentations on the same terminal device.

[0055] Also, the number of clicks on the presented content item 142, such as a click in FIG. 3B, is also counted. The number of clicks on the content item 142 may include one or more clicks on multiple terminal devices and / or one or more presentations on the same terminal device. In some examples, for a specific content item 142, further, the probability that the click operation of the content item 142 is executed may be determined, which is also called the click through rate (CTR). The click rate may be determined as the ratio of the number of times the click operation of the content item 142 is executed to the number of times the content item 142 is presented. For example, if out of 10,000 presentations of the content item 142, the click operation is detected in 8,000 presentations, the click rate may be determined to be 0.8.

[0056] In conversion evaluation, it is also expected to evaluate the conversion information in which the conversion operation on the content item 142 is executed, which is also called the conversion value rate (CVR). The conversion rate may be determined as the ratio of the number of times the conversion operation of the object is executed to the number of times the click operation of the content item 142 is executed. For example, if out of 10,000 presentations of the content item, the click operation is detected in 8,000 times, and out of 8,000 click operations, a total of 6,000 conversion operations are detected, the conversion rate may be determined to be 0.75.

[0057] In an embodiment of the present invention, a solution is proposed to evaluate the conversion information of the content item based on the click feedback on the portal in the auxiliary content item. In this solution, when a further click on the portal of the conversion operation in the auxiliary content item is detected, there is sufficient information available to consider that the probability that the conversion operation actually occurs is very high. That is, the click operation on the auxiliary content item has a stronger correlation with the conversion operation of the object.

[0058] Content management system 120 can determine the number of times a click operation has been performed on a specific content item 142. The statistics here may be executed over a certain period and multiple terminal devices 130. Content management system 120 can further determine the number of times a portal for a conversion operation is clicked when an auxiliary content item 242 is presented based on click feedback from terminal device 130, which may also be referred to as the intermediate click count herein. Content management system 120 can determine the number of times a portal for a conversion operation indicated by a specific auxiliary content item 242 is clicked over a certain period and multiple terminal devices 130.

[0059] In some embodiments, considering that the click operation on the auxiliary content item and the conversion operation of the object have a stronger correlation, content management system 130 may approximate the number of clicks on the portal of the conversion operation as the number of times the conversion operation is executed, that is, the number of times the conversion operation on the object is "expected" to be executed. By determining the predicted conversion information for content item 142 based on this, content management system 130 can indicate the probability that a conversion operation is expected to occur when content item 142 is clicked. For example, the proportion of the number of clicks on the portal of the conversion operation to the number of clicks on content item 142 may be determined as the predicted conversion information. For example, out of 10,000 presentations of a content item, 8,000 click operations are detected, and out of the 8,000 click operations, 7,000 click operations on the portal indicated by the auxiliary content item are detected, the predicted conversion information may be determined to indicate a conversion rate of 7 / 8.

[0060] The predicted conversion information is used to estimate the presentation effect of content item 142. If the user does not perform a download operation after jumping to the download interface in Figure 3D, or if the interface jump is blocked, etc., the click operation on the auxiliary content item does not accurately mean that the conversion operation on the object has actually occurred. Therefore, there is a certain error between the predicted conversion information for content item 142 and the true conversion information of content item 142. This error is called discount information or discount rate. The discount information indicates the probability that the conversion operation is not actually executed when the predicted conversion information indicates that the conversion operation on the object is expected to be executed. That is, the discount information is determined by the statistical data of the part of the conversion operation actually executed after clicking the portal on the auxiliary content item. The discount rate (denoted as drop_ratIo) is determined by the following formula.

Number

[0061] In some embodiments, the content management system 130 can utilize the collected predicted conversion information to perform more subsequent strategies and calculate other metric indicators of the content item.

[0062] FIG. 4 shows a block diagram of an exemplary configuration of a content management system 120 according to some embodiments of the present invention. As shown in the drawing, the content management system 120 may at least include an information collection subsystem 410, a model training subsystem 420, and a model application subsystem 430.

[0063] The information collection subsystem 410 may be configured to determine prediction conversion information for one or more content items 142 by the process described with respect to the signaling flow 200. Since these content items 142 have auxiliary content items 242, as described above, the prediction conversion information is determined based on click operations detected in the auxiliary content items 242.

[0064] In some embodiments, the model training subsystem 420 is configured to train a conversion prediction model 422 based at least on the collected prediction conversion information. The conversion prediction model 422 is configured to predict conversion information for a content item associated with a predetermined object, such as the probability that a conversion operation for the predetermined object will be performed when the content item is presented or clicked.

[0065] The trained conversion prediction model 422 is provided to the model application subsystem 430, and the model application subsystem 430 is configured to predict prediction conversion information 432 for a specific content item 412 by using the trained conversion prediction model 422 to indicate the probability that a conversion operation for the object associated with the content item 142 is expected to be performed.

[0066] In some embodiments, the model training subsystem 420 is further configured to train at least a discount prediction model 424. The discount prediction model 424 is configured to determine discount information for content items. The training data for the discount prediction model 424 includes at least the true total number of times a conversion action has been actually performed on the corresponding object and the number of click actions detected on the auxiliary content item.

[0067] The trained discount prediction model 424 can be provided to the model application subsystem 430, and the model application subsystem 430 is configured to use the trained discount prediction model 424 to predict discount information 434 for a specific content item 412 in order to indicate the probability that the conversion action will not be actually performed when the predicted conversion information indicates that a conversion action is expected to be performed on the related object.

[0068] In some embodiments, the trained conversion prediction model 422 is used to determine predicted conversion information 432 (denoted as pcvr) for a specific content item 142, and when a click action on the content item 142 triggers the presentation of the auxiliary content item 242, the trained discount prediction model 424 can also be used to determine discount information 434 for the content item 142, such as drop_ratIo. By determining updated conversion information for the content item 142 based on the predicted conversion information 432 and the discount information 434, the probability that a conversion action on the object will be actually performed can be indicated, which may be expressed as cvr = pcvr * (1 - drop_ratIo). Among the updated conversion information, the statistical data of the part of the conversion action that has not been actually performed is removed.

[0069] In some embodiments, based on the updated conversion information, the model application subsystem 430 can further determine other information associated with the content item 142, such as revenue-related information 436. Revenue-related information 426 is typically a content item metric of interest to the object provider. Revenue-related information 436 is determined based at least on the conversion information for the content item 142 and the conversion unit price information for the content item 142.

[0070] In some examples, the revenue-related information 436 can indicate the revenue obtained from a predetermined number of presentations of the content item, for example, the advertising revenue obtained per thousand presentations (i.e., effective cost per mile, ECPM). The revenue-related information 436 may have other estimation criteria, and for ease of explanation, ECPM is used as an example for explanation below. ECPM can be determined based on the following formula.

Number

[0071] Revenue-related information 436 such as ECPM can be used to determine the unit price strategy for content items. For example, the unit price strategy may include the lowest cost unit price and the cost cap unit price. The lowest cost unit price method is the plan budget set by the object provider, which realizes the lowest cost unit price goods while consuming all the budget as much as possible over a certain period (for example, one day or until the end of the advertising plan schedule). The cost cap unit price method ensures that the actual cost of the content item distributed by the object provider is as close as possible to its unit price (bId). The cost cap unit price method needs to consider revenue-related information to accurately determine the revenue related to the content item. Of course, the revenue-related information 436 may also be used in each scenario such as other evaluation methods for content items and the execution of other strategies.

[0072] First, the configuration of the conversion prediction model 422 and its training process will be described below, and then the training process of the discount prediction model 424 will be described.

[0073] The conversion prediction model 422 can be constructed based on any suitable machine learning model or neural network model. In some embodiments, a particular content item 142 may be immediately reversed and may not provide an auxiliary content item for a particular content item due to conversion constraints or other reasons. In some embodiments, the conversion prediction model 422 is expected to be trained to be applicable to predicting the conversion information of such content items 142 and predicting the conversion information for which it is necessary to provide an auxiliary content item. In such cases, the training data for the conversion prediction model 422 includes the predicted conversion information determined based on the provision of the auxiliary content item and the conversion information determined based on the non - provision of the auxiliary content item. The latter conversion information indicates the probability that the conversion operation of the corresponding object is executed and may be obtained by directly feedback transmission from a platform other than the content - providing platform (the platform that executes the conversion operation). The platform that executes the conversion operation may be, for example, a platform managed by an object provider or a third - party platform.

[0074] In some embodiments, to enable the conversion prediction model 422 to provide more accurate predictions for various types of content items, the conversion prediction model 422 may be constructed to determine predicted conversion information using different output portions for various types of content items. FIG. 5 shows a block diagram of an exemplary configuration of the conversion prediction model 422 of such an embodiment. As shown in FIG. 5, the conversion prediction model 422 includes a general portion 510, a first output portion 512, and a second output portion 514. The general portion 510 functions as a shared substrate portion of the first output portion 512 and the second output portion 514, receives model inputs, processes the model inputs, and obtains intermediate feature information. When the model input is related to a content item that provides an auxiliary content item, the intermediate feature information is provided to the first output portion 512, and the predicted conversion information is determined by that portion. When the model input is related to a content item that does not require an auxiliary content item to be provided, the intermediate feature information is provided to the second output portion 514, and the predicted conversion information is determined by that portion.

[0075] The model input may include information related to the content item itself, information related to the object of the content item, and other relevant information for predicting conversion information, and embodiments of the present invention are not limited thereto.

[0076] In the model training stage, the general part 510 can be trained by using the predicted conversion information related to the content items that provide auxiliary content items and the predicted conversion information related to the content items that do not need to provide auxiliary content items. That is, the predicted conversion information related to the two types of content items can both affect the update of the parameters of the general part 510. For the first output part 512, it is trained by using the predicted conversion information related to the content items that provide auxiliary content items, and for the second output part 514, it is trained by using the predicted conversion information related to the content items that do not need to provide auxiliary content items. In this way, each of the first output part 512 and the second output part 514 can be specialized in predicting the conversion information of the corresponding type of content item.

[0077] An exemplary configuration of the conversion prediction model 422 has been described with reference to FIG. 5. In other embodiments, instead of designing multiple output parts, the conversion prediction model 422 can be jointly trained by using the predicted conversion information corresponding to the two types of content items. According to actual needs, in some embodiments, for the two types of content items, two conversion prediction models can also be designed respectively and used for predicting the conversion information.

[0078] The training data for the discount prediction model 424 includes at least the true total number of conversion operations actually performed on the corresponding object. However, the true total number of conversion operations actually performed is often provided to the content management system 120 by another platform after a certain time delay. For example, to prevent tracking, the feedback transmission time for each conversion operation may be random. However, generally, as more time elapses, it is possible that all conversion operations that occur during this period have been feedback-transmitted to the content management system 120. The inventor has also found that the overall delay time of feedback transmission follows a certain distribution. Therefore, in some embodiments, an improved training method for training the discount prediction model 424 is proposed.

[0079] FIG. 6 shows a block diagram of an exemplary architecture 600 for training the discount prediction model 424 according to some embodiments of the present invention. The content items targeted for training the discount prediction model 424 have associated auxiliary content items.

[0080] In the embodiment of FIG. 6, the discount prediction model 424 is trained considering one or more content items x(1), x(2), x(3),..., x(m). For each content item, assume that the true total number 602 (shown as ad_skan) of conversion operations actually performed on the object related to the content item over a certain time range can be determined based on one or more conversion feedback transmissions. Also, a plurality of numbers 620 of click operations performed on the portal indicated by the auxiliary content item 242 at a plurality of time points (shown as 1, 2, 3,..., n) within the time range can be obtained, which are shown as mId_clIck(1), mId_clIck(2), mId_clIck(3)......mId_clIck(n).

[0081] In architecture 600, time model 610 is composed of a plurality of conversion feedback transmission probabilities 612 corresponding to a plurality of time points, shown as tIme_prob(1), tIme_prob(2), tIme_prob(3)……tIme_prob(n). Each conversion feedback transmission probability indicates the probability that the number of times the conversion operation on the object at the corresponding time point has been feedback transmitted at time point n. The input of time model 610 is time difference information 605 for indicating each specific time point, shown as tIme_dIff(1), tIme_dIff(2), tIme_dIff(3)……tIme_dIff(n).

[0082] Determine the total number of times 632 that the conversion operation on the object within the time range is expected to be executed based on the plurality of times and the plurality of conversion feedback transmission probabilities shown in mId_clIck_aggr. For example, mId_clIck(1), mId_clIck(2), mId_clIck(3)……mId_clIck(n) and tIme_prob(1), tIme_prob(2), tIme_prob(3)……tIme_prob(n) can be multiplied element by element and then summed to determine mId_clIck_aggr.

[0083] In each iteration of training, the discount prediction model 424 uses the current model parameter values to determine the discount information drop_ratIo’ for the corresponding content item (for example, a specific content item among x(1), x(2), x(3),…, x(m)). Based on mId_clIck_aggr and (1-drop_ratIo’), the predicted total number of times 660 y’ that the conversion operation on the related object within the time range from time point 1 to n is actually executed can be determined, that is, y’ = mId_clIck_aggr * (1-drop_ratIo’).

[0084] During training, the model update module 660 within the model training subsystem 420 can determine the difference between the predicted total number 660y' and the true total number 602ad_skan, and update the discounted prediction model 424 based on this difference. For example, the parameter values of the discounted prediction model 424 can be updated. In some embodiments, the model update module 660 constructs a loss function based on the difference between the predicted total number 660y' and the true total number 602ad_skan, and based on the loss function, can determine the update of the discounted prediction model 424 by using an appropriate model training algorithm. Embodiments of the present invention are not limited in terms of the model training algorithm.

[0085] For each of the content items x(1), x(2), x(3), …, x(m), the discounted prediction model 424 can be updated by determining the difference between the predicted total number 660y' and the true total number 602ad_skan through the steps described above. The discounted prediction model 424 can be updated to continuously reduce or minimize the difference between the predicted total number 660y' and the true total number 602ad_skan until the model training goal is reached.

[0086] FIG. 7 shows a flowchart of a process 700 for evaluating conversions executed in the content management system 120 according to some embodiments of the present invention.

[0087] In block 710, the content management system 120 presents a first content item associated with a first object on a terminal device corresponding to the content delivery platform. In block 720, the content management system 120 determines whether a click operation on the first content item is detected. In block 730, in response to determining that a click operation on the first content item is detected, the content management system 120 presents an auxiliary content item associated with the first object on the terminal device, and the auxiliary content item at least indicates a portal for performing a conversion operation on the first object. In block 740, based on click feedback on the portal, first predicted conversion information for the first content item is determined, the click feedback indicates whether the portal is clicked on the terminal device, and the first predicted conversion information indicates the probability that a conversion operation on the first object is expected to be performed.

[0088] In some embodiments, the conversion operation includes a download operation on the first object.

[0089] In some embodiments, the auxiliary content item further indicates explanatory information associated with the first object.

[0090] In some embodiments, presenting the auxiliary content item includes determining whether the auxiliary content item should be presented on the terminal device based at least on feedback transmission constraints of a conversion operation on the first object, and selectively presenting the auxiliary content item based on the determination.

[0091] In some embodiments, process 700 further includes training a conversion prediction model based at least on the first predicted conversion information, and the conversion prediction model is trained to predict conversion information for a content item associated with a given object.

[0092] In some embodiments, training the conversion prediction model includes obtaining second predicted conversion information for a second content item, where the second content item is associated with a second object, and the second predicted conversion information is determined based on the probability that a conversion action for the second object is performed, obtained from a platform other than the content providing platform, and training a general part of the conversion prediction model based on the first predicted conversion information and the second predicted conversion information, training a first output part of the conversion prediction model based on an intermediate output of the general part and the first predicted conversion information, and training a second output part of the conversion prediction model based on the intermediate output of the general part and the second predicted conversion information.

[0093] In some embodiments, process 700 further includes predicting third predicted conversion information for a third content item using the trained conversion prediction model, where the third content item is associated with a third object, and the third predicted conversion information indicates the probability that a conversion action for the third object is expected to be performed.

[0094] In some embodiments, when a click action on the third content item is detected, another auxiliary content item is presented. In some embodiments, process 700 further includes obtaining discount information for the third content item, where the discount information indicates the probability that the conversion action is not actually performed when the third predicted conversion information indicates that a conversion action for the third object is expected to be performed, and determining updated conversion information for the third content item based on the third predicted conversion information and the discount information, where the updated conversion information indicates the probability that a conversion action for the third object is actually performed.

[0095] In some embodiments, process 700 is to determine revenue-related information for a third content item based on updated conversion information and conversion unit price information for the third content item, where the conversion unit price information further includes indicating the unit price of one conversion operation for a third object.

[0096] In some embodiments, obtaining discount information includes obtaining a trained discount prediction model and using the discount prediction model to determine discount information for a third content item.

[0097] In some embodiments, when a fourth content item associated with a fourth object is presented, process 700 further includes determining the total number of times that a conversion operation for the fourth object is expected to be executed within a time range, using the discount prediction model to be trained to determine the expected discount information for the fourth content item, determining the predicted total number of times that the conversion operation for the fourth object is actually executed within the time range based on the expected discount information and the total number of times, and updating the discount prediction model based on the difference between the predicted total number of times and the true total number of times that the conversion operation for the fourth object is actually executed within the time range, thereby training the discount prediction model.

[0098] In some embodiments, when a click operation on a fourth content item is detected, another auxiliary content item is presented. In some embodiments, determining the total number of times a conversion operation on a fourth object is expected to be performed within a time range includes obtaining the number of times a click operation on a portal indicated by another auxiliary content item is performed at multiple time points within the time range, and determining a plurality of conversion feedback transmission probabilities corresponding to the multiple time points, where the conversion feedback transmission probability indicates the probability that the number of times a conversion operation on the fourth object has been feedback transmitted at the latest time point among the multiple time points, and determining the total number of times a conversion operation on the fourth object is expected to be performed within the time range based on the plurality of times and the plurality of conversion feedback transmission probabilities.

[0099] FIG. 8 shows a flowchart of a process 800 for evaluating conversions executed in a terminal device 140 according to some embodiments of the present invention.

[0100] In block 810, the terminal device 140 presents a first content item associated with a first object. In block 820, the terminal device 140 determines whether a click operation on the first content item is detected. In response to detecting a click operation on the first content item, in block 830, the terminal device 140 presents an auxiliary content item associated with the first object, and the auxiliary content item at least indicates a portal for performing a conversion operation on the first object. In block 840, the terminal device 140 provides click feedback on the portal to a content management system, and the click feedback indicates whether the portal has been clicked on the terminal device.

[0101] FIG. 9 shows a block diagram of a schematic configuration of an apparatus 800 for evaluating conversions according to some embodiments of the present invention. The apparatus 900 can be implemented as a content management system 120 or can be included in the content management system 120. Each module / component of the apparatus 900 can be implemented by hardware, software, firmware, or any combination thereof.

[0102] As shown in the drawing, the apparatus 900 includes a first presentation control module 910 for presenting a first content item associated with a first object on a terminal device corresponding to a content delivery platform. The apparatus 900 further includes a second presentation control module 920 for presenting an auxiliary content item associated with the first object on the terminal device in response to determining that a click operation on the first content item has been detected, where the auxiliary content item at least shows a portal for performing a conversion operation on the first object. The apparatus 900 further includes a conversion determination module 930 for determining first predicted conversion information for the first content item based on click feedback on the portal, where the click feedback indicates whether the portal has been clicked on the terminal device, and the first predicted conversion information indicates the probability that a conversion operation on the first object is expected to be performed.

[0103] In some embodiments, the conversion operation includes a download operation on the first object.

[0104] In some embodiments, the auxiliary content item further shows explanatory information associated with the first object.

[0105] In some embodiments, the second presentation control module 920 includes a determination module for determining whether an auxiliary content item should be presented on the terminal device based on at least the feedback transmission constraint of the conversion operation for the first object, and a selective presentation module for selectively presenting the auxiliary content item based on the determination.

[0106] In some embodiments, the apparatus 900 is a first training module for training a conversion prediction model based on at least the first predicted conversion information, and the conversion prediction model is further provided with the first training module trained to predict conversion information for a content item associated with a predetermined object.

[0107] In some embodiments, the first training model is an information acquisition module for acquiring second predicted conversion information for a second content item, where the second content item is associated with a second object, and the second predicted conversion information is determined based on the probability that a conversion operation for the second object is executed, obtained from a platform other than the content providing platform; a general part training module for training the general part of the conversion prediction model based on the first predicted conversion information and the second predicted conversion information; a first output part training module for training the first output part of the conversion prediction model based on the intermediate output of the general part and the first predicted conversion information; and a second output part training module for training the second output part of the conversion prediction model based on the intermediate output of the general part and the second predicted conversion information.

[0108] In some embodiments, the apparatus 900 is a first model application module for predicting third predicted conversion information for a third content item by using a trained conversion prediction model, where the third content item is associated with a third object, and the third predicted conversion information further includes a first model application module indicating the probability that a conversion operation for the third object is expected to be executed.

[0109] In some embodiments, when a click operation on the third content item is detected, another auxiliary content item is presented. In some embodiments, the apparatus 900 further includes a discount acquisition module for acquiring discount information for the third content item, where the discount information indicates the probability that the conversion operation is not actually executed when the third predicted conversion information indicates that the conversion operation for the third object is expected to be executed, and a conversion update module for determining updated conversion information for the third content item based on the third predicted conversion information and the discount information, where the updated conversion information indicates the probability that the conversion operation for the third object is actually executed.

[0110] In some embodiments, the apparatus 900 further includes a revenue determination module for determining revenue-related information for the third content item based on the updated conversion information and conversion unit price information for the third content item, where the conversion unit price information indicates the unit price of one conversion operation for the third object.

[0111] In some embodiments, the discount acquisition module includes a model acquisition module for acquiring a trained discount prediction model, and a second model application module for determining discount information for the third content item by using the discount prediction model.

[0112] In some embodiments, when presenting a fourth content item associated with a fourth object, the apparatus 900 determines the total number of times a conversion operation on the fourth object is expected to be executed within a time range, determines predicted discount information for the fourth content item using a discount prediction model to be trained, determines a predicted total number of times a conversion operation on the fourth object is actually executed within the time range based on the predicted discount information and the total number of times, and updates the discount prediction model based on the difference between the predicted total number of times and the true total number of times a conversion operation on the fourth object is actually executed within the time range. Further provided is a second model training module for training the discount prediction model by doing so.

[0113] In some embodiments, when a click operation on the fourth content item is detected, another auxiliary content item is presented. In some embodiments, the second model training module obtains the number of times a click operation is executed on a portal indicated by another auxiliary content item at multiple time points within the time range, and determines a plurality of conversion feedback transmission probabilities corresponding to the multiple time points, where the conversion feedback transmission probability indicates the probability that the number of times a conversion operation on the fourth object is executed at the corresponding time point has already been feedback transmitted at the latest time point among the multiple time points, and determines the total number of times a conversion operation on the fourth object is expected to be executed within the time range based on the number of times and the plurality of conversion feedback transmission probabilities. Thus, the total number of times a conversion operation on the fourth object is expected to be executed within one time range is determined.

[0114] FIG. 10 shows a block diagram of an apparatus for evaluating conversion according to some other embodiments of the present invention. The apparatus 1000 can be implemented as or included in the terminal device 140. Each module / component of the apparatus 1000 can be implemented by hardware, software, firmware, or any combination thereof.

[0115] As shown in the drawings, the apparatus 1000 includes a first presentation module 1010 that presents a first content item associated with a first object in a terminal device corresponding to a content providing platform, and a second presentation module 1020 that presents an auxiliary content item associated with the first object in response to detecting a click operation on the first content item, where the auxiliary content item at least shows a portal for performing a conversion operation on the first object. The apparatus 1000 further includes a feedback providing module 1030 that provides click feedback on the portal to a content management system, where the click feedback indicates whether the portal has been clicked on the terminal device.

[0116] FIG. 11 shows a block diagram of an electronic device 1100 capable of implementing one or more embodiments of the present invention. It should be understood that the electronic device 1100 shown in FIG. 11 is merely exemplary and should not limit the functions and scope of the embodiments described herein. The electronic device 1100 shown in FIG. 11 can be used to implement the terminal device 140 or the content management system 120 of FIG. 1.

[0117] As shown in FIG. 11, the electronic device 1100 is in the form of a general-purpose electronic device. The components of the electronic device 1100 may include, but are not limited to, one or more processors or processing units 1110, a memory 1120, a storage device 1130, one or more communication units 1140, one or more input devices 1150, and one or more output devices 1160. The processing unit 1110 may be an actual or virtual processor and can execute various processes based on programs stored in the memory 1120. In a multi-processor system, the parallel processing ability of the electronic device 1100 is improved by multiple processing units executing computer-executable instructions in parallel.

[0118] The electronic device 1100 typically includes a plurality of computer storage media. Such media may be any accessible media that can be accessed by the electronic device 1100, including, but not limited to, volatile and non-volatile media, removable and non-removable media. The memory 1120 may be volatile memory (e.g., registers, caches, random access memory (RAM)), non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or a specific combination thereof. The storage device 1130 may be removable or non-removable media, and may include machine-readable media such as a flash memory drive, a magnetic disk, or any other media, and can be used to store information and / or data (e.g., training data for training), and may be accessible within the electronic device 1100.

[0119] The electronic device 1100 can further include another removable / non-removable, volatile / non-volatile storage medium. Although not shown in FIG. 11, a magnetic disk drive for reading from or writing to a removable, non-volatile magnetic disk (e.g., a "floppy disk"), and an optical disk drive for reading from or writing to a removable, non-volatile optical disk can be provided. In these cases, each drive may be connected to a path (not shown) by one or more data media interfaces. The memory 1120 may include a computer program product 1125 having one or more program modules, and these program modules are configured to execute various methods or operations of various embodiments of the present invention.

[0120] The communication unit 1140 implements communication with other computing devices via a communication medium. Additionally, the functions of the components of the electronic device 1100 may be implemented as a single computing cluster or multiple computing machines, which can communicate via a communication connection. Thus, the electronic device 1100 can operate in a networked environment using the logical connection of one or more other servers, network personal computers (PCs), or other network nodes.

[0121] The input device 1150 may be one or more input devices such as a mouse, keyboard, trackball, etc. The output device 1160 may be one or more output devices such as a display, speaker, printer, etc. The electronic device 1100 can further communicate, as needed, with one or more external devices (not shown) such as a storage device, display device, etc. via the communication unit 1140, communicate with one or more devices that enable a user to interact with the electronic device 1100, or communicate with any device (e.g., a netbook card, modem, etc.) that enables the electronic device 1100 to communicate with one or more other computing devices. Such communication can be performed via an input / output (I / O) interface (not shown).

[0122] According to an exemplary implementation of the present invention, a computer-readable storage medium storing one or more computer instructions is provided, and the one or more computer instructions are executed by a processor to implement the above method. According to an exemplary implementation of the present invention, a computer program product is further provided, and the computer program product is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions executed by a processor to implement the above method.

[0123] Here, each aspect of the present invention has been described with reference to the flowcharts and / or block diagrams of the method, apparatus (system), and computer program product implemented by the present invention. It should be understood that each box in the flowcharts and / or block diagrams and combinations of boxes in the flowcharts and / or block diagrams can all be implemented by computer-readable program instructions.

[0124] These computer-readable program instructions are provided to the processing unit of a general-purpose computer, special-purpose computer, or other programmable data processing device to generate a machine, so that when these instructions are executed by the processing unit of the computer or other programmable data processing device, an apparatus for implementing the functions / operations specified in one or more boxes in the flowchart and / or block diagram can be generated. These computer-readable program instructions may be stored in a computer-readable storage medium, and by operating the computer, programmable data processing device, and / or other devices in a specific form, the computer-readable medium storing the instructions constitutes a manufactured product including instructions for implementing each aspect of the functions / operations specified in one or more boxes in the flowchart and / or block diagram.

[0125] By loading the computer-readable program instructions into a computer, other programmable data processing device, or other device, a series of operation steps are executed on the computer, other programmable data processing device, or other device to generate a process implemented by the computer, whereby the instructions executed on the computer, other programmable data processing device, or other device implement the functions / operations specified in one or more boxes in the flowchart and / or block diagram.

[0126] The flowcharts and block diagrams in the drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a program fragment, or a part of an instruction, and the module, program fragment, or part of an instruction can include one or more executable instructions for implementing the specified logical function. In some implementations as an alternative, the functions represented in the boxes may occur in a different order than that shown in the drawings. For example, two consecutive boxes may actually be executed substantially in parallel, or depending on the functions involved, may be executed in the reverse order. It should also be noted that each box in the block diagram and / or flowchart, and combinations of boxes in the block diagram and / or flowchart, may be implemented by a special-purpose hardware-based system that performs the specified functions or operations, or may be implemented by a combination of special-purpose hardware and computer instructions.

[0127] As described above for each implementation of the present invention, the above description is exemplary, not exhaustive, and not limited to the disclosed implementations. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described implementations. The selection of terms used herein is intended to best interpret the principles of each implementation, actual application, or improvement of the technology in the market, or to enable those skilled in the art to understand each implementation form disclosed herein.

Claims

1. presenting, on a terminal device corresponding to a content providing platform, a first content item associated with a first object; in response to determining that a click operation on the first content item has been detected, presenting, on the terminal device, an auxiliary content item associated with the first object, the auxiliary content item at least indicating a portal for performing a conversion operation on the first object; determining first predicted conversion information for the first content item based on click feedback on the portal, the click feedback indicating whether the portal has been clicked on the terminal device, and the first predicted conversion information indicating a probability that the conversion operation on the first object is expected to be performed; A method for evaluating conversion.

2. The conversion operation includes a download operation on the first object. The method according to claim 1.

3. The auxiliary content item further indicates explanatory information associated with the first object. The method according to claim 1.

4. Presenting the auxiliary content item includes: determining whether the auxiliary content item should be presented on the terminal device based at least on feedback transmission constraints of a conversion operation on the first object; selectively presenting the auxiliary content item based on the determination. The method according to claim 1.

5. training a conversion prediction model based at least on the first predicted conversion information, the conversion prediction model further including being trained to predict conversion information for a content item associated with a predetermined object. The method according to claim 1.

6. Training the conversion prediction model includes: obtaining second predicted conversion information for a second content item, the second content item being associated with a second object, and the second predicted conversion information being determined based on a probability that the conversion operation on the second object is performed, obtained from a platform other than the content providing platform. Training the general part of the conversion prediction model based on the first predicted conversion information and the second predicted conversion information; Training the first output part of the conversion prediction model based on the intermediate output of the general part and the first predicted conversion information; Training the second output part of the conversion prediction model based on the intermediate output of the general part and the second predicted conversion information, including: The method according to claim 5.

7. Predicting third predicted conversion information for a third content item by using the trained conversion prediction model, where the third content item is associated with a third object, and the third predicted conversion information further includes indicating the probability that the conversion operation for the third object is expected to be executed; The method according to claim 5.

8. When a click operation on the third content item is detected, presenting another auxiliary content item; The method for evaluating the conversion is: Obtaining discount information for the third content item, where the discount information indicates the probability that the conversion operation is not actually executed when the third predicted conversion information indicates that the conversion operation for the third object is expected to be executed; Determining updated conversion information for the third content item based on the third predicted conversion information and the discount information, where the updated conversion information indicates the probability that the conversion operation for the third object is actually executed, further including: The method according to claim 7.

9. Determining revenue-related information for the third content item based on the updated conversion information and conversion unit price information for the third content item, where the conversion unit price information indicates the unit price of one conversion operation for the third object, further including: The method according to claim 7.

10. Obtaining the discount information includes: Obtaining a trained discount prediction model; Determining the discount information for the third content item by using the discount prediction model, including: The method according to claim 9.

11. When a fourth content item associated with a fourth object is presented, determining a total number of times that the conversion operation on the fourth object is expected to be executed within one time range; Determining predicted discount information for the fourth content item by using a discount prediction model to be trained; Based on the predicted discount information and the total number of times, determining a predicted total number of times that the conversion operation on the fourth object is actually executed within the time range; Further including training the discount prediction model by updating the discount prediction model based on a difference between the predicted total number of times and an actual total number of times that the conversion operation on the fourth object is actually executed within the time range. The method according to claim 10.

12. When a click operation on the fourth content item is detected, presenting another auxiliary content item. Determining the total number of times that the conversion operation on the fourth object is expected to be executed within one time range is: Obtaining a plurality of times of click operations on a portal indicated by the another auxiliary content item at a plurality of time points within the time range; Determining a plurality of conversion feedback transmission probabilities corresponding to the plurality of time points, where the conversion feedback transmission probability indicates a probability that the number of times the conversion operation on the fourth object is executed at the corresponding time point has already been feedback transmitted at the latest time point among the plurality of time points; Including determining the total number of times that the conversion operation on the fourth object is expected to be executed within the time range based on the plurality of times and the plurality of conversion feedback transmission probabilities. The method according to claim 11.

13. In a terminal device corresponding to a content providing platform, presenting a first content item associated with a first object; In response to detecting a click operation on the first content item, presenting an auxiliary content item associated with the first object, where the auxiliary content item at least indicates a portal for executing a conversion operation on the first object. To provide click feedback on the portal to the content management system, the click feedback including indicating whether the portal has been clicked on the terminal device A method for evaluating conversion Claim 14 A first presentation control module for presenting a first content item associated with a first object on a terminal device corresponding to a content providing platform A second presentation control module for presenting an auxiliary content item associated with the first object on the terminal device in response to determining that a click operation on the first content item has been detected, the auxiliary content item including at least a portal for performing a conversion operation on the first object A conversion determination module for determining first predicted conversion information for the first content item based on click feedback on the portal, the click feedback indicating whether the portal has been clicked on the terminal device, and the first predicted conversion information indicating a probability that the conversion operation on the first object is expected to be performed An apparatus for evaluating conversion Claim 15 The second presentation control module A determination module for determining whether the auxiliary content item should be presented on the terminal device based on at least feedback transmission constraints of a conversion operation on the first object A selective presentation module for selectively presenting the auxiliary content item based on the determination The apparatus according to claim 14 Claim 16 A first training module for training a conversion prediction model based on at least the first predicted conversion information, the conversion prediction model being further provided with the first training module trained to predict conversion information for a content item associated with a predetermined object The apparatus according to claim 14 Claim 17 A first model application module for predicting third predicted conversion information for a third content item by using the trained conversion prediction model, wherein the third content item is associated with a third object, and the third predicted conversion information indicates a probability that the conversion operation for the third object is expected to be executed, and the first model application module further comprises: The apparatus according to claim 16.

18. In a terminal device corresponding to a content providing platform, a first presentation module for presenting a first content item associated with a first object, and A second presentation module for presenting an auxiliary content item associated with the first object in response to detecting a click operation on the first content item, wherein the auxiliary content item at least indicates a portal for executing a conversion operation on the first object, and the second presentation module; A feedback providing module for providing click feedback on the portal to a content management system, wherein the click feedback indicates whether the portal has been clicked on the terminal device, and the feedback providing module; An apparatus for evaluating conversion.

19. An electronic device, comprising At least one processing unit, and Coupled to the at least one processing unit and used for storing instructions to be executed by the at least one processing unit, and when the instructions are executed by the at least one processing unit, causing the electronic device to execute the method according to any one of claims 1 to 12 or the method according to claim 13, and at least one memory; An electronic device.

20. A computer program is stored, and when the computer program is executed by a processor, the method according to any one of claims 1 to 12 or the method according to claim 13 is implemented. A computer-readable storage medium.

Images