Method, device, apparatus and storage medium for conversion evaluation

Abstract

According to an embodiment of the present disclosure, a method, apparatus, device and storage medium for conversion evaluation are provided. The method includes causing a first content item related to a first object to be presented at a terminal device corresponding to a content providing platform; in response to detecting a click behavior for the first content item, causing an auxiliary content item related to the first object to be presented at the terminal device, the auxiliary content item at least indicating an entry for performing a conversion behavior for the first object; and determining first expected conversion information for the first content item based on click feedback for the entry, the click feedback indicating whether the entry is clicked at the terminal device, and the first expected conversion information indicating the probability that the conversion behavior for the first object is expected to be performed. According to this scheme, the expected conversion information for the content item can be effectively estimated without relying on conversion feedback information outside the content providing platform.

Description

Technical Field

[0001] Example embodiments of the present disclosure generally relate to the field of computer technology, and more particularly, to methods, devices, apparatuses, and computer-readable storage media for conversion evaluation. Background Art

[0002] The Internet provides access to a wide variety of content. For example, various images, audio, video, web pages, etc. can be accessed through the Internet. In addition, the accessible content also includes specific content items related to various objects, such as advertisements. Such content items are usually provided by the object provider in consultation with the content provider, and the object provider may also pay the content provider based on the access to the content items. Therefore, it is usually necessary to measure indicators related to the presentation of content items, including the conversion of content items, where "conversion" refers to downloading, registration, purchase or other information demand behaviors that occur under the influence of the presented content items. Summary of the invention

[0003] In a first aspect of the present disclosure, a method for conversion evaluation is provided. The method includes: causing a first content item related to a first object to be presented at a terminal device corresponding to a content providing platform; in response to detection of a click behavior on the first content item, causing an auxiliary content item related to the first object to be presented at the terminal device, the auxiliary content item at least indicating an entry for performing a conversion behavior on the first object; and determining first expected conversion information for the first content item based on click feedback on the entry, the click feedback indicating whether the entry is clicked at the terminal device, and the first expected conversion information indicating a probability that the conversion behavior on the first object is expected to be performed.

[0004] In a second aspect of the present disclosure, a method for conversion evaluation is provided. The method includes: presenting a first content item related to a first object at a terminal device corresponding to a content providing platform; in response to detecting a click behavior on the first content item, presenting an auxiliary content item related to the first object, the auxiliary content item at least indicating an entry for performing a conversion behavior on the first object; and providing a content management system with click feedback on the entry, the click feedback indicating whether the entry is clicked at the terminal device.

[0005] In a third aspect of the present disclosure, a device for conversion evaluation is provided. The device includes: a first presentation control module, configured to make a first content item related to a first object be presented at a terminal device corresponding to a content providing platform; a second presentation control module, configured to make an auxiliary content item related to the first object be presented at the terminal device in response to the detection of a click behavior for the first content item, the auxiliary content item at least indicating an entry for performing a conversion behavior for the first object; and a conversion determination module, configured to determine first expected conversion information for the first content item based on click feedback for the entry, the click feedback indicating whether the entry is clicked at the terminal device, and the first expected conversion information indicating the probability that the conversion behavior for the first object is expected to be performed.

[0006] In a fourth aspect of the present disclosure, a device for conversion evaluation is provided. The device includes: a first presentation module configured to present a first content item related to a first object at a terminal device corresponding to a content providing platform; a second presentation module configured to present an auxiliary content item related to the first object in response to detecting a click behavior on the first content item, the auxiliary content item at least indicating an entry for performing a conversion behavior on the first object; and a feedback providing module configured to provide a content management system with click feedback on the entry, the click feedback indicating whether the entry is clicked at the terminal device.

[0007] In a fifth aspect of the present disclosure, an electronic device is provided. The device includes at least one processing unit; and at least one memory, the at least one memory is coupled to the at least one processing unit and stores instructions for execution by the at least one processing unit. When the instructions are executed by the at least one processing unit, the device executes the method of the first aspect or the second aspect.

[0008] In a sixth aspect of the present disclosure, a computer-readable storage medium is provided, wherein a computer program is stored on the medium, and when the computer program is executed by a processor, the method of the first aspect or the second aspect is implemented.

[0009] It should be understood that the contents described in the summary of the present invention are not intended to limit the key features or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, wherein:

[0011] Figure 1A schematic diagram showing an example environment in which embodiments of the present disclosure can be implemented;

[0012] Figure 2 A flow chart showing a signaling flow for conversion evaluation according to some embodiments of the present disclosure;

[0013] FIG. 3A to FIG. 3D A schematic diagram showing a presentation interface related to a content item according to some embodiments of the present disclosure;

[0014] Figure 4 A block diagram showing an example structure of a content management system according to some embodiments of the present disclosure;

[0015] Figure 5 A block diagram showing an example structure of a conversion prediction model according to some embodiments of the present disclosure;

[0016] Figure 6 A block diagram showing an example architecture for training a loss prediction model according to some embodiments of the present disclosure;

[0017] Figure 7 A flowchart showing a process of conversion evaluation performed at a content management system according to some embodiments of the present disclosure;

[0018] Figure 8 A flowchart showing a process of conversion evaluation performed at a terminal device according to some embodiments of the present disclosure;

[0019] Fig. 9 A block diagram of an apparatus for conversion evaluation according to some embodiments of the present disclosure is shown;

[0020] Fig.10 A block diagram showing an apparatus for conversion evaluation according to other embodiments of the present disclosure; and

[0021] Fig.11 An electronic device is shown in which one or more embodiments of the present disclosure may be implemented. DETAILED DESCRIPTION

[0022] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0023] In the description of the embodiments of the present disclosure, the term "including" and similar terms should be understood as open inclusion, that is, "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may also be included below.

[0024] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and relevant provisions.

[0025] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0026] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information, so that the user can independently choose whether to provide personal information to software or hardware such as electronic devices, applications, servers or storage media that execute operations of the technical solution of the present disclosure based on the prompt message.

[0027] As an optional but non-limiting implementation, in response to receiving an active request from the user, the prompt information is sent to the user in a manner such as a pop-up window, in which the prompt information can be presented in text form. In addition, the pop-up window can also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0028] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0029] As used herein, the term "model" can learn the association between the corresponding input and output from the training data, so that after the training is completed, the corresponding output can be generated for a given input. The generation of the model can be based on machine learning technology. Deep learning is a machine learning algorithm that processes inputs and provides corresponding outputs by using multi-layer processing units. A neural network model is an example of a model based on deep learning. In this article, "model" may also be referred to as "machine learning model", "learning model", "machine learning network" or "learning network", and these terms are used interchangeably in this article.

[0030] A "neural network" is a machine learning network based on deep learning. A neural network is capable of processing inputs and providing corresponding outputs, and typically includes an input layer and an output layer and one or more hidden layers between the input layer and the output layer. Neural networks used in deep learning applications typically include many hidden layers, thereby increasing the depth of the network. The layers of a neural network are connected in sequence so that the output of the previous layer is provided as input to the next layer, where the input layer receives the input of the neural network and the output of the output layer serves as the final output of the neural network. Each layer of a neural network includes one or more nodes (also called processing nodes or neurons), each of which processes input from the previous layer.

[0031] Generally, machine learning can be roughly divided into three stages, namely the training stage, the testing stage, and the application stage (also called the inference stage). In the training stage, a given model can be trained using a large amount of training data, and the parameter values ​​are continuously updated iteratively until the model can obtain consistent inferences that meet the expected goals from the training data. Through training, the model can be considered to be able to learn the association from input to output (also called the mapping of input to output) from the training data. The parameter values ​​of the trained model are determined. In the testing stage, the test input is applied to the trained model to test whether the model can provide the correct output, thereby determining the performance of the model. In the application stage, the model can be used to process the actual input based on the parameter values ​​obtained from the training to determine the corresponding output.

[0032] Figure 1 A schematic diagram of an example environment 100 in which an embodiment of the present disclosure can be implemented is shown. One or more content providers can use a content management system 120 to manage content to be provided on a content providing platform 110. One or more terminal devices 130-1, 130-2, 130-3, etc. (for ease of discussion, collectively or individually referred to as terminal devices 130) are associated with the content providing platform 110 and can access various types of content provided on the content providing platform 110. As an example, the content providing platform 110 can be an application, a website, a web page, and other accessible resources. The terminal device 130 can be installed with an application for accessing the content providing platform 110, or can access the content providing platform 110 in an appropriate manner.

[0033] The content provider may provide different content to different terminal devices 130 based on management requirements and based on user operations at the terminal device 130. The content management system 120 may provide one or more specific content items related to one or more objects to the terminal device 130, such as one or more content items 142-1, 142-2, ... 142-M (collectively or individually referred to as content items 142 for ease of discussion) in the content database 140. These content items may include advertisements, for example. Objects related to the content items include, for example, objects targeted by the advertisements.

[0034] In some embodiments, the content management system 120 may determine the content item 142 to be provided to one or more terminal devices 130 based at least on a request from an object provider, such as a bid request from an object provider. In the context of advertising delivery, an object provider is sometimes also referred to as an advertiser. In some embodiments, the object provider may also pay a fee to the content provider based on the presentation of the content item.

[0035] In environment 100, terminal device 110 can be any type of mobile terminal, fixed terminal or portable terminal, including mobile phone, desktop computer, laptop computer, notebook computer, netbook computer, tablet computer, media computer, multimedia tablet, personal communication system (PCS) device, personal navigation device, personal digital assistant (PDA), audio / video player, digital camera / camcorder, positioning device, television receiver, radio broadcast receiver, e-book device, game device or any combination of the foregoing, including accessories and peripherals of these devices or any combination thereof. In some embodiments, terminal device 110 can also support any type of interface for users (such as "wearable" circuits, etc.). Content management system 120 can, for example, be various types of computing systems / servers that can provide computing capabilities, including but not limited to mainframes, edge computing nodes, computing devices in cloud environments, and the like.

[0036] It should be understood that the structure and function of the various elements in the environment 100 are described for exemplary purposes only and do not imply any limitation on the scope of the present disclosure.

[0037] In the context of content item provision, it is usually necessary to measure indicators related to content item presentation, including the conversion of content items, where "conversion" refers to downloading, registration, purchasing or other information demand behaviors that occur under the influence of the presented content items. These indicators may be used to determine the presentation effect of content items, determine how content providers charge object providers, determine content providers' supply strategies for content items between terminal devices, etc. For example, for advertisements of certain objects, advertisers may expect to pay content providers according to the number of conversion behaviors of the advertisements.

[0038] The conversion of content items usually occurs on platforms other than the content provider platform, such as a platform managed by the object provider or a third-party platform. For example, the download of multimedia files may occur on the multimedia file source website; the download of certain applications may occur on an application download platform, such as an application store on a terminal device or an application download website; the registration of applications may occur on an application platform, and so on. In this case, the content provider expects the platform managed by the object provider or a third-party platform to provide feedback on the conversion of the content item, such as statistical data on the conversion behavior performed on the object over a period of time. Feedback on the conversion of content items by the platform managed by the object provider or a third-party platform is also called "conversion feedback."

[0039] In some scenarios, conversion feedback may be blocked or delayed for privacy reasons. This means that after presenting a content item, the content provider cannot track in real time whether the conversion behavior for the content item has occurred, and thus cannot accurately evaluate the conversion of the content item. This will be detrimental to measuring the presentation effect of the content item, as well as subsequent payment and supply strategy adjustments.

[0040] In an example embodiment of the present disclosure, an improved conversion evaluation scheme is provided. According to the scheme, after presenting a content item related to an object at a terminal device corresponding to a content providing platform, if a click behavior on the content item is detected, instead of directly jumping to another platform that performs a conversion behavior on the object, an auxiliary content item related to the object continues to be provided. The auxiliary content item is used to further indicate an entry for performing a conversion behavior on the object. Based on the click feedback on the entry, the expected conversion information for the content item is determined, and the information indicates the probability that the conversion behavior on the object is expected to be performed.

[0041] According to this solution, the expected conversion information for the content item can be effectively estimated without relying on conversion feedback information outside the content providing platform, avoiding the situation where conversion evaluation cannot be achieved due to conversion feedback delays or conversion feedback acquisition failures caused by various reasons.

[0042] Some example embodiments of the present disclosure will be described below with continued reference to the accompanying drawings.

[0043] Figure 2 FIG. 2 is a flow chart showing a signaling flow 200 for conversion evaluation according to some embodiments of the present disclosure. Figure 1 The signaling flow 200 is described in detail with reference to the environment 100 of FIG. The signaling flow 200 involves the content management system 120 and the terminal device 130. Figure 2 Although a single terminal device 130 is shown in FIG. 1 , the content management system 120 may perform similar signaling flows with multiple terminal devices 130 .

[0044] In signaling flow 200, content management system 120 causes 205 content item 142 associated with an object to be presented at terminal device 130 corresponding to content providing platform 110. Terminal device 130 presents 210 the corresponding content item 142, for example on a user interface corresponding to the content providing platform.

[0045] The content management system 120 can determine how and when to cause a content item to be presented based on various policies and trigger conditions of the content item 142. In some embodiments, the content management system 120 can also determine to present different content items 142 in the content database 140 to different terminal devices 130. In some embodiments, the content management system 120 can control to cause one content item 142 to be presented at multiple terminal devices 132.

[0046] During the presentation of the content item 142, the terminal device 132 detects a click behavior on the content item 142. If a click behavior on the content item 142 is detected, the terminal device 132 provides 215 feedback to the content management system 120, indicating the detection of the click behavior on the content item 142. Thus, the content management system 120 can determine 220 the detection of the click behavior on the content item 142.

[0047] In an embodiment of the present disclosure, one or more content items 142 in the content database 140 have associated auxiliary content items. As an example, Figure 2 It is shown that content item 142-1 has an associated auxiliary content item 242-1, content item 142-M has an associated auxiliary content item 242-M, and so on. For ease of discussion, these auxiliary content items are collectively or individually referred to as auxiliary content items 242 in this article. In some embodiments, one or more content items 142 (e.g., content item 142-2) may not have auxiliary content items. Embodiments of the present disclosure are not limited to this.

[0048] In response to detecting a click behavior for the content item 142 at a certain terminal device 120, the content management system 120 causes 225 an auxiliary content item 242 related to the object to be presented at the terminal device 130. The terminal device 130 presents 230 the corresponding auxiliary content item 142, for example, on a user interface corresponding to the content providing platform.

[0049] The auxiliary content item 242 and the associated content item 142 are both for the same object. The auxiliary content item 242 at least indicates an entry for performing a conversion behavior for the object. Clicking on the entry enables the terminal device to access a platform for performing the conversion behavior. In an embodiment of the present disclosure, the presentation of the auxiliary content item can be used to further confirm whether the user has the intention to perform a conversion behavior for the object. Such auxiliary content items may sometimes also be referred to as summary items or promotion items for advertisements, etc.

[0050] In some embodiments, the conversion behavior evaluated based on the auxiliary content item may be the next behavior that may be performed after the click behavior of the content item. For example, if the object is a downloadable object, the conversion behavior may include a download behavior. If the object is a paid object, the conversion behavior may include, for example, an add-to-cart behavior. In other embodiments, other conversion behaviors may also be evaluated according to actual needs, such as registration behavior, payment behavior, and other information demand behaviors, etc.

[0051] In some embodiments, the auxiliary content item may also indicate descriptive information related to the object, so as to provide more information related to the object, thereby helping the user to determine whether to perform a conversion behavior. For example, if the object is a downloadable application, the descriptive information indicated by the auxiliary content item may include the application's icon, ranking, page screenshots, basic introduction, installation package size, developer information, etc. It should be understood that the descriptive information here can be configured according to the actual scenario.

[0052] In some embodiments, the content management system 120 may determine whether the auxiliary content item is to be presented at a certain terminal device 130 based at least on a feedback constraint for the conversion behavior of the object. The feedback constraint may indicate whether the conversion behavior performed at a certain terminal device 130 will be delayed or blocked from being transmitted back to the content management system 120. The feedback constraint may be set due to the privacy protection policy of the platform of the object provider or the third-party platform that performs the conversion behavior or other reasons. The feedback constraint may be related to the type of the terminal device 120, the platform of the object provider to be accessed by the terminal device 120, the configuration of the third platform, etc. In one example, if the feedback constraint indicates that the conversion behavior performed at a certain terminal device 130 will be delayed or transmitted back to the content management system 120, the content management system 120 may determine that the auxiliary content item 242 is to be presented.

[0053] In addition to the postback constraints, the content management system 120 may determine whether to present the auxiliary content item 242 based on other factors, such as a request from an object provider regarding whether the auxiliary content item is to be presented. Embodiments of the present disclosure are not limited in this regard.

[0054] During the presentation of the auxiliary content item 242, the terminal device 130 detects a click behavior for the entrance of the conversion behavior, and provides 235 click feedback for the entrance to the content management system 120, the click feedback indicating whether the entrance is clicked at the terminal device 130. The content management system 120 receives 240 the click feedback. Since the auxiliary content item 242 is presented under the control of the content management system 120 and is presented within the content providing platform, the content management system 120 is able to receive and collect such feedback information without being constrained by other platforms.

[0055] In some embodiments, if the terminal device 130 detects a click behavior on the entry, click feedback indicating that the entry is clicked may be provided to the content management system 120. In some embodiments, if the terminal device 130 still does not detect a click behavior on the entry after the presentation of the auxiliary content item 242 ends, or does not detect a click behavior on the entry within a period of time after the presentation starts, click feedback indicating that the entry is not clicked may be provided to the content management system 120. In some embodiments, if the auxiliary content item 242 is presented at multiple terminal devices 130, the content management system 120 may receive feedback information from the multiple terminal devices 130 on whether the entry in the auxiliary content item 142 is clicked.

[0056] To better understand the example embodiments, the following description will be made with reference to example pages.

[0057] FIG. 3A to FIG. 3C A schematic diagram of a presentation interface related to a content item according to some embodiments of the present disclosure is shown. Figure 3A A user interface 301 for presenting content items is shown. In this example, the user interface 301 is shown as a page of an application running on a terminal device, in which content items (i.e., advertisements) related to an object (in this example, "Application XX") are presented. The user interface 301 also includes a pop-up window 310 indicating relevant information about "Application XX" and a download indicator 312 indicating "Application XX".

[0058] The terminal device 130 monitors the click behavior on the content item in the user interface 301. In some embodiments, the detection of the click behavior can be determined based on the user's click operation at one or more locations in the presentation interface of the content item, or other selection operations (e.g., sliding gestures, voice control commands, etc.). In some embodiments, the click behavior at one or more locations in the user interface 301 can trigger the presentation of the auxiliary content item. Figure 3B As shown, if the user clicks on the pop-up window 310 in the user interface 301, clicks on the download mark 312, and performs a sliding gesture in a predetermined direction on the page, it can be determined that the click behavior on the content item is executed.

[0059] In response to detecting a click behavior on a content item, a display such as Figure 3C The user interface 303 shown in the figure presents auxiliary content items related to "Application XX". Specifically, the user interface 303 indicates the download entry of "Application XX", such as download identifiers 322, 324, etc. The user interface 303 also indicates the description information of "Application XX", such as the application icon, development company, banner information, brief introduction, installation package size, etc. of "Application XX".

[0060] If a click behavior on the entry is detected, such as a click operation on the download mark 322, 324, the terminal device 130 can provide a click feedback indicating that the entry is clicked to the content management system 120. In some examples, in addition to the download mark 322, 324, one or more other positions in the user interface 303 can also be configured as an entry for the conversion behavior of the object. This can be configured according to actual needs, and the embodiments of the present disclosure are not limited in this respect.

[0061] In some embodiments, if the entrance indicated by the auxiliary content item 242 is clicked, the terminal device 130 can also access or jump to a platform for performing conversion behavior for the object. Figure 3D A user interface 304 for executing a conversion behavior for an object is shown, and the user interface 304 is a download interface of application XX. In this interface, the user can execute the download behavior as needed.

[0062] It should be understood that Figures 3A to 3D The user interfaces shown are merely examples, and various interface designs may exist in practice. The various graphical elements in the interface may have different arrangements and different visual representations, one or more of which may be omitted or replaced, and one or more other elements may also exist. The embodiments of the present disclosure are not limited in this respect.

[0063] Continue to return to reference Figure 2 By providing the auxiliary content item, the content management system 120 can collect click feedback on the entry of the conversion behavior from the terminal device 130. Further, the content management system 120 determines 245 the expected conversion information for the content item based on the click feedback on the entry, and the expected conversion information indicates the probability that the conversion behavior for the object is expected to be executed.

[0064] The presentation of content item 142 may be referred to as a display or impression of the content item, e.g. Figure 3AIn the evaluation of content item 142, the number of presentations, i.e., the number of displays or impressions, of content item 142 is also counted. The number of presentations of content item 142 may include one or more presentations on multiple terminal devices, and / or one or more presentations on the same terminal device.

[0065] In addition, the number of clicks on the presented content item 142 is counted, e.g. Figure 3B The number of clicks on content item 142 may include one or more clicks on multiple terminal devices, and / or one or more presentations on the same terminal device. In some examples, for a specific content item 142, the probability that a click behavior on content item 142 is performed may also be determined, also known as a click-through rate (CTR). The click-through rate may be determined as the ratio of the number of times a click behavior on content item 142 is performed to the number of presentations of content item 142. For example, if content item 142 is presented 10,000 times, and a click behavior is detected in 8,000 of the presentations, the click-through rate may be determined to be 0.8.

[0066] In the conversion evaluation, it is also desirable to evaluate the conversion information of the conversion actions performed for the content item 142, also referred to as the conversion value rate (CVR). The conversion rate can be determined as the number of times the conversion actions of the object are performed accounted for the number of times the click actions of the content item 142 are performed. For example, if eight thousand click actions are detected in ten thousand presentations of the content item, and a total of six thousand conversion actions are detected in the eight thousand click actions, then the conversion rate can be determined as 0.75.

[0067] In an embodiment of the present disclosure, it is proposed to evaluate the conversion information of a content item based on click feedback for an entry in an auxiliary content item. In this scheme, if a further click on an entry for a conversion behavior in an auxiliary content item is detected, sufficient information can be used to believe that the probability of the conversion behavior actually occurring is very high. In other words, the click behavior on the auxiliary content item has a strong correlation with the conversion behavior of the object.

[0068] The content management system 120 can determine the number of times the click behavior for a specific content item 142 is performed. The statistics here can be performed across a period of time and multiple terminal devices 130. The content management system 120 can also determine the number of times the entrance to the conversion behavior is clicked when the auxiliary content item 242 is presented based on the click feedback from the terminal device 130, which is sometimes referred to as the number of intermediate clicks in this article. The content management system 120 can determine the number of clicks on the entrance to the conversion behavior indicated by the specific auxiliary content item 242 across a period of time and multiple terminal devices 130.

[0069] In some embodiments, considering that the click behavior on the auxiliary content item has a strong correlation with the conversion behavior of the object, the content management system 130 can approximate the number of clicks on the entry of the conversion behavior to the number of times the conversion behavior is performed, that is, the number of times the conversion behavior for the object is "expected" to be performed. Based on this, the content management system 130 can determine the expected conversion information for the content item 142 to indicate the probability that the conversion behavior is expected to occur when the content item 142 is clicked. For example, the expected conversion information can be determined as the ratio of the number of clicks on the entry of the conversion behavior to the number of clicks on the content item 142. For example, if eight thousand click behaviors are detected in ten thousand presentations of the content item, and among the eight thousand click behaviors, seven thousand click behaviors are detected for the entry indicated by the auxiliary content item, then the expected conversion information can be determined to indicate a conversion rate of 7 / 8.

[0070] The estimated conversion information can be used to measure the presentation effect of the content item 142. Since the click behavior on the auxiliary content item does not completely mean that the conversion behavior for the object actually occurs, for example, when jumping to Figure 3D After the user displays the download interface, the user does not perform the download behavior, or the interface jump is blocked. Therefore, there may be a certain error between the expected conversion information for content item 142 and the actual conversion information of content item 142. This part of the error is called drop information, or drop rate. Drop information indicates: when the expected conversion information indicates that the conversion behavior for the object is expected to be executed, the probability that the conversion behavior is not actually executed. In other words, the drop information is determined by the statistical data of the part of the conversion behavior that is not actually executed after clicking the entrance on the auxiliary content item. The drop rate (expressed as drop_ratio) can be determined as follows:

[0071] drop_ratio=1-sum(install_convert) / sum(middle_click) (1)

[0072] Where sum(install_convert) indicates the total number of times the conversion behavior for a specific object is "actually" executed; sum(middle_click) indicates the total number of times the click behavior for the entrance is detected on the auxiliary content item, which is used to represent the total number of times the conversion behavior is "expected" to be executed. Generally, if the drop_ratio is lower, sum(middle_click) means that it is approximately close to the real sum(install_convert).

[0073] In some embodiments, the content management system 130 may utilize the collected estimated conversion information to execute more subsequent strategies and calculate other metrics of the content items.

[0074] Figure 4 The block diagram shows an example structure of the content management system 120 according to some embodiments of the present disclosure. As shown in the figure, the content management system 120 may include at least an information collection subsystem 410, a model training subsystem 420 and a model application subsystem 430.

[0075] The information collection subsystem 410 may be configured to determine projected conversion information for one or more content items 142 according to the process described with respect to the signaling flow 200. These content items 142 have auxiliary content items 242, so the projected conversion information is determined based on click behavior detected on the auxiliary content items 242 as discussed above.

[0076] In some embodiments, the model training subsystem 420 is configured to train a conversion prediction model 422 based at least on the collected expected conversion information. The conversion prediction model 422 is configured to predict conversion information for content items related to a given object, such as the probability that a conversion action for a given object is performed when the content item is presented or clicked.

[0077] The trained conversion prediction model 422 is provided to the model application subsystem 430 , which is configured to use the trained conversion prediction model 422 to predict expected conversion information 432 for a certain content item 412 to indicate the probability that a conversion behavior of an object related to the content item 412 is expected to be executed.

[0078] In some embodiments, the model training subsystem 420 is further configured to train at least a loss prediction model 424. The loss prediction model 424 is configured to determine loss information for a content item. The training data of the loss prediction model 424 includes at least the actual total number of conversion behaviors actually performed for the corresponding object and the number of click behaviors detected on the auxiliary content item.

[0079] The trained impairment prediction model 424 may be provided to a model application subsystem 430, which is configured to utilize the trained impairment prediction model 424 to predict impairment information 434 for a content item 412 to indicate the probability that a conversion behavior is not actually performed when the expected conversion information indicates that a conversion behavior for the relevant object is expected to be performed.

[0080] In some embodiments, if the predicted conversion information 432 (expressed as pcvr) for a certain content item 142 is determined using the trained conversion prediction model 422, and the click behavior on the content item 142 triggers the presentation of the auxiliary content item 242, then the loss information 434, such as drop_ratio, for the content item 142 can also be determined based on the trained loss prediction model 424. Updated conversion information for the content item 142 can be determined based on the predicted conversion information 432 and the loss information 434 to indicate the probability that the conversion behavior for the object is actually performed, which can be expressed as cvr=pcvr*(1-drop_ratio). In the updated conversion information, the part of the statistical data where the conversion behavior is not actually performed is eliminated.

[0081] In some embodiments, based on the updated conversion information, the model application subsystem 430 may further determine other information related to the content item 142, such as revenue-related information 436. The revenue-related information 426 is generally a content item metric that the object provider is concerned about. The revenue-related information 436 may be determined based on at least the conversion information for the content item 142 and the conversion bid information for the content item 142.

[0082] In some examples, the revenue-related information 436 may indicate the revenue that can be obtained at a predetermined number of presentations of the content item, for example, the advertising revenue that can be obtained for every thousand presentations (i.e., effective cost per mile, ECPM). The revenue-related information 436 may also have other measurement criteria, and for ease of discussion, ECPM is used as an example below. ECPM may be determined based on:

[0083] ECPM=bid*pctr*pcvr*(1-drop_ratio) (2)

[0084] Wherein bid refers to the bid of the object provider for a single conversion behavior of the object, pctr indicates the expected click-through rate of the related content item, pcvr indicates the expected conversion information for the content item (for example, it may be the expected conversion information 432 determined by the conversion prediction model 422), and drop_ratio indicates the loss information for the content item (for example, it may be the loss information 434 determined by the loss prediction model 424).

[0085] Revenue-related information 436, such as ECPM, can be used to determine the bidding strategy for the content item. For example, the bidding strategy may include the lowest cost bid (lowest cost) and the cost cap bid (cost cap). The lowest cost bidding scheme is that the object provider sets a planned budget and realizes the lowest cost bidding product under the premise of consuming the entire budget as much as possible within a period of time (for example, one day or before the end of the advertising plan schedule). The cost cap bidding scheme is to ensure that the actual cost of the content item placed by the object provider is as close as possible to its bid (bid). For the cost cap bidding scheme, revenue-related information needs to be considered in order to accurately determine the revenue related to the content item. Of course, revenue-related information 436 can also be used in other evaluation schemes for content items, or in various scenarios such as the execution of other strategies.

[0086] The following will first introduce the structure of the conversion prediction model 422 and its training process, and then introduce the training process of the loss prediction model 424.

[0087] The conversion prediction model 422 can be constructed based on any appropriate machine learning model or neural network model. In some embodiments, due to the absence of conversion constraints or other reasons, the immediate conversion backflow of certain content items 142 may be possible, and thus the auxiliary content items of certain content items may not be provided. In some embodiments, it is expected that the conversion prediction model 422 is trained to be able to be universally applied to the conversion information prediction of such content items 142 and the conversion information prediction of the auxiliary content items that need to be provided. In this case, the training data of the conversion prediction model 422 includes the expected conversion information determined on the basis of providing the auxiliary content items, and the conversion information determined on the basis of not providing the auxiliary content items. The latter conversion information indicates the probability that the conversion behavior of the corresponding object is executed, which can be directly returned from a platform other than the content providing platform (the platform that performs the conversion behavior). The platform that performs the conversion behavior can be, for example, a platform managed by the object provider or a third-party platform.

[0088] In some embodiments, in order to enable the conversion prediction model 422 to provide more accurate predictions for different types of content items, the conversion prediction model 422 may be constructed to use different output parts to determine the expected conversion information for different types of content items. Figure 5 A block diagram showing an example structure of a conversion prediction model 422 according to such an embodiment is shown. Figure 5As shown, the conversion prediction model 422 includes a general part 510, a first output part 512, and a second output part 514. The general part 510 serves as a shared base part of the first output part 512 and the second output part 514, which receives the model input and processes the model input to obtain intermediate feature information. If the model input is related to a content item that provides an auxiliary content item, the intermediate feature information is provided to the first output part 512, which determines the expected conversion information. If the model input is related to a content item that does not need to provide an auxiliary content item, the intermediate feature information is provided to the first output part 514, which determines the expected conversion information.

[0089] The model input may include information related to the content item itself, information related to the object of the content item, and other relevant information that can be used to predict conversion information, which is not limited in the embodiments of the present disclosure.

[0090] In the model training stage, the general part 510 can be trained using the expected conversion information related to the content items that provide auxiliary content items and the expected conversion information related to the content items that do not need to provide auxiliary content items. In other words, the expected conversion information related to the two types of content items can be used to affect the parameter update of the general part 510. For the first output part 512, the expected conversion information related to the content items that provide auxiliary content items is used for training, and for the second output part 514, the expected conversion information related to the content items that do not need to provide auxiliary content items is used for training. In this way, the first output part 512 and the second output part 514 can each be dedicated to the prediction of conversion information of the corresponding type of content items.

[0091] Although the reference Figure 5 The example structure of the conversion prediction model 422 is described. In other embodiments, multiple output parts may not be designed, but the predicted conversion information corresponding to the two types of content items may be used to jointly train the conversion prediction model 422. According to actual needs, in some embodiments, two conversion prediction models may be designed for the two types of content items, respectively, for predicting conversion information.

[0092] The training data of the impairment prediction model 424 includes at least the actual total number of times the conversion behaviors for the corresponding objects are actually executed. However, in many cases, the actual total number of times the conversion behaviors are actually executed may be delayed for a period of time before being provided to the content management system 120 by other platforms. For example, in order to prevent tracking, the return time of each conversion behavior can be random. But usually, after a long period of time, the conversion behaviors that occurred during this period of time may have been returned to the content management system 120. In addition, the inventors have also found that the delay time of the overall return follows a fixed distribution. Therefore, in some embodiments, an improved training method for training the impairment prediction model 424 is proposed.

[0093] Figure 6 A block diagram of an example architecture 600 for training a depreciation prediction model 424 according to some embodiments of the present disclosure is shown. The content item for which the depreciation prediction model 424 is trained has an associated auxiliary content item.

[0094] exist Figure 6 In the embodiment of the invention, one or more content items x(1), x(2), x(3), ... x(m) are considered to train the impairment prediction model 424. For each content item, it is assumed that the actual total number of conversion behaviors actually performed for the object related to the content item within a time range 602 (represented as ad_skan) can be determined, for example, based on one or more conversion feedbacks. In addition, multiple times 620 of the click behaviors performed for the entrance indicated by the auxiliary content item 242 at multiple time points (represented as 1, 2, 3, ... n) within the time range can be obtained, represented as mid_click(1), mid_click(2), mid_click(3) ... mid_click(n).

[0095] In the architecture 600, the time model 610 is configured as a plurality of conversion return probabilities 612 corresponding to a plurality of time points, represented as time_prob(1), time_prob(2), time_prob(3) ... time_prob(n), each conversion return probability indicating the probability that the number of times the conversion behavior for the object is executed at the corresponding time point has been returned at the time point n. The input of the time model 610 is the time difference information 605, represented as time_diff(1), time_diff(2), time_diff(3) ... time_diff(n), which is used to characterize each specific time point.

[0096] The total number of times 632 that the conversion behavior for the object is expected to be performed within the time range is determined based on the multiple times and the multiple conversion return probabilities, which is represented as mid_click_aggr. For example, mid_click(1), mid_click(2), mid_click(3) ... mid_click(n) can be determined by multiplying mid_click(1), mid_click(2), time_prob(3) ... time_prob(n) element by element and then adding them.

[0097] In each iteration of training, the loss prediction model 424 uses the current model parameter values ​​to determine the loss information drop_ratio' for the corresponding content item (e.g., a content item in x(1), x(2), x(3), ... x(m)). Based on mid_click_aggr and (1-drop_ratio'), the predicted total number of conversion behaviors actually performed for the relevant object within the time range from time point 1 to n can be determined 660y', that is, y'=mid_click_aggr*(1-drop_ratio').

[0098] During training, the model update module 660 in the model training subsystem 420 may determine the difference between the predicted total number of times 660y' and the actual total number of times 602ad_skan, and update the impairment prediction model 424 based on the difference, for example, update the parameter value of the impairment prediction model 424. In some embodiments, the model update module 660 may construct a loss function based on the difference between the predicted total number of times 660y' and the actual total number of times 602ad_skan, and based on the loss function, use an appropriate model training algorithm to determine the update for the impairment prediction model 424. The embodiments of the present disclosure are not limited in terms of the model training algorithm.

[0099] For each content item x(1), x(2), x(3), ... x(m), the above steps can be used to determine the difference between the predicted total number of times 660y' and the actual total number of times 602ad_skan to update the impairment prediction model 424. The impairment prediction model 424 can be updated to continuously reduce or minimize the difference between the predicted total number of times 660y' and the actual total number of times 602ad_skan until the model training target is reached.

[0100] Figure 7 A flow chart illustrating a process 700 of conversion evaluation performed at content management system 120 according to some embodiments of the present disclosure is shown.

[0101] In box 710, the content management system 120 causes the first content item related to the first object to be presented at the terminal device corresponding to the content providing platform. In box 720, the content management system 120 determines whether a click behavior on the first content item is detected. In box 730, in response to determining the detection of the click behavior on the first content item, the content management system 120 causes an auxiliary content item related to the first object to be presented at the terminal device, and the auxiliary content item at least indicates an entry for performing a conversion behavior on the first object. In box 740, first expected conversion information for the first content item is determined based on the click feedback on the entry, the click feedback indicates whether the entry is clicked at the terminal device, and the first expected conversion information indicates the probability that the conversion behavior on the first object is expected to be performed.

[0102] In some embodiments, the conversion behavior includes a downloading behavior of the first object.

[0103] In some embodiments, the auxiliary content item also indicates descriptive information related to the first object.

[0104] In some embodiments, causing the auxiliary content item to be presented includes: determining whether the auxiliary content item is to be presented at the terminal device based at least on a postback constraint for a conversion behavior of the first object; and selectively causing the auxiliary content item to be presented based on the determination.

[0105] In some embodiments, process 700 further includes: training a conversion prediction model based on at least the first predicted conversion information, the conversion prediction model being trained to predict conversion information for content items related to the given object.

[0106] In some embodiments, training a conversion prediction model includes: obtaining second expected conversion information for a second content item, the second content item being related to a second object, and the second expected conversion information being determined based on: a probability of a conversion behavior for the second object being executed obtained from a platform other than a content providing platform; training a general part of the conversion prediction model based on the first expected conversion information and the second expected conversion information; training a first output part of the conversion prediction model based on an intermediate output of the general part and the first expected conversion information; and training a second output part of the conversion prediction model based on the intermediate output of the general part and the second expected conversion information.

[0107] In some embodiments, process 700 further includes: using the trained conversion prediction model to predict third predicted conversion information for a third content item, where the third content item is related to a third object, and the third predicted conversion information indicates a probability that a conversion behavior for the third object is expected to be performed.

[0108] In some embodiments, when a click behavior on the third content item is detected, another auxiliary content item is presented. In some embodiments, process 700 further includes: obtaining impairment information on the third content item, the impairment information indicating: when the third predicted conversion information indicates that the conversion behavior on the third object is expected to be performed, the probability that the conversion behavior is not actually performed; and determining updated conversion information on the third content item based on the third predicted conversion information and the impairment information, the updated conversion information indicating the probability that the conversion behavior on the third object is actually performed.

[0109] In some embodiments, process 700 further includes determining revenue-related information for the third content item based on the updated conversion information and conversion bid information for the third content item, the conversion bid information indicating a bid for a single conversion behavior of the third object.

[0110] In some embodiments, obtaining the impairment information includes: obtaining a trained impairment prediction model; and determining the impairment information for the third content item using the impairment prediction model.

[0111] In some embodiments, process 700 also includes training the depreciation prediction model by: determining the total number of times conversion behaviors for the fourth object are expected to be performed within a time range when a fourth content item related to the fourth object is presented; determining the expected depreciation information for the fourth content item using the depreciation prediction model to be trained; determining the predicted total number of times conversion behaviors for the fourth object are actually performed within the time range based on the expected depreciation information and the total number of times; and updating the depreciation prediction model based on the difference between the predicted total number of times and the actual total number of times conversion behaviors for the fourth object are actually performed within the time range.

[0112] In some embodiments, the additional auxiliary content item is presented when a click behavior for the fourth content item is detected. In some embodiments, determining the total number of times a conversion behavior for the fourth object is expected to be performed within a time range includes: obtaining a plurality of times a click behavior for an entry indicated by the additional auxiliary content item is performed at a plurality of time points within the time range; determining a plurality of conversion return probabilities corresponding to the plurality of time points, the conversion return probability indicating the probability that the number of times a conversion behavior for the fourth object is performed at the corresponding time point has been returned at the latest time point among the plurality of time points; and determining the total number of times a conversion behavior for the fourth object is expected to be performed within the time range based on the plurality of times and the plurality of conversion return probabilities.

[0113] Figure 8 A flowchart of a conversion evaluation process 800 performed at a terminal device 140 according to some embodiments of the present disclosure is shown.

[0114] In box 810, the terminal device 140 presents a first content item related to the first object. In box 820, the terminal device 140 determines whether a click behavior on the first content item is detected. In response to detecting the click behavior on the first content item, in box 830, the terminal device 140 presents an auxiliary content item related to the first object, the auxiliary content item at least indicating an entry for performing a conversion behavior on the first object. In box 840, the terminal device 140 provides a click feedback on the entry to the content management system, and the click feedback indicates whether the entry is clicked at the terminal device.

[0115] Fig. 9 A schematic structural block diagram of an apparatus 800 for conversion evaluation according to some embodiments of the present disclosure is shown. The apparatus 900 may be implemented as or included in a content management system 120. Each module / component in the apparatus 900 may be implemented by hardware, software, firmware, or any combination thereof.

[0116] As shown in the figure, the device 900 includes a first presentation control module 910, which is configured to make a first content item related to a first object be presented at a terminal device corresponding to a content providing platform, and the device 900 also includes a second presentation control module 920, which is configured to make an auxiliary content item related to the first object be presented at the terminal device in response to the detection of a click behavior on the first content item, and the auxiliary content item at least indicates an entry for performing a conversion behavior on the first object. The device 900 also includes a conversion determination module 930, which is configured to determine first expected conversion information for the first content item based on click feedback on the entry, the click feedback indicating whether the entry is clicked at the terminal device, and the first expected conversion information indicating the probability that the conversion behavior on the first object is expected to be performed.

[0117] In some embodiments, the conversion behavior includes a downloading behavior of the first object.

[0118] In some embodiments, the auxiliary content item also indicates descriptive information related to the first object.

[0119] In some embodiments, the second presentation control module 920 includes: a determination module, configured to determine whether an auxiliary content item is to be presented at a terminal device based at least on a feedback constraint on a conversion behavior of a first object; and a selective presentation module, configured to selectively cause the auxiliary content item to be presented based on the determination.

[0120] In some embodiments, the apparatus 900 further includes: a first training module configured to train a conversion prediction model based at least on the first predicted conversion information, wherein the conversion prediction model is trained to predict conversion information for content items related to a given object.

[0121] In some embodiments, the first training model includes: an information acquisition module, configured to acquire second expected conversion information for a second content item, the second content item being related to a second object, and the second expected conversion information being determined based on: a probability of a conversion behavior for the second object being executed obtained from a platform other than a content providing platform; a common part training module, configured to train a common part of the conversion prediction model based on the first expected conversion information and the second expected conversion information; a first output part training module, configured to train a first output part of the conversion prediction model based on an intermediate output of the common part and the first expected conversion information; and a second output part training module, configured to train a second output part of the conversion prediction model based on the intermediate output of the common part and the second expected conversion information.

[0122] In some embodiments, the device 900 also includes: a first model application module, configured to use the trained conversion prediction model to predict third expected conversion information for a third content item, the third content item is related to a third object, and the third expected conversion information indicates the probability that a conversion behavior for the third object is expected to be executed.

[0123] In some embodiments, when a click behavior on the third content item is detected, another auxiliary content item is presented. In some embodiments, the device 900 further includes a loss acquisition module configured to acquire loss information on the third content item, the loss information indicating: when the third predicted conversion information indicates that the conversion behavior on the third object is expected to be executed, the probability that the conversion behavior is not actually executed; and a conversion update module configured to determine updated conversion information on the third content item based on the third predicted conversion information and the loss information, the updated conversion information indicating the probability that the conversion behavior on the third object is actually executed.

[0124] In some embodiments, the device 900 also includes a revenue determination module configured to determine revenue-related information for the third content item based on the updated conversion information and conversion bid information for the third content item, where the conversion bid information indicates a bid for a single conversion behavior for the third object.

[0125] In some embodiments, the impairment acquisition module includes: a model acquisition module configured to acquire a trained impairment prediction model; and a second model application module configured to determine impairment information for the third content item using the impairment prediction model.

[0126] In some embodiments, the device 900 also includes a second model training module, which is configured to train the depreciation prediction model by: determining the total number of times conversion behaviors for the fourth object are expected to be performed within a time range when a fourth content item related to the fourth object is presented; determining the expected depreciation information for the fourth content item using the depreciation prediction model to be trained; determining the predicted total number of times conversion behaviors for the fourth object are actually performed within the time range based on the expected depreciation information and the total number of times; and updating the depreciation prediction model based on the difference between the predicted total number of times and the actual total number of times conversion behaviors for the fourth object are actually performed within the time range.

[0127] In some embodiments, an additional auxiliary content item is presented when a click behavior for the fourth content item is detected. In some embodiments, the second model training module is configured to determine the total number of times the conversion behavior for the fourth object is expected to be performed within a time range by: obtaining multiple times of click behaviors for the entrance indicated by the additional auxiliary content item at multiple time points within the time range; determining multiple conversion return probabilities corresponding to the multiple time points, the conversion return probability indicating the probability that the number of times the conversion behavior for the fourth object is performed at the corresponding time point has been returned at the latest time point among the multiple time points; and determining the total number of times the conversion behavior for the fourth object is expected to be performed within the time range based on the multiple times and the multiple conversion return probabilities.

[0128] Fig.10 A block diagram of an apparatus for conversion evaluation according to some other embodiments of the present disclosure is shown. The apparatus 1000 may be implemented as or included in a terminal device 140. Each module / component in the apparatus 1000 may be implemented by hardware, software, firmware, or any combination thereof.

[0129] As shown in the figure, the apparatus 1000 includes a first presentation module 1010, which is configured to present a first content item related to a first object at a terminal device corresponding to a content providing platform; and a second presentation module 1020, which is configured to present an auxiliary content item related to the first object in response to detecting a click behavior on the first content item, the auxiliary content item at least indicating an entry for performing a conversion behavior on the first object. The apparatus 1000 also includes a feedback providing module 1030, which is configured to provide a click feedback on the entry to the content management system, the click feedback indicating whether the entry is clicked at the terminal device.

[0130] Fig.11 1 shows a block diagram of an electronic device 1100 in which one or more embodiments of the present disclosure may be implemented. It should be understood that Fig.11The electronic device 1100 shown is merely exemplary and should not constitute any limitation on the functionality and scope of the embodiments described herein. Fig.11 The electronic device 1100 shown can be used to implement Figure 1 The terminal device 140 or the content management system 120.

[0131] like Fig.11 As shown, the electronic device 1100 is in the form of a general-purpose computing device. The components of the electronic device 1100 may include, but are not limited to, one or more processors or processing units 1110, a memory 1120, a storage device 1130, one or more communication units 1140, one or more input devices 1150, and one or more output devices 1160. The processing unit 1110 may be an actual or virtual processor and is capable of performing various processes according to a program stored in the memory 1120. In a multi-processor system, multiple processing units execute computer executable instructions in parallel to improve the parallel processing capability of the electronic device 1100.

[0132] The electronic device 1100 typically includes a plurality of computer storage media. Such media may be any available media accessible to the electronic device 1100, including but not limited to volatile and non-volatile media, removable and non-removable media. The memory 1120 may be a volatile memory (e.g., a register, a cache, a random access memory (RAM)), a non-volatile memory (e.g., a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. The storage device 1130 may be a removable or non-removable medium, and may include a machine-readable medium, such as a flash drive, a disk, or any other medium, which may be capable of being used to store information and / or data (e.g., training data for training) and may be accessed within the electronic device 1100.

[0133] The electronic device 1100 may further include additional removable / non-removable, volatile / non-volatile storage media. Fig.11 As shown in , a disk drive for reading or writing from a removable, non-volatile disk (e.g., a "floppy disk") and an optical drive for reading or writing from a removable, non-volatile optical disk may be provided. In these cases, each drive may be connected to the bus (not shown) by one or more data media interfaces. Memory 1120 may include a computer program product 1125 having one or more program modules that are configured to perform various methods or actions of various embodiments of the present disclosure.

[0134] The communication unit 1140 implements communication with other electronic devices through a communication medium. Additionally, the functions of the components of the electronic device 1100 can be implemented in a single computing cluster or multiple computing machines that can communicate through a communication connection. Therefore, the electronic device 1100 can operate in a networked environment using a logical connection with one or more other servers, a network personal computer (PC), or another network node.

[0135] The input device 1150 may be one or more input devices, such as a mouse, a keyboard, a tracking ball, etc. The output device 1160 may be one or more output devices, such as a display, a speaker, a printer, etc. The electronic device 1100 may also communicate with one or more external devices (not shown) through the communication unit 1140 as needed, such as a storage device, a display device, etc., communicate with one or more devices that allow a user to interact with the electronic device 1100, or communicate with any device that allows the electronic device 1100 to communicate with one or more other electronic devices (e.g., a network card, a modem, etc.). Such communication may be performed via an input / output (I / O) interface (not shown).

[0136] According to an exemplary implementation of the present disclosure, a computer-readable storage medium is provided, on which computer-executable instructions are stored, wherein the computer-executable instructions are executed by a processor to implement the method described above. According to an exemplary implementation of the present disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, and the computer-executable instructions are executed by a processor to implement the method described above.

[0137] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices, equipment, and computer program products implemented according to the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0138] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0139] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, so that the instructions executed on the computer, other programmable data processing apparatus, or other device implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0140] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple implementations of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of a module, program segment or instruction includes one or more executable instructions for realizing the logical function of the specification. In some implementations as replacements, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be realized by a special hardware-based system that performs the function or action of the specification, or can be realized by a combination of special hardware and computer instructions.

[0141] The above descriptions of various implementations of the present disclosure are exemplary, non-exhaustive, and not limited to the disclosed implementations. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described implementations. The selection of terms used herein is intended to best explain the principles of the implementations, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the various implementations disclosed herein.

Claims

1. A method for conversion evaluation, comprising: causing a first content item related to the first object to be presented at a terminal device corresponding to a content providing platform; In response to determining the detection of the click behavior on the first content item, causing an auxiliary content item related to the first object to be presented at the terminal device, the auxiliary content item at least indicating an entry for performing a conversion behavior on the first object; as well as determining first predicted conversion information for the first content item based on a click feedback for the entry, the click feedback indicating whether the entry is clicked at the terminal device, the first predicted conversion information indicating a probability that the conversion behavior for the first object is expected to be executed, Wherein, the method further comprises: presenting another auxiliary content item when a click behavior on a third content item is detected, the third content item being related to the third object; acquiring impairment information for the third content item, the impairment information indicating: a probability that the conversion behavior for the third object is not actually executed when the third predicted conversion information for the third content item indicates that the conversion behavior for the third object is predicted to be executed; and Updated conversion information for the third content item is determined based on the third predicted conversion information and the depreciation information, the updated conversion information indicating a probability that the conversion behavior for the third object is actually performed. The method according to claim 1 , wherein the conversion behavior comprises a downloading behavior of the first object. 3 . The method according to claim 1 , wherein the auxiliary content item further indicates descriptive information related to the first object.

4. The method of claim 1 , wherein causing the auxiliary content item to be presented comprises: determining whether the auxiliary content item is to be presented at the terminal device based at least on a postback constraint for a conversion behavior of the first object; as well as The auxiliary content item is selectively caused to be presented based on the determination.

5. The method according to claim 1, further comprising: A conversion prediction model is trained based at least on the first predicted conversion information, the conversion prediction model being trained to predict conversion information for content items related to a given object.

6. The method according to claim 5, wherein training the conversion prediction model comprises: Acquire second predicted conversion information for a second content item, the second content item being related to a second object, and the second predicted conversion information being determined based on: a probability of the conversion behavior for the second object being performed obtained from a platform other than the content providing platform; training a common portion of the conversion prediction model based on the first predicted conversion information and the second predicted conversion information; training a first output portion of the conversion prediction model based on the intermediate output of the general portion and the first predicted conversion information; as well as A second output portion of the conversion prediction model is trained based on the intermediate output of the general portion and the second predicted conversion information.

7. The method according to claim 5, further comprising: The trained conversion prediction model is used to predict third predicted conversion information for a third content item, where the third content item is related to a third object, and the third predicted conversion information indicates a probability that the conversion behavior for the third object is expected to be performed.

8. The method according to claim 1, further comprising: Revenue-related information for the third content item is determined based on the updated conversion information and conversion bid information for the third content item, where the conversion bid information indicates a bid for a single conversion behavior for the third object.

9. The method according to claim 8, wherein obtaining the damage information comprises: Get the trained loss prediction model; as well as The impairment information for the third content item is determined using the impairment prediction model.

10. The method according to claim 9, further comprising training the impairment prediction model by: determining a total number of times the conversion behavior for the fourth object is expected to be performed within a time range when a fourth content item related to the fourth object is presented; Determining estimated loss information for the fourth content item using the loss prediction model to be trained; Determine, based on the estimated depreciation information and the total number of times, a predicted total number of times that the conversion behavior for the fourth object is actually executed within the time range; The depreciation prediction model is updated based on the difference between the predicted total number of times and the actual total number of times the conversion behavior for the fourth object is actually performed within the time range.

11. The method of claim 10, wherein additional auxiliary content items are presented if a click action on the fourth content item is detected, and wherein determining the total number of times the conversion action on the fourth object is expected to be performed within a time range comprises: Acquire a plurality of times that the click behavior is performed on the entry indicated for the additional auxiliary content item at a plurality of time points within the time range; Determine a plurality of conversion return probabilities corresponding to the plurality of time points, the conversion return probability indicating a probability that the number of times the conversion behavior for the fourth object is executed at the corresponding time point has been returned at the latest time point among the plurality of time points; as well as The total number of times the conversion action for the fourth object is expected to be performed within the time range is determined based on the multiple times and the multiple conversion return probabilities.

12. A method for conversion evaluation, comprising: Presenting, at a terminal device corresponding to the content providing platform, a first content item related to the first object; In response to detecting a click behavior on the first content item, presenting an auxiliary content item related to the first object, the auxiliary content item at least indicating an entry for performing a conversion behavior on the first object; as well as providing a content management system with click feedback for the portal, the click feedback indicating whether the portal is clicked at the terminal device, Wherein, the method further comprises: presenting a further auxiliary content item when a click action on a third content item is detected, the third content item being related to the third object; acquiring impairment information for the third content item, the impairment information indicating: a probability that the conversion behavior for the third object is not actually executed when the third predicted conversion information for the third content item indicates that the conversion behavior for the third object is predicted to be executed; and Updated conversion information for the third content item is determined based on the third predicted conversion information and the depreciation information, the updated conversion information indicating a probability that the conversion behavior for the third object is actually performed.

13. A device for transformation assessment, comprising: A first presentation control module is configured to enable a first content item related to a first object to be presented at a terminal device corresponding to a content providing platform; a second presentation control module, configured to, in response to detecting a click behavior on the first content item, cause an auxiliary content item related to the first object to be presented at the terminal device, the auxiliary content item at least indicating an entry for performing a conversion behavior on the first object; as well as a conversion determination module, configured to determine first predicted conversion information for the first content item based on a click feedback for the entry, the click feedback indicating whether the entry is clicked at the terminal device, the first predicted conversion information indicating a probability that the conversion behavior for the first object is expected to be executed, Wherein, the device further comprises: a second auxiliary content item presentation module, configured to present another auxiliary content item in case of detecting a click behavior on a third content item, wherein the third content item is related to a third object; a loss acquisition module configured to acquire loss information for the third content item, the loss information indicating: a probability that the conversion behavior for the third object is not actually executed when the third predicted conversion information for the third content item indicates that the conversion behavior for the third object is predicted to be executed; and The conversion update module is configured to determine updated conversion information for the third content item based on the third predicted conversion information and the depreciation information, wherein the updated conversion information indicates a probability that the conversion behavior for the third object is actually executed.

14. The apparatus according to claim 13, wherein the second presentation control module comprises: a determination module configured to determine whether the auxiliary content item is to be presented at the terminal device based at least on a return constraint for a conversion behavior of the first object; as well as A selective presentation module is configured to selectively cause the auxiliary content item to be presented based on the determination.

15. The apparatus according to claim 13, further comprising: The first training module is configured to train a conversion prediction model based at least on the first predicted conversion information, wherein the conversion prediction model is trained to predict conversion information for content items related to a given object.

16. The apparatus according to claim 15, further comprising: The first model application module is configured to use the trained conversion prediction model to predict third expected conversion information for a third content item, where the third content item is related to a third object, and the third expected conversion information indicates a probability that the conversion behavior for the third object is expected to be executed.

17. A device for transformation assessment, comprising: A first presentation module is configured to present a first content item related to the first object at a terminal device corresponding to the content providing platform; A second presentation module is configured to present, in response to detecting a click behavior on the first content item, an auxiliary content item related to the first object, the auxiliary content item at least indicating an entry for performing a conversion behavior on the first object; as well as a feedback providing module, configured to provide a click feedback for the portal to the content management system, wherein the click feedback indicates whether the portal is clicked at the terminal device, Wherein, the device further comprises: a second auxiliary content item presentation module, configured to present another auxiliary content item in case of detecting a click behavior on a third content item, wherein the third content item is related to a third object; a loss acquisition module configured to acquire loss information for the third content item, the loss information indicating: a probability that the conversion behavior for the third object is not actually executed when the third predicted conversion information for the third content item indicates that the conversion behavior for the third object is predicted to be executed; and The conversion update module is configured to determine updated conversion information for the third content item based on the third predicted conversion information and the depreciation information, wherein the updated conversion information indicates a probability that the conversion behavior for the third object is actually executed.

18. An electronic device, comprising: at least one processing unit; as well as At least one memory, the at least one memory being coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions causing the device to perform the method according to any one of claims 1 to 11 or the method according to claim 12 when executed by the at least one processing unit.

19. A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the method according to any one of claims 1 to 11 or the method according to claim 12 is implemented.

Images