Methods, apparatus, devices, and storage media for transformation assessment

Abstract

According to an embodiment of the present disclosure, a method, an apparatus, a device and a storage medium for conversion evaluation are provided. The method comprises: extracting resource features from resource-related data of a target resource; extracting audience characteristics of a target audience group from audience-related data of the target audience group, the target audience group being to be distributed with a recommended content item related to the target resource; and determining a target expected conversion rate for the target resource based on the resource features and the audience characteristics, the target expected conversion rate indicating an expected probability of the target audience group performing a conversion for the target resource through a predetermined association between the resource features and the audience characteristics and the expected conversion rate. According to the scheme, the accuracy of the conversion rate evaluation can be improved, and the delivery effect of the recommended content item for the resource can also be improved.

Description

Technical Field

[0001] The exemplary embodiments disclosed herein generally relate to the field of computer technology, and particularly to methods, apparatus, devices, and computer-readable storage media for conversion evaluation. Background Technology

[0002] The internet provides access to a wide variety of resources. For example, it offers access to various applications, products, audio and video content, and more. With the rapid increase in both quantity and variety, finding resources of interest from a vast amount of information is extremely difficult for the audience. Resource providers, on the other hand, also hope their resources will attract the attention of their target audience. Therefore, recommendation systems are used to recommend resources that meet user needs to a specific group of users. When providing recommendations related to specific resources, it is often expected that the audience will perform specific conversion events, such as downloading, registering, adding to cart, purchasing, or other resource-related actions. The evaluation of conversions can affect the delivery of resource recommendations and payment methods; therefore, accurate conversion evaluation is essential. Summary of the Invention

[0003] In a first aspect of this disclosure, a method for conversion evaluation is provided. The method includes: extracting resource features from resource-related data of a target resource; extracting audience characteristics of a target audience from audience-related data of a target audience of the target resource, wherein the target audience is to be distributed with recommended content items related to the target resource; and determining a target projected conversion rate for the target resource based on the resource features and audience characteristics, through a predetermined correlation between the resource features, audience characteristics, and projected conversion rate, wherein the target projected conversion rate indicates the projected probability that the target audience will perform a conversion for the target resource.

[0004] In a second aspect of this disclosure, an apparatus for conversion evaluation is provided. The apparatus includes: a resource feature extraction module configured to extract resource features from resource-related data of a target resource; an audience feature extraction module configured to extract audience features of a target audience from audience-related data of a target audience of the target resource, the target audience being distributed with recommended content items related to the target resource; and a conversion rate determination module configured to determine a target expected conversion rate for the target resource based on the resource features and audience features, through a predetermined correlation between the resource features, audience features, and expected conversion rate, the target expected conversion rate indicating the expected probability that the target audience will perform a conversion for the target resource.

[0005] In a third aspect of this disclosure, an electronic device is provided. The device includes at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit. When executed by the at least one processing unit, the instructions cause the device to perform the method of the first aspect.

[0006] In a fourth aspect of this disclosure, a computer-readable storage medium is provided. The medium stores a computer program that, when executed by a processor, implements the method of the first aspect.

[0007] It should be understood that the content described in this summary section is not intended to limit the key or essential features of the embodiments of this disclosure, nor is it intended to restrict the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0008] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0009] Figure 1 A schematic diagram of an example environment in which embodiments of the present disclosure can be implemented is shown;

[0010] Figure 2 A schematic block diagram of a content management system according to some embodiments of the present disclosure is shown;

[0011] Figure 3 A schematic diagram illustrating the training process of a conversion rate prediction model according to some embodiments of the present disclosure is shown.

[0012] Figure 4 The model architecture and training architecture of a conversion rate prediction model according to some embodiments of the present disclosure are shown;

[0013] Figure 5 A schematic block diagram illustrating an example of a strategy maker according to some embodiments of the present disclosure is shown;

[0014] Figure 6 A flowchart illustrating a conversion evaluation process according to some embodiments of this disclosure is shown;

[0015] Figure 7 A block diagram of an apparatus for conversion evaluation according to some embodiments of the present disclosure is shown; and

[0016] Figure 8 An electronic device in which one or more embodiments of the present disclosure may be implemented is shown. Detailed Implementation

[0017] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0018] In the description of embodiments of this disclosure, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may also be included below.

[0019] It is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and related provisions.

[0020] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure through appropriate means in accordance with relevant laws and regulations, and user authorization should be obtained.

[0021] For example, in response to receiving a user's active request, a prompt message is sent to the user to clearly inform the user that the requested operation will require the acquisition and use of the user's personal information, thereby enabling the user to choose whether to provide personal information to the software or hardware such as electronic devices, applications, servers or storage media that perform the operation of the technical solution disclosed herein, based on the prompt message.

[0022] As an optional but non-restrictive implementation, in response to a user's active request, a prompt message can be sent to the user, such as a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0023] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0024] As used in this paper, the term "model" refers to a model that learns the relationship between inputs and outputs from training data, enabling it to generate corresponding outputs for a given input after training. Model generation can be based on machine learning techniques. Deep learning is a machine learning algorithm that processes inputs and provides corresponding outputs using multiple layers of processing units. A neural network model is an example of a deep learning-based model. In this paper, "model" may also be referred to as a "machine learning model," "learning model," "machine learning network," or "learning network," and these terms are used interchangeably.

[0025] A neural network is a machine learning network based on deep learning. A neural network processes input and provides a corresponding output, typically consisting of an input layer, an output layer, and one or more hidden layers between the input and output layers. Neural networks used in deep learning applications often include many hidden layers, thus increasing the network's depth. The layers of a neural network are connected sequentially, so that the output of the previous layer is provided as the input to the next layer. The input layer receives the input to the neural network, while the output layer's output serves as the final output. Each layer of a neural network includes one or more nodes (also called processing nodes or neurons), each node processing the input from the layer above.

[0026] Machine learning typically comprises three phases: training, testing, and application (also known as inference). In the training phase, a given model is trained using a large amount of training data, iteratively updating parameter values ​​until the model can consistently generate inferences that meet the expected goals from the training data. Through training, the model can be considered to have learned the relationship between inputs and outputs (also known as the input-output mapping) from the training data. The parameter values ​​of the trained model are determined. In the testing phase, test inputs are applied to the trained model to test whether it can provide the correct output, thus determining the model's performance. The testing phase can sometimes be integrated into the training phase. In the application phase, the trained model can be used to process actual model inputs based on the trained parameter values ​​to determine the corresponding model output.

[0027] Figure 1A schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented is shown. One or more content providers may use a content management system 120 to manage content to be distributed on a content distribution platform 110. One or more terminal devices 130-1, 130-2, 130-3, etc. (collectively or individually referred to as terminal device 130 for ease of discussion) are associated with the content distribution platform 110 and may access various types of content offered on the content distribution platform 110, for example, based on a corresponding audience 132-1, 132-2, 132-3, etc. (collectively or individually referred to as audience 132 for ease of discussion). As an example, the content distribution platform 110 may be an application, a website, a webpage, or other accessible platform. Terminal devices 130 may have applications installed for accessing the content distribution platform 110, or may access the content distribution platform 110 in a suitable manner.

[0028] The content management system 120 can be configured to provide an audience with one or more specific recommended content items related to one or more resources (e.g., provided to terminal device 130) based on corresponding strategies. Recommended content items may include, for example, one or more recommended content items 142-1, 142-2, ... 142-M (collectively or individually referred to as recommended content item 142 for ease of discussion) in the content database 140.

[0029] In this document, resources can include various promotional objects, such as applications, physical goods, virtual goods, audio and video content, etc. Recommended content items refer to content presented to recommend a corresponding resource. Examples of recommended content items include advertisements. Audience groups can include one or more audience members, such as audience 132. Audience members can be any potential consumers of the resource, such as users, groups, organizations, entities, etc.

[0030] In some embodiments, multiple different recommended content items 142 can be provided for a given resource. The content management system 120 can distribute the corresponding recommended content items on the content distribution platform 110 based on requests from resource providers 150-1, 150-2, 150-3, etc. (collectively or individually referred to as resource providers 150 for ease of discussion).

[0031] In some embodiments, the content management system 120 may distribute recommended content items 142 to the corresponding audience 132 on the content distribution platform 110 based on requests from at least the various resource providers 150-1, 150-2, 150-3, etc. (collectively or individually referred to as resource provider 150 for ease of discussion). In advertising scenarios, resource provider 150 is sometimes also referred to as the advertiser. In some embodiments, the resource provider may also pay the content provider based on the presentation of the recommended content items and subsequent conversions.

[0032] In some embodiments, the content management system 120 may also select recommended content items for presentation to a specific terminal device 130 in a content distribution opportunity (e.g., at a specific time and location) on the content distribution platform 110 based on bidding results. For example, the content management system 120 may receive bids from resource providers 150 and allocate content distribution opportunities to the highest bidder. A bid may be the cost that resource providers 150 are willing to pay to utilize content distribution opportunities to present their recommended content items. For example, a bid may indicate the cost that a resource provider is willing to pay for the presentation of a recommended content item, referred to as eCPM (Expected Cost Per Mile). Alternatively, a bid may specify the amount that a resource provider is willing to pay for audience interaction with the recommended content item (e.g., clicking on it or hovering a pointer over it) or a “conversion” following the audience’s interaction with the recommended content item.

[0033] A conversion occurs when an audience performs a desired action related to a recommended content item. The events constituting a conversion can vary and can be determined in several ways. For example, a conversion can occur when a recommended content item is clicked. A conversion can also be defined by the resource provider as any measurable / observable event, such as, for example, a download, navigating to a given depth on a website or app, viewing a certain number of web pages, spending a predetermined amount of time on a website, web page, or app, registering an account on a website or app, adding an item to a shopping cart, completing a purchase, etc. Other events constituting a conversion can also be used.

[0034] In environment 100, terminal device 110 can be any type of mobile terminal, fixed terminal, or portable terminal, including mobile phones, desktop computers, laptop computers, notebook computers, netbook computers, tablet computers, media computers, multimedia tablets, personal communication system (PCS) devices, personal navigation devices, personal digital assistants (PDAs), audio / video players, digital cameras / camcorders, positioning devices, television receivers, radio receivers, e-book devices, gaming devices, or any combination thereof, including accessories and peripherals of these devices or any combination thereof. In some embodiments, terminal device 110 may also support any type of user-facing interface (such as "wearable" circuitry). Content management system 120 can be, for example, various types of computing systems / servers capable of providing computing power, including but not limited to mainframes, edge computing nodes, computing devices in cloud environments, etc.

[0035] It should be understood that the structure and function of the various elements in environment 100 are described for illustrative purposes only and do not imply any limitation on the scope of this disclosure.

[0036] In scenarios involving recommended content distribution, it's typically necessary to measure conversion metrics related to this distribution, including estimating the probability of a user making a conversion. This conversion probability is also known as the conversion rate (CVR). The conversion rate significantly impacts the formulation of many subsequent strategies.

[0037] Currently, conversion rate prediction models are typically trained to evaluate conversion rates. Many conversion rate models typically evaluate conversion rates at the audience and recommended content item levels to attribute conversions performed by an audience group to a specific recommended content item. Therefore, these models are configured to assess the probability of a conversion occurring when a user clicks on a specific recommended content item. Conversion rate prediction models are constantly iterating and optimizing, but the optimization direction mainly focuses on collecting more model input samples to enrich audience characteristics and designing more complex model structures to model audience preferences for recommended content items (e.g., advertisements).

[0038] However, in recommendation scenarios, conversions occur less frequently than the number of recommended content items presented, especially deep conversion events. This leads to insufficient data for model training, preventing the model from learning effectively and resulting in low prediction accuracy. Inaccurate conversion rate predictions can cause numerous problems, such as inaccurate billing methods for recommended content and imprecise content recommendation strategies. Furthermore, because it's necessary to assess the probability of a conversion event occurring when a user clicks on a specific recommended content item, data related to that item and its audience is scarce before it's distributed or in the early stages of distribution (i.e., the cold start phase), further exacerbating the problem of inaccurate model predictions. On the other hand, while designing complex models can improve performance, model operation incurs excessive resource consumption and time consumption, making it difficult to balance the gains and costs of complex models.

[0039] In the embodiments of this disclosure, a more generalized conversion evaluation scheme is proposed. Unlike conversion evaluation schemes that rely solely on attribution data, in the scheme of this disclosure, both attribution data and non-attribution data can be used to model the conversion evaluation process during its execution. For ease of understanding, attribution data and non-attribution data are first defined here.

[0040] Specifically, conversions for a particular resource may occur not only on the current content distribution platform but also on other platforms, such as platforms managed by the resource provider, other content distribution platforms, or third-party platforms. For example, an app provider might run ads for the app on multiple different platforms. For a specific ad on a particular platform, attributable data refers to conversions (e.g., app downloads) that occur when a user clicks on that specific ad on that platform; non-attributable data refers to conversions that cannot be attributed to that specific ad, including conversions attributed to ads on other content distribution platforms or conversions performed spontaneously by the user. Comparatively, non-attributable data is often more abundant than attributable data.

[0041] According to the scheme disclosed herein, a method is proposed to assess the probability of a user converting to a resource at both the user and resource levels. This conversion assessment is based on audience-related data and resource-related data. By extracting audience and resource characteristics, it is possible to construct the audience's preferences for resources, particularly their preferences for resources that lead to conversions. This improves the accuracy of conversion rate assessment and, consequently, enhances the effectiveness of targeted content recommendations for that resource.

[0042] The following description will continue with reference to the accompanying drawings, which will provide some exemplary embodiments of this disclosure.

[0043] Figure 2 A schematic block diagram of a content management system according to some embodiments of the present disclosure is shown. Some embodiments of the present disclosure can be implemented in this content management system. For ease of discussion, reference will be made to... Figure 1 The content management system 120 in environment 100 is described.

[0044] like Figure 2 As shown, the content management system 120 includes a feature extractor 210 and a conversion rate prediction model 220. In some embodiments described below, the content management system 120 may also include a strategy maker 230.

[0045] Feature extractor 210 is configured to acquire resource-related data 202 of the target resource and audience-related data 204 of the target audience of the target resource. The target resource may be a resource supplied by a specific resource provider 150, such as an application, physical goods, virtual goods, etc. The target audience refers to the audience that expects to receive recommended content items related to the target resource, which may include one or more audience members.

[0046] Feature extractor 210 is configured to extract resource features 212 from resource-related data 202 and audience features 214 from audience-related data 204. Resource features 212 are used to characterize the relevant characteristics of the target resource, and audience features 214 are used to characterize the relevant characteristics of the target audience. Resource features 212 and audience features 214 can be represented in the form of multidimensional vectors.

[0047] In some embodiments, resource-related data 202 may include attribute information related to one or more aspects of the target resource, such as resource type, supplier, industry information, appearance information, price, review information, rating information, etc. Audience-related data 204 may include attribute information related to one or more aspects of the target audience, such as demographic information of the target audience, historical conversion behavior for various resources, historical operations performed on the content distribution platform (e.g., browsing behavior), and other data authorized by the target audience.

[0048] In some embodiments, resource-related data 202 and audience-related data 204 may be provided to the content management system 120 based on authorization from resource provider 150 and / or target audience (e.g., one or more audiences 132). As an example, the target audience may authorize resource provider 150 to obtain audience-related data 204. The content management system 120 may receive resource-related data 202 and audience-related data 204 from resource provider 150. In other examples, the content management system 120 may obtain the required data from resource provider 150 and other data sources authorized by the target audience, respectively.

[0049] Resource feature 212 and audience feature 214 are used to determine the target projected conversion rate 222 for the target resource. The target projected conversion rate 222 indicates the projected probability that the target audience will perform a conversion for the target resource. Specifically, the target projected conversion rate corresponding to resource feature 212 and audience feature 214 is determined through a predetermined correlation between resource feature and audience feature and the projected conversion rate.

[0050] The "predetermined correlation" between resource and audience characteristics and projected conversion rates is used to characterize the mapping between resource and audience characteristics and conversion rates for a given resource. This correlation models audience preferences for resources to assess the likelihood of a target audience converting for a target resource.

[0051] In some embodiments, the "predetermined correlation" between resource features and audience features and the projected conversion rate can be represented in the form of a machine learning model, which may be called a conversion rate prediction model, such as... Figure 2The conversion rate prediction model 220 is shown. The model input of the conversion rate prediction model 220 includes the resource characteristics of a resource and the audience characteristics of the resource's audience. The model output includes the predicted conversion rate for the resource, indicating the predicted probability that the audience will perform a conversion for the resource.

[0052] Unlike modeling audience preferences for specific recommended content items, embodiments of this disclosure model audience preferences for resources in a more generalized manner at the level of both the target audience and the target resource to be recommended, in order to achieve conversion evaluation for the target resource. According to this modeling approach, relationships can be determined not only from conversion data attributable to specific recommended content items, but also from a variety of conversion data (including attributive and non-attributive data) for a specific resource. This allows for the use of more conversion data to model more accurate relationships, which in turn can be used to perform more accurate conversion evaluations (e.g., conversion rate prediction models).

[0053] The predetermined correlation between resource characteristics, audience characteristics, and projected conversion rates, or the conversion rate prediction model 220, can be determined through the training process. This will be referenced below. Figure 3 and Figure 4 Let's describe the training process of the conversion rate prediction model in more detail.

[0054] The target projected conversion rate 222 can be provided to the strategy maker 230, which is configured to perform various strategy specification regarding the recommended content item 142 associated with the target resource. In some embodiments, the strategy maker 230 is configured to determine a distribution strategy for the recommended content item 142 associated with the target resource within a target audience based on the target projected conversion rate. In some embodiments, the distribution strategy may indicate one or more parameters related to the distribution of the recommended content item 142, including in which / which distribution opportunities it is distributed to the target audience, and / or to which audience members within the target audience. The distribution strategy may also indicate other parameters related to the distribution of the recommended content item 142, such as distribution cost data, such as the cost of converting the recommended content item 142, etc. These specific parameters are associated with the specific management strategies of the content distribution platform. Some example embodiments related to the formulation of distribution strategies will be described below.

[0055] Figure 3 A schematic diagram of a training process 300 for a conversion rate prediction model 220 according to some embodiments of the present disclosure is shown. The training process 300 includes a data preprocessing stage 320, a sample construction stage 340, and a model training stage 360.

[0056] In the data preprocessing stage 320, conversion data 310 of one or more sample resources is acquired. Sample resources can be of any type, or they can be specific to a particular type of resource (e.g., all applications). Conversion data 310 indicates resource-related data for the sample resources, as well as audience-related data for the sample audience who have performed conversions for those sample resources. That is, recommended content items related to the sample resources may have been distributed to these sample audiences, and data can be collected showing that these sample audiences have performed conversions.

[0057] Conversion data 310 may include multi-platform conversion data for the sample resource, without needing to be attributed to a specific recommended content item of the sample resource. In some embodiments, conversion data 310 may include conversion data occurring within an audience after being attributed to one or more recommended content items served on content delivery platform 110, such as an app download performed by an audience after clicking an ad on content delivery platform 110. Conversion data 310 may also include conversion data attributed to recommended content items served on other content delivery platforms, as well as conversion data from user-initiated conversions. In some embodiments, conversion data 310 may be obtained based on authorization from the resource provider 150 of the sample resource and / or authorization from the sample audience. In some embodiments, conversion data 310 may be obtained from the resource provider 150.

[0058] In the data preprocessing stage 320, resource features of sample resources and audience features of the sample audience for the sample resources are extracted from the conversion data 310, and these features are stored in the resource pool 330. The specific features extracted can be determined based on the design of the model input for the conversion rate prediction model 220.

[0059] In the sample construction phase 340, training samples required for training the conversion rate prediction model 220 are constructed from the features included in the feature pool 330. In some embodiments, the training samples include positive training samples, which consist of resource features of a specific sample resource obtained from the feature pool and audience features of a first sample audience group for that sample resource. Positive training samples indicate that the sample audience group corresponding to the audience features has performed a conversion for the sample resource, that is, the expected sample conversion rate of the sample audience group for the sample resource is 100% or close to 100%. Positive training samples can be constructed from resource features extracted from conversion data 310 and the corresponding audience features. From the positive training samples, the model can learn which audience groups are likely to convert for which resources.

[0060] In some embodiments, model training also requires negative training samples to allow the model to learn which audience groups are unlikely to convert for which resources. Since the conversion data 310 provides features related to conversions that have already occurred, in some embodiments, negative training samples can be constructed based on a random sampling method of the audience. Specifically, sample features from the positive training samples can be kept unchanged, and audience features can be randomly selected from the feature pool 330 to form the negative training samples. The randomly selected audience features are highly likely to involve audience groups that have not converted for the sample resources (sometimes referred to herein as the "second sample audience group"). In other embodiments, negative training samples can also be constructed in other ways, and the embodiments of this disclosure are not limited in this respect.

[0061] In the sample construction stage 340, a certain number of positive training samples and negative training samples can be constructed according to the needs of model training, and the ratio between positive training samples and negative training samples can be determined.

[0062] In some embodiments, training samples may be identified by corresponding audience identifiers (IDs) and resource IDs to indicate the audience characteristics and resource characteristics included in the training samples, respectively. In some embodiments, positive training samples may also be identified by event identifiers (IDs) (also called event tags) to indicate the event type of conversion of the sample audience in the positive training samples. During conversion, events corresponding to various demand behaviors performed by the audience on resources may all be defined as conversions, such as downloading, registering, adding to cart, purchasing, or other resource demand behaviors.

[0063] In the training samples, the occurrence of different events will characterize the probability of conversion to varying degrees. For example, deeper-level conversion events (e.g., download or purchase) may allow the model to better determine the audience's preference for the resource, while shallower-level conversion events (e.g., adding to cart or visiting a webpage) may allow the model to partially learn the audience's preference for the resource. Therefore, the training of the model will differ for different types of events. In some embodiments, it is expected that the conversion rate prediction model 220 can differentially learn the impact of different interactive behaviors on the occurrence of conversion. The training objective of the conversion rate prediction model 220 is configured to update the model parameter values ​​of the conversion rate prediction model based at least on the event type.

[0064] The samples constructed in the sample construction phase 340 are stored in the sample pool 350. In the model training phase 360, the conversion rate prediction model 220 is trained using the positive and negative training samples in the sample pool 350.

[0065] In some embodiments, the conversion rate prediction model 220 can be constructed as a model that is adapted to receive resource features and audience features as model inputs, process the model inputs, and provide a predicted conversion rate as model outputs.

[0066] Considering that the conversion rate prediction model 220 needs to explore the audience's preferences for resources and accurately predict the probability of conversion, in some embodiments, during the conversion prediction process, the conversion rate prediction model 220 can be constructed to determine the feature cross results of the input resource features and audience features, and output the predicted conversion rate based on the feature cross results. Feature cross refers to increasing the nonlinearity of the model by combining two or more features to achieve a nonlinear transformation of the sample space. Through feature cross, the correlation between the audience and resources can be better explored.

[0067] The model can be trained using various model training techniques. During training, training objectives can be constructed (e.g., training objectives based on the loss function), and the model's parameter values ​​can be updated using algorithms such as stochastic gradient descent. The model parameter values ​​can be continuously updated until the training objective is achieved, such as minimizing or reducing the loss function to an expected value, or reaching a predetermined number of update iterations, and so on.

[0068] Because it is not limited to attribution for specific recommended content items, more attribution and non-attribution conversion data can be obtained to characterize sample resources and sample audiences. Conversion rate prediction models trained based on this conversion data can learn the audience's posterior conversion history for resources, better generalize to other resources and audiences, better determine the audience's conversion preferences for resources, and thus accurately predict the audience's potential conversion probability for resources. In some embodiments, the conversion rate prediction model trained based on attribution and non-attribution data can also be called a non-attribution perturbation model.

[0069] Figure 4 The model architecture and training architecture of a conversion rate prediction model 220 according to some embodiments of the present disclosure are shown. Figure 4 In the example, conversion rate prediction model 220 is built on a wide and deep architecture to explore multi-level feature interactions between resource features and learned features that serve as model inputs. During training, the parameter values ​​of conversion rate prediction model 220 can also be personalized for different types of conversion events by using a Learn Hidden Unit Contribution (LHUC) network.

[0070] Specifically, such as Figure 4As shown, the conversion rate prediction model 220 includes a logistic regression (LR) part, a factorization machine (FM) part, and a deep learning (DL) part.

[0071] In the LR section, input 410 may include a combination of resource features and audience features. The LR section may include a summing module 412 and a cascading module 414, which perform feature summing and feature cascading on the features in input 410, respectively, to obtain merged features 418 and cascaded features 419.

[0072] The Feature FM (Factor Optimization) module is configured to perform automatic cross-features of resource feature 420 and audience feature 421. The FM module includes pooling modules 423 and 424, configured to perform feature pooling on resource feature 420 and audience feature 421, respectively, to increase or decrease the feature dimension to a specific dimension. FM module 425 can be configured to learn the weights of the cross-features through the inner product of the feature vectors. In some embodiments, the feature cross-features used in FM module 425 can be manually designed, for example, a second-order cross-feature (i.e., pairwise cross-features). The feature cross-feature results output by FM module 425 are provided to summing module 426 and cascading module 427, which perform feature summing and feature cascading, respectively, to obtain merged feature 428 and cascaded feature 429.

[0073] In the deep learning section, a combination 430 of resource features and audience features is provided to the deep learning module 432, which is configured to automatically learn how resource features and audience features perform feature cross-interaction based on deep learning technology to obtain cross-features 434.

[0074] The conversion rate prediction model 220 may also include a cascade module 440 for cascading features 419, 429, and 434 from the various parts to obtain cascaded features 442. The cascaded features 442 are provided to the LHUC network 450.

[0075] The LHUC network 450, also known as the LHUC tower, is configured to determine updates to the parameter values ​​of the conversion rate prediction model 220 based on the event type of the conversion indicated by the event label (e.g., event ID) of the training sample corresponding to the current model input. The LHUC network 450 can control the parameter values ​​of the conversion rate prediction model 220 to be adjusted differently for different types of conversion events. In some examples, the adjustment step of the parameter values ​​of the conversion rate prediction model 220 can be determined to be larger for deeper-level conversions, while the adjustment part of the model's parameter values ​​can be determined to be smaller for lower-level conversions. In this way, the conversion rate prediction model 220 can converge faster and learn to evaluate deeper-level conversions.

[0076] The summation module 460 is configured to aggregate summation features 418, summation features 428, and the features processed by the LHUC network 450, and the output module 470 determines the model output based on the summation result. The loss function module 480 is configured to determine the loss value of the loss function based on the difference between the currently given model output and the expected model output of the training samples (i.e., the expected conversion rate), and update the model's parameter values ​​based on the loss value.

[0077] The training process of a model is completed by iteratively updating the model parameter values ​​until the training objective is reached. Note that in Figure 4 In this model, the LHUC network 450 and loss function module 480 are modules set up for the purpose of model training and will no longer be needed in the model application stage after training is completed.

[0078] It should be understood that Figure 4 The model architecture and training architecture of the conversion rate prediction model shown are merely examples for illustrative purposes. Depending on the specific circumstances, other model architectures (e.g., other models based on deep learning or neural networks) can be designed, and other model training techniques can be utilized (e.g., the LHUC network may not be used). The embodiments disclosed herein are not limited in this respect. For example, the FM portion can be replaced by Wie's structure based on Field-aware Factorization Machines (FFM), and so on.

[0079] After the model is trained, the trained conversion rate prediction model 220 can be provided to the content management system 120 to perform conversion rate prediction for target resources and target audiences.

[0080] As mentioned earlier, the target projected conversion rate 222 output by the conversion rate prediction model 220 for the target resources and target audience can be provided to the strategy executor to determine the distribution strategy.

[0081] Figure 5 A schematic block diagram of an example of a strategy maker 230 according to some embodiments of the present disclosure is shown. It is assumed that the strategy maker 230 wants to determine a distribution strategy for recommended content items 142 (e.g., specific advertisements) associated with a target resource within a target audience.

[0082] In some embodiments, in the strategy maker 230, the target projected conversion rate 222 is used to adjust (or perturb) the cost data for the recommended content item 142, thereby affecting the distribution strategy of the recommended content item 142.

[0083] For example, in the distribution of auction-based recommended content items, specific cost data for recommended content item 142 is provided. This cost data can indicate the cost that resource provider 150 is willing to pay to utilize content distribution opportunities to present the recommended content item. For example, the bid can indicate the cost that resource provider is willing to pay for the presentation of the recommended content item, which can affect the evaluation of eCPM.

[0084] From the perspective of resource providers, eCPM is the expected revenue per thousand impressions. Content distribution platforms rely on content distribution opportunities for revenue, so they tend to allocate distribution opportunities to content distribution plans with higher and more stable eCPMs to maximize expected revenue. Expected revenue is a statistically theoretical concept, primarily related to the resource provider's willingness to bid and the quality of the recommended content (e.g., the likelihood of clicks and conversions). The calculation formula is: eCPM = Bid ​​(Pbid) * Estimated Click-Through Rate (eCTR) * Estimated Conversion Rate (eCVR) * 1000

[0085] From the perspective of resource providers, eCPM is the estimated cost per thousand impressions. The higher the eCPM, the more competitive the recommendations are, and the greater the number of conversions.

[0086] Therefore, more reasonable cost data and more accurate eCPM can better measure the expected distribution effect, thereby optimizing the distribution effect of recommended content items.

[0087] exist Figure 5 In the illustrated cost-adjustment-based strategy formulation embodiment, the strategy maker 230 includes a coefficient determination module 510, a cost adjustment module 520, and a strategy formulation module 530. The coefficient determination module 510 is configured to determine a cost adjustment coefficient 512 for the recommended content item 142 based on the target projected conversion rate 222.

[0088] The target projected conversion rate 222 indicates the projected conversion probability for the target audience currently expected to receive the recommended content item 142 from the target resource. In some embodiments, to further reduce the instantaneous conversion error of the conversion rate prediction model 220, the conversion evaluation results can also be optimized based on historical conversion data.

[0089] In some embodiments, the coefficient determination module 510 can perform historical conversion rate accumulation based on a sliding window at the granularity of recommended content items. Specifically, the coefficient determination module 510 can invoke the conversion rate prediction model 220 to determine the historical predicted conversion rate for the target resource, which indicates the predicted probability that the historical audience of the target resource will perform a conversion for the target resource. Here, the historical audience refers to the recommended content items 142 related to the target resource that have been provided within a historical time period. The historical time period can include a predetermined time window calculated backward from the current time, which can be a time window at the hour level or a window at other time granularities.

[0090] When determining the historical projected conversion rate, audience-related data from one or more historical audience groups to which the recommended content item was distributed during a historical time period can be obtained. One or more corresponding audience features can be extracted, and each audience feature and the resource features corresponding to the target resource can be used as input. This input is then applied to the conversion rate prediction model 220 to obtain the historical projected conversion rate provided by the conversion rate prediction model 220. The historical projected conversion rates calculated multiple times during a historical time period can be averaged to obtain the historical projected conversion rate for the target historical time period. Compared to continuously accumulating conversion rate information over a historical period after the recommended content item was distributed, considering the historical conversion rate within a single historical time period allows for a better capture of more recent conversion trends.

[0091] The coefficient determination module 510 determines a cost adjustment coefficient for the recommended content item 142 based on the ratio between the target projected conversion rate and the historical projected conversion rate. In some embodiments, if the ratio indicates that the target projected conversion rate exceeds the historical projected conversion rate, the cost adjustment coefficient is increased by a first value. In some embodiments, if the ratio indicates that the target projected conversion rate exceeds the historical projected conversion rate, the cost adjustment coefficient is decreased by a second value. In some embodiments, the first and / or second values ​​may be calculated based on a specific ratio between the target projected conversion rate and the historical projected conversion rate.

[0092] The following is an example of how to calculate the cost adjustment factor:

[0093]

[0094] Where perturb_coef represents the cost adjustment coefficient, perturb_ratio represents the disturbance weight, which can be a predetermined value; full_cvr represents the target expected conversion rate; and avg_full_cvr represents the historical expected conversion rate.

[0095] According to equation (1) above, if full_cvr exceeds avg_full_cvr, perturb_coef increases by a certain amount and is greater than 1; conversely, if full_cvr is lower than avg_full_cvr, perturb_coef increases by a certain amount and is less than 1. In some embodiments, an upper limit and / or lower limit for the cost adjustment coefficient can also be set. If the cost adjustment coefficient determined based on the target projected conversion rate and the historical projected conversion rate exceeds a predetermined upper limit, the cost adjustment coefficient can be directly determined as the predetermined upper limit. If the cost adjustment coefficient determined based on the target projected conversion rate and the historical projected conversion rate is lower than a predetermined lower limit, the cost adjustment coefficient can be directly determined as the predetermined lower limit, which can be represented, for example, by the following equation (2):

[0096]

[0097] Where perturb_coef_clip represents the cost adjustment coefficient determined under upper and lower bound constraints; perturb_coef represents the cost adjustment coefficient determined based on the target expected conversion rate and the historical expected conversion rate; upper_bound represents the predetermined upper limit; and lower_bound represents the predetermined lower limit.

[0098] In the strategy maker 230, the cost adjustment module 520 is configured to adjust the cost data for the recommended content item 142 based on the cost adjustment coefficient 512, resulting in adjusted cost data 522. As previously mentioned, the cost data for the recommended content item 142 may include the bid for the recommended content item 142. The cost adjustment coefficient 512 can be used to determine whether to increase or decrease the current bid, which can be represented, for example, as:

[0099] perturb_real_bid=real_bid*perturb_coef_clip (3) where real_bid represents the current cost data of recommended content item 142, and perturb_real_bid represents the adjusted cost data. According to the above equations (1) to (3), it can be seen that if the conversion rate of the target resource of the audience is observed to be continuously increasing, the cost data can be appropriately increased, and vice versa.

[0100] The strategy formulation module 530 is configured to determine a distribution strategy 232 based on adjusted cost data 522. The distribution strategy 232 can instruct how recommended content items 142 related to the target resource are distributed among the target audience.

[0101] With accurate assessment of conversion rates, audience preferences for resources can be effectively determined, leading to the development of better distribution strategies to promote conversions after receiving recommended content and optimize content recommendation effectiveness.

[0102] In some embodiments, the strategy maker 230 can dynamically adjust the distribution strategy of recommended content items by perturbing the cost data, thereby optimizing the recommendation effect. Furthermore, during the strategy formulation process, historical conversion data can be taken into account through a sliding average accumulation method to achieve dynamic adjustment of the cost perturbation coefficient.

[0103] In addition to cost data used to adjust recommended content items, the target projected conversion rate for a resource at the resource level can also be applied to other aspects related to the resource and / or its recommended content items. For example, in some embodiments, the projected conversion rate for the target audience at the recommended content item level can be determined based on a conventional conversion rate assessment model, and the projected conversion rates at both levels can be combined to better assess the projected conversion of the target audience and further influence the specification of subsequent content distribution strategies based on conversion. The embodiments of this disclosure do not limit the specific application of the target projected conversion rate.

[0104] Figure 6 A flowchart of a conversion evaluation process 600 according to some embodiments of the present disclosure is shown. Process 600 can be implemented in a content management system 120.

[0105] In box 610, the content management system 120 extracts resource features from the resource-related data of the target resource.

[0106] In box 620, the content management system 120 extracts the audience characteristics of the target audience from the audience-related data of the target audience of the target resource, and the target audience is to be distributed recommended content items related to the target resource.

[0107] In box 630, the content management system 120 determines a target projected conversion rate for a target resource based on resource features and audience features, through a predetermined correlation between resource features and audience features and projected conversion rate. The target projected conversion rate indicates the projected probability that the target audience will perform a conversion for the target resource.

[0108] In some embodiments, process 600 further includes determining a distribution strategy for recommended content items related to the target resource among the target audience based on the target projected conversion rate.

[0109] In some embodiments, determining a distribution strategy includes: determining a cost adjustment factor based on a target projected conversion rate; adjusting cost data for recommended content items based on the cost adjustment factor; and determining a distribution strategy based on the adjusted cost data.

[0110] In some embodiments, determining the cost adjustment factor includes: determining the historical projected conversion rate for a target resource, the historical projected conversion rate indicating the projected probability that a historical audience of the target resource will perform a conversion for the target resource, the historical audience having been provided with recommended content items related to the target resource during a historical time period; and determining the cost adjustment factor based on the ratio between the target projected conversion rate and the historical projected conversion rate.

[0111] In some embodiments, determining the cost adjustment factor based on the ratio includes: increasing the cost adjustment factor by a first value if the ratio indicates that the target projected conversion rate exceeds the historical projected conversion rate; and decreasing the cost adjustment factor by a second value if the ratio indicates that the target projected conversion rate exceeds the historical projected conversion rate.

[0112] In some embodiments, determining the target projected conversion rate includes: determining the feature cross-result of resource features and audience features; and determining the projected conversion rate for the target resource based on the feature cross-result.

[0113] In some embodiments, the predetermined association between resource features and audience features and the expected conversion rate is represented as a conversion rate prediction model, which is trained based on at least the following: positive training samples, including resource features of the sample resource and audience features of a first sample audience group for the sample resource, the first sample audience group being distributed sample recommended content items related to the sample resource and being marked as having performed a conversion for the sample resource; and negative training samples, including resource features of the sample resource and audience features of a second sample audience group randomly selected from the audience set of the sample resource.

[0114] In some embodiments, the conversion rate prediction model is also trained based on event labels of training samples, whereby the event labels indicate the event type of conversion for the first sample audience. In some embodiments, the training objective of the conversion rate prediction model is configured to update the parameter values ​​of the conversion rate prediction model based at least on the event type.

[0115] In some embodiments, process 600 further includes: obtaining resource-related data and audience-related data based on authorization from the supplier of the target resource and the target audience.

[0116] Figure 7Block diagrams of apparatus for conversion evaluation according to other embodiments of the present disclosure are shown. Apparatus 700 may be implemented as or included in content management system 120. Various modules / components in apparatus 700 may be implemented by hardware, software, firmware, or any combination thereof.

[0117] As shown in the figure, the device 700 includes a resource feature extraction module 710, configured to extract resource features from resource-related data of the target resource. The device 700 also includes an audience characteristic extraction module 720, configured to extract audience characteristics of the target audience from audience-related data of the target audience of the target resource, wherein the target audience is to be distributed with recommended content items related to the target resource. The device 700 further includes a conversion rate determination module 730, configured to determine a target expected conversion rate for the target resource based on resource features and audience features, through a predetermined correlation between resource features, audience features, and expected conversion rate. The target expected conversion rate indicates the expected probability that the target audience will convert for the target resource.

[0118] In some embodiments, although not shown, the apparatus 700 further includes a distribution strategy determination module configured to determine a distribution strategy for recommended content items related to the target resource among a target audience based on a target projected conversion rate.

[0119] In some embodiments, the distribution strategy determination module includes: a coefficient determination module configured to determine a cost adjustment coefficient based on a target projected conversion rate; a cost adjustment module configured to adjust cost data for recommended content items based on the cost adjustment coefficient; and a cost-based strategy determination module configured to determine a distribution strategy based on the adjusted cost data.

[0120] In some embodiments, the coefficient determination module includes: a historical conversion rate determination module configured to determine a historical projected conversion rate for a target resource, the historical projected conversion rate indicating the projected probability that a historical audience of the target resource will perform a conversion for the target resource, the historical audience having been provided with recommended content items related to the target resource during a historical time period; and a ratio-based coefficient determination module configured to determine a cost adjustment coefficient based on the ratio between the target projected conversion rate and the historical projected conversion rate.

[0121] In some embodiments, the ratio-based coefficient determination module includes: a coefficient increase module configured to increase the cost adjustment coefficient by a first value if the ratio indicates that the target expected conversion rate exceeds the historical expected conversion rate; and a coefficient decrease module configured to decrease the cost adjustment coefficient by a second value if the ratio indicates that the target expected conversion rate exceeds the historical expected conversion rate.

[0122] In some embodiments, the conversion rate determination module 730 includes: a feature cross-referencing module configured to determine the feature cross-referencing result of resource features and audience features; and a feature cross-referencing-based conversion rate determination module configured to determine the expected conversion rate for a target resource based on the feature cross-referencing result.

[0123] In some embodiments, the predetermined association between resource features and audience features and the expected conversion rate is represented as a conversion rate prediction model, which is trained based on at least the following: positive training samples, including resource features of the sample resource and audience features of a first sample audience group for the sample resource, the first sample audience group being distributed sample recommended content items related to the sample resource and being marked as having performed a conversion for the sample resource; and negative training samples, including resource features of the sample resource and audience features of a second sample audience group randomly selected from the audience set of the sample resource.

[0124] In some embodiments, the conversion rate prediction model is also trained based on event labels of training samples, whereby the event labels indicate the event type of conversion for the first sample audience. In some embodiments, the training objective of the conversion rate prediction model is configured to update the parameter values ​​of the conversion rate prediction model based at least on the event type.

[0125] In some embodiments, the apparatus 700 further includes a data acquisition module configured to acquire resource-related data and audience-related data based on authorization from the supplier of the target resource and the target audience.

[0126] Figure 8 A block diagram of an electronic device 800 in which one or more embodiments of the present disclosure may be implemented is shown. It should be understood that... Figure 8 The electronic device 800 shown is merely exemplary and should not be construed as limiting the functionality and scope of the embodiments described herein. Figure 8 The electronic device 800 shown can be used to achieve Figure 1 Content management system 120.

[0127] like Figure 8 As shown, electronic device 800 is in the form of a general-purpose computing device. Components of electronic device 800 may include, but are not limited to, one or more processors or processing units 810, memory 820, storage device 830, one or more communication units 840, one or more input devices 850, and one or more output devices 860. Processing unit 810 may be a physical or virtual processor and is capable of performing various processes according to programs stored in memory 820. In a multiprocessor system, multiple processing units execute computer-executable instructions in parallel to improve the parallel processing capability of electronic device 800.

[0128] Electronic device 800 typically includes multiple computer storage media. Such media can be any available media accessible to electronic device 800, including but not limited to volatile and non-volatile media, removable and non-removable media. Memory 820 can be volatile memory (e.g., registers, cache, random access memory (RAM)), non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. Storage device 830 can be a removable or non-removable medium and can include machine-readable media, such as flash drives, disks, or any other media that can be used to store information and / or data (e.g., training data for training) and can be accessed within electronic device 800.

[0129] Electronic device 800 may further include additional removable / non-removable, volatile / non-volatile storage media. Although not explicitly stated... Figure 8 As shown, disk drives for reading from or writing to removable, non-volatile disks (e.g., "floppy disks") and optical disk drives for reading from or writing to removable, non-volatile optical disks can be provided. In these cases, each drive can be connected to a bus (not shown) via one or more data media interfaces. Memory 820 may include computer program product 825 having one or more program modules configured to perform various methods or actions of various embodiments of this disclosure.

[0130] The communication unit 840 enables communication with other electronic devices via a communication medium. Additionally, the functionality of the components of the electronic device 800 can be implemented using a single computing cluster or multiple computing machines capable of communicating via communication connections. Therefore, the electronic device 800 can operate in a networked environment using logical connections to one or more other servers, network personal computers (PCs), or another network node.

[0131] Input device 850 can be one or more input devices, such as a mouse, keyboard, trackball, etc. Output device 860 can be one or more output devices, such as a monitor, speaker, printer, etc. Electronic device 800 can also communicate with one or more external devices (not shown) via communication unit 840 as needed. These external devices include storage devices, display devices, etc., and can communicate with one or more devices that enable user interaction with electronic device 800, or with any device that enables electronic device 800 to communicate with one or more other electronic devices (e.g., network card, modem, etc.). Such communication can be performed via input / output (I / O) interface (not shown).

[0132] According to an exemplary implementation of this disclosure, a computer-readable storage medium is provided that stores computer-executable instructions thereon, wherein the computer-executable instructions are executed by a processor to implement the methods described above. According to an exemplary implementation of this disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, which are executed by a processor to implement the methods described above.

[0133] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatuses, devices, and computer program products implemented according to this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0134] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0135] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions that execute on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0136] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0137] Various implementations of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed implementations. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described implementations. The terminology used herein is chosen to best explain the principles, practical applications, or improvements to technology in the market, or to enable others skilled in the art to understand the various implementations disclosed herein.

Claims

1. A method for conversion evaluation, comprising: Extract resource features from resource-related data of the target resource, wherein the target resource includes the object being promoted; The audience characteristics of the target audience are extracted from the audience-related data of the target audience of the target resource. The target audience is to be distributed with recommended content items, wherein the recommended content items are used to recommend the target resource. as well as Based on the resource features and the audience features, a target expected conversion rate for the target resource is determined through a predetermined correlation between the resource features, the audience features, and the expected conversion rate. The target expected conversion rate indicates the expected probability that the target audience will perform a conversion for the target resource. The predetermined correlation is represented as a conversion rate prediction model. The conversion rate prediction model is configured to obtain feature cross-results of the audience features and the resource features based on the resource features and the audience features to characterize the target audience's preference for the target resource, and to determine the target expected conversion rate based on the feature cross-results. The conversion rate prediction model is trained on conversion data that includes attribution data and non-attribution data. The attribution data refers to the conversion that occurs when a user clicks on the recommended content item related to the target resource. The non-attribution data refers to the conversion that cannot be attributed to the recommended content item. The non-attribution data includes conversions attributed to recommended content items distributed on other content distribution platforms or conversions performed spontaneously by the user.

2. The method according to claim 1, further comprising: Based on the target projected conversion rate, a distribution strategy for recommended content items related to the target resource within the target audience is determined.

3. The method of claim 2, wherein determining the distribution strategy comprises: The cost adjustment factor is determined based on the target projected conversion rate. The cost data for the recommended content item is adjusted based on the cost adjustment coefficient. as well as The distribution strategy is determined based on the adjusted cost data.

4. The method of claim 3, wherein determining the cost adjustment factor comprises: Determine the historical projected conversion rate for the target resource, which indicates the projected probability that a historical audience of the target resource will perform a conversion for the target resource, the historical audience having been provided with recommended content items related to the target resource during a historical time period; as well as The cost adjustment factor is determined based on the ratio between the target projected conversion rate and the historical projected conversion rate.

5. The method of claim 4, wherein determining the cost adjustment factor based on the ratio comprises: If the ratio indicates that the target projected conversion rate exceeds the historical projected conversion rate, the cost adjustment factor is increased by a first value; as well as If the ratio indicates that the target projected conversion rate exceeds the historical projected conversion rate, the cost adjustment factor is reduced by a second value.

6. The method of claim 1, wherein the conversion rate prediction model is trained based on at least the following: The training samples include resource features of the sample resources and audience features of a first sample audience group targeting the sample resources. The first sample audience group is distributed with sample recommendation content items related to the sample resources and is marked as having performed a conversion targeting the sample resources. Negative training samples include the resource features of the sample resources and the audience features of a second sample audience group randomly selected from the audience group set of the sample resources.

7. The method of claim 6, wherein the conversion rate prediction model is further trained based on: event labels of the positive training samples, the event labels indicating the event type of conversion for the first sample audience, and The training objective of the conversion rate prediction model is configured to update the parameter values ​​of the conversion rate prediction model based at least on the event type.

8. The method according to claim 1, further comprising: Based on the authorization of the supplier of the target resource and the target audience, obtain the resource-related data and the audience-related data.

9. An apparatus for translational evaluation, comprising: The resource feature extraction module is configured to extract resource features from resource-related data of a target resource, wherein the target resource includes the object being promoted; The audience feature extraction module is configured to extract the audience features of the target audience from the audience-related data of the target audience of the target resource, wherein the target audience is to be distributed with recommended content items, and wherein the recommended content items are used to recommend the target resource; as well as A conversion rate determination module is configured to determine a target expected conversion rate for the target resource based on the resource features and the audience features, through a predetermined correlation between the resource features, the audience features, and the expected conversion rate. The target expected conversion rate indicates the expected probability that the target audience will perform a conversion for the target resource. The predetermined correlation is represented as a conversion rate prediction model. The conversion rate prediction model is configured to obtain feature cross-results of the audience features and the resource features based on the resource features and the audience features to characterize the target audience's preference for the target resource, and to determine the target expected conversion rate based on the feature cross-results. The conversion rate prediction model is trained at least on conversion data including attribution data and non-attribution data. The attribution data refers to conversions that occur when a user clicks on a recommended content item related to the target resource. The non-attribution data refers to conversions that cannot be attributed to the recommended content item, including conversions attributed to recommended content items distributed on other content distribution platforms or conversions performed spontaneously by the user.

10. The apparatus according to claim 9, further comprising: The distribution strategy determination module is configured to determine the distribution strategy of recommended content items related to the target resource in the target audience based on the target expected conversion rate.

11. The apparatus of claim 10, wherein the distribution strategy determination module comprises: The coefficient determination module is configured to determine the cost adjustment coefficient based on the target projected conversion rate; The cost adjustment module is configured to adjust the cost data for the recommended content item based on the cost adjustment coefficient. as well as The cost-based strategy determination module is configured to determine the distribution strategy based on the adjusted cost data.

12. The apparatus of claim 11, wherein the coefficient determining module comprises: The historical conversion rate determination module is configured to determine the historical projected conversion rate for the target resource, wherein the historical projected conversion rate indicates the projected probability that the historical audience of the target resource will perform a conversion for the target resource, and the historical audience has been provided with recommended content items related to the target resource during the historical time period; as well as The ratio-based coefficient determination module is configured to determine the cost adjustment coefficient based on the ratio between the target projected conversion rate and the historical projected conversion rate.

13. The apparatus of claim 12, wherein the ratio-based coefficient determination module comprises: The coefficient increment module is configured to increase the cost adjustment coefficient by a first value if the ratio indicates that the target projected conversion rate exceeds the historical projected conversion rate; as well as The coefficient reduction module is configured to reduce the cost adjustment coefficient by a second value if the ratio indicates that the target projected conversion rate exceeds the historical projected conversion rate.

14. The apparatus of claim 9, wherein the conversion rate prediction model is trained based on at least the following: The training samples include resource features of the sample resources and audience features of a first sample audience group targeting the sample resources. The first sample audience group is distributed with sample recommendation content items related to the sample resources and is marked as having performed a conversion targeting the sample resources. Negative training samples include the resource features of the sample resources and the audience features of a second sample audience group randomly selected from the audience group set of the sample resources.

15. The apparatus of claim 14, wherein the conversion rate prediction model is further trained based on: event labels of the positive training samples, the event labels indicating the event type of conversion for the first sample audience, and The training objective of the conversion rate prediction model is configured to update the parameter values ​​of the conversion rate prediction model based at least on the event type.

16. The apparatus of claim 9, further comprising: The data acquisition module is configured to acquire resource-related data and audience-related data based on the authorization of the supplier of the target resource and the target audience.

17. An electronic device comprising: At least one processing unit; as well as At least one memory, coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions causing the device to perform the method according to any one of claims 1 to 8 when executed by the at least one processing unit.

18. A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method according to any one of claims 1 to 8.

Images