Content delivery control methods and related equipment

Abstract

This application discloses a content delivery control method and related equipment, comprising: acquiring delivery feedback data and delivery consumption data of target content; determining the tag quality level corresponding to each content tag of the target content based on the delivery feedback records and delivery consumption records under each content tag of the target content; adjusting the level of each content tag according to the tag quality level corresponding to each content tag to obtain the adjusted level of each content tag; determining a filtering threshold based on the adjusted levels of all content tags of the target content; and controlling the delivery of the target content based on the filtering threshold and the quality score of the target content, wherein if the content quality score of the target content is not lower than the filtering threshold, the target content is delivered; if the content quality score of the target content is lower than the filtering threshold, the target content is filtered out. This solution can flexibly adjust the exposure probability of the target content.

Description

Technical Field

[0001] This application relates to the field of computer technology, and more specifically, to a content delivery control method and related equipment. Background Technology

[0002] To improve the efficiency of content acquisition for users, content is delivered to users that is highly relevant to their needs, thereby reducing the time users spend filtering and selecting content. However, the content delivery process in this technology suffers from low flexibility. Summary of the Invention

[0003] In view of the above problems, this application proposes a content delivery control method and related equipment to improve the above problems.

[0004] According to one aspect of the embodiments of this application, a content delivery control method is provided, comprising: acquiring delivery feedback data and delivery consumption data of target content; the delivery feedback data including delivery feedback records under each content tag of the target content; the delivery consumption data including delivery consumption records under each content tag of the target content; determining the tag quality level corresponding to each content tag of the target content based on the delivery feedback records and delivery consumption records under each content tag of the target content; adjusting the level of each content tag according to the tag quality level corresponding to each content tag to obtain the adjusted level of each content tag; determining a filtering threshold based on the adjusted levels of all content tags of the target content; and performing delivery control on the target content based on the filtering threshold and the quality score of the target content, wherein if the content quality score of the target content is not lower than the filtering threshold, the target content is delivered; and if the content quality score of the target content is lower than the filtering threshold, the target content is filtered out.

[0005] According to one aspect of the embodiments of this application, a content delivery control device is provided, comprising: an acquisition module, configured to acquire delivery feedback data and delivery consumption data of target content; the delivery feedback data includes delivery feedback records under each content tag of the target content; the delivery consumption data includes delivery consumption records under each content tag of the target content; a tag quality level determination module, configured to determine the tag quality level corresponding to each content tag of the target content based on the delivery feedback records and delivery consumption records under each content tag of the target content; a level adjustment module, configured to adjust the level of each content tag according to the tag quality level corresponding to each content tag, to obtain the adjusted level of each content tag; a filtering threshold determination module, configured to determine a filtering threshold based on the adjusted levels of all content tags of the target content; and a delivery control module, configured to perform delivery control on the target content based on the filtering threshold and the quality score of the target content, wherein if the content quality score of the target content is not lower than the filtering threshold, the target content is delivered; and if the content quality score of the target content is lower than the filtering threshold, the target content is filtered out.

[0006] In some embodiments of this application, the tag quality level determination module includes: a feedback parameter calculation unit, configured to calculate feedback parameters under each content tag of the target content based on the delivery feedback records under each content tag of the target content; a consumption parameter calculation unit, configured to calculate consumption parameters under each content tag of the target content based on the delivery consumption records under each content tag of the target content; and a tag quality level determination unit, configured to determine the tag quality level corresponding to each content tag of the target content based on the correspondence between tag quality level and feedback parameters and consumption parameters, as well as the feedback parameters and consumption parameters under each content tag of the target content.

[0007] In some embodiments of this application, the tag quality level includes high-quality tags and low-quality tags; the level adjustment module is further configured to: if the tag quality level corresponding to the content tag is a high-quality tag, then reduce the level of the content tag to obtain the adjusted level of the content tag; if the tag quality level corresponding to the content tag is low-quality, then increase the level of the content tag to obtain the adjusted level of the content tag.

[0008] In some embodiments of this application, the filtering threshold determination module includes: a quality score determination unit, configured to determine the quality score corresponding to the adjusted level of each content tag according to the correspondence between the level and the quality score; and a filtering threshold determination unit, configured to determine the minimum quality score among the quality scores corresponding to the adjusted level of each content tag, and determine the minimum quality score as the filtering threshold.

[0009] In some embodiments of this application, the quality score of the target content includes the quality score of the target content relative to each candidate user; the delivery control module includes: an interaction probability acquisition unit, used to acquire the interaction probability of a target candidate user interacting with the target content; a quality score determination unit, used to determine the quality score of the target content relative to the target candidate user based on the interaction probability; an adding unit, used to send the target content to the target candidate user if a content request initiated by the target candidate user is received, if the quality score of the target content relative to the target candidate user is not lower than the filtering threshold; and a filtering unit, used to filter out the target content if the quality score of the target content relative to the target candidate user is lower than the filtering threshold.

[0010] In some embodiments of this application, the content delivery control device further includes: a tag matching degree calculation unit, used to calculate the tag matching degree between the content tags of the target content and the user tags of each user in the user set; and a candidate user determination unit, used to determine users whose tag matching degree exceeds the matching degree threshold as candidate users of the target content.

[0011] In some embodiments of this application, the interaction probability acquisition unit includes: an acquisition unit, configured to acquire user information of the target candidate user and content information of the target content; and an interaction probability determination unit, configured to output the interaction probability of the target candidate user triggering an interactive behavior on the target content based on the user information of the target candidate user and the content information of the target content, using an interaction probability prediction model.

[0012] In some embodiments of this application, the interactive behavior includes a first interactive behavior and a second interactive behavior arranged chronologically from first to last; the interaction probability includes a first interaction probability and a second interaction probability, wherein the first interaction probability refers to the probability of triggering the first interactive behavior, and the second interaction probability refers to the probability of triggering the second interactive behavior; the interaction probability prediction model includes a shared embedding layer, a first sub-neural network, and a second sub-neural network; the interaction probability determination unit includes: an embedding feature determination unit, used by the shared embedding layer to generate user embedding features of the target candidate user and content embedding features of the target content based on user information of the target candidate user and content information of the target content; a first prediction unit, used by the first sub-neural network to perform probability prediction based on user embedding features of the target candidate user and content embedding features of the target content to obtain a first interaction probability; and a second prediction unit, used by the second sub-neural network to perform probability prediction based on user embedding features of the target candidate user, content embedding features of the target content, and the first interaction probability to obtain a second interaction probability.

[0013] In other embodiments of this application, the interactive behavior further includes a third interactive behavior, wherein the third interactive behavior is later than the second interactive behavior in terms of timing; the interactive probability further includes a third interactive probability, which refers to the probability of triggering the third interactive behavior; the interactive probability prediction model further includes a third sub-neural network; the interactive probability determination unit further includes: a third prediction unit, used by the third sub-neural network to perform probability prediction based on the user embedding features of the target candidate user, the content embedding features of the target content, and the second interactive probability to obtain the third interactive probability.

[0014] In some embodiments of this application, the content delivery control device further includes: a training data acquisition module, used to acquire training data, the training data including multiple training samples, the training samples including user information of sample users, content information of sample content, and interaction tags, the interaction tags including a first interaction tag, a second interaction tag, and a third interaction tag, the first interaction tag being used to indicate whether the sample user triggers a first interaction behavior on the sample content, the second interaction tag being used to indicate whether the sample user triggers a second interaction behavior on the sample content, and the third interaction tag being used to indicate whether the sample user triggers a third interaction behavior on the sample content; an embedding feature generation module, used by the shared embedding layer to generate user embedding features of the sample user and content embedding features of the sample content according to the user information of the sample user and the content information of the sample content; a first predicted interaction probability determination module, used by the first sub-neural network to perform probability prediction based on the user embedding features of the sample user and the content embedding features of the sample content to obtain a first predicted interaction probability corresponding to the training sample; and a second... The system includes a prediction interaction probability determination module, which uses the second sub-neural network to predict the second predicted interaction probability corresponding to the training sample by performing probability prediction based on the user embedding features of the sample user, the content embedding features of the sample content, and the first predicted interaction probability; a third prediction interaction probability determination module, which uses the third sub-neural network to predict the third predicted interaction probability corresponding to the training sample by performing probability prediction based on the user embedding features of the sample user, the content embedding features of the sample content, and the second predicted interaction probability; a first loss calculation module, which calculates a first loss based on the first predicted interaction probability and the first interaction label; a second loss calculation module, which calculates a second loss based on the second predicted interaction probability and the second interaction label; a third loss calculation module, which calculates a third loss based on the third predicted interaction probability and the third interaction label; a target loss determination module, which determines a target loss based on the first loss, the second loss, and the third loss; and a reverse adjustment module, which reversely adjusts the parameters of the interaction probability prediction model based on the target loss until the training termination condition is met.

[0015] According to one aspect of the embodiments of this application, an electronic device is provided, including: a processor; a memory, the memory storing computer-readable instructions, which, when executed by the processor, implement the delivery control method described above.

[0016] According to one aspect of the embodiments of this application, a computer-readable storage medium is provided, on which computer-readable instructions are stored, which, when executed by a processor, implement the delivery control method described above.

[0017] According to one aspect of the embodiments of this application, a computer program product is provided, including computer instructions that, when executed by a processor, implement the delivery control method described above.

[0018] In this application, the tag quality level of each content tag is determined by combining the delivery feedback records and delivery consumption records of the target content under each content tag, and then a filtering threshold for controlling the delivery of the target content is determined. This ensures that the target content is delivered when its content quality score is not lower than the filtering threshold, and filtered out when its content quality score is lower than the filtering threshold. It is determined that delivering or filtering the target content directly affects the exposure probability of the target content. Therefore, this solution achieves flexible and dynamic adjustment of the exposure probability of the target content by combining the tag quality level of the content tags, thus flexibly controlling the delivery of the target content. Moreover, this application uses a probability smoothing method to adjust the exposure probability of the target content, which can avoid drastic fluctuations in the exposure probability of the content. Attached Figure Description

[0019] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0020] Figure 1 This is a schematic diagram illustrating an application scenario of a content delivery control method according to an embodiment of this application.

[0021] Figure 2 This is a flowchart illustrating a content delivery control method according to an embodiment of this application.

[0022] Figure 3 This is a schematic diagram illustrating the matching of user tags with advertising content tags according to one embodiment.

[0023] Figure 4 This is a flowchart illustrating step 250 according to an embodiment of this application.

[0024] Figure 5 This is a flowchart illustrating step 410 according to an embodiment of this application.

[0025] Figure 6 This is a flowchart illustrating step 520 according to an embodiment of this application.

[0026] Figure 7This is a flowchart illustrating step 520 according to another embodiment of this application.

[0027] Figure 8A This is a schematic diagram of an interaction probability prediction model according to an embodiment of this application.

[0028] Figure 8B This is a schematic diagram of a twin-tower model according to an embodiment of this application.

[0029] Figure 9 This is a flowchart illustrating a content delivery control method according to another embodiment of this application.

[0030] Figure 10 This is a block diagram of a delivery control device according to an embodiment of this application.

[0031] Figure 11 A schematic diagram of the structure of a computer system suitable for implementing the electronic device of the present application is shown. Detailed Implementation

[0032] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make this application more comprehensive and complete, and to fully convey the concept of the exemplary embodiments to those skilled in the art.

[0033] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this application. However, those skilled in the art will recognize that the technical solutions of this application can be practiced without one or more of the specific details, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this application.

[0034] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0035] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0036] It should be noted that "multiple" in this article refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0037] Figure 1 This is a schematic diagram illustrating an application scenario of a content delivery control method according to an embodiment of this application. For example... Figure 1 As shown, the application scenario includes terminal 111 and server 112, and terminal 111 and server 112 establish a communication connection through wired or wireless network.

[0038] Terminal 111 can be a smartphone, tablet, laptop, desktop computer, smart speaker, in-vehicle terminal, smart TV, or other interactive electronic device, without specific limitations.

[0039] Server 112 can be a standalone physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0040] Server 112 can be used to execute the method of this application to control the delivery of content, so as to deliver the content to the user's terminal 111. In this solution, controlling the delivery of target content means determining whether to deliver the target content to the user's terminal.

[0041] In some embodiments, the server 112 may determine the target content to be delivered to the user's terminal according to the method of this application, and associate and store the user's user identifier with the content identifier of the target content to be delivered, so that after the user's terminal 111 initiates a content request, the server 112 sends the target content to be delivered to the user to the terminal 111.

[0042] In some embodiments, after receiving a content request sent by the terminal 111, the server 112 may determine whether to send the target content to the terminal 111 according to the method of this application. That is, if the quality score of the target content is determined to be not lower than the filtering threshold, the target content will be sent to the terminal 111; otherwise, if the quality score of the target content is determined to be lower than the filtering threshold, the target content will be filtered, that is, the target content will not be sent to the terminal 111.

[0043] In some embodiments, a content database 123 may also be set up in the server 112 to store the content to be delivered. Of course, in other embodiments, the content database 123 may also be set up independently of the server 112.

[0044] In some embodiments, the server 112 can obtain the delivery feedback data and delivery consumption data of the target content in a set statistical period, and control the delivery of the target content in the next statistical period according to the method of this application, thereby realizing flexible control of the delivery of the target content based on the delivery effect of the target content.

[0045] The implementation details of the technical solutions in the embodiments of this application are described in detail below:

[0046] Figure 2 A flowchart illustrating a content delivery control method according to an embodiment of this application is shown. This method can be executed by a computer device with processing capabilities, such as a server, and is not specifically limited thereto. (Refer to...) Figure 2 As shown, the method includes at least steps 210 to 250, which are described in detail below:

[0047] Step 210: Obtain the delivery feedback data and delivery consumption data of the target content; the delivery feedback data includes the delivery feedback records under each content tag of the target content; the delivery consumption data includes the delivery consumption records under each content tag of the target content.

[0048] The target content can refer to any content that needs to be controlled for delivery. Target content can be news, videos (long videos, short videos, etc.), audio (music), blog articles, WeChat official account articles, product links, merchant links, advertisements, e-books, etc., without specific limitations.

[0049] Delivery feedback records refer to the records generated based on user interactions after targeted content is sent to the user. These records indicate the user's interactive behavior with the targeted content. Interactive behaviors can include clicking on the targeted content, adding it to favorites, placing an order (purchase), repurchasing, commenting, sharing, liking, viewing (reading or staying on the target content's page) for a set duration, sending comments, etc., with specific definitions provided here. For example, if the target content is a merchant's advertisement, a click record is generated when a user clicks on the advertisement; a purchase record is generated when the user clicks on the merchant's advertisement, enters the merchant's page, and purchases a product.

[0050] Understandably, the interactive behaviors that can be triggered for target content differ across different application scenarios. Furthermore, the campaign feedback log can also indicate the user ID corresponding to the user who triggered the interactive behavior.

[0051] Delivery cost records are used to indicate the costs incurred in delivering target content. In some embodiments, content delivery is typically done through a media platform. In this case, the owner of the target content needs to pay a certain fee to the media platform's administrator. The fee paid by the owner of the target content to the media platform can be considered as the cost incurred in delivering the target content.

[0052] Content tags for target content refer to the tags affixed to target content to match it with users. Content tags, also known as TAGs, provide a concise description of a specific aspect of the target content, highlighting its unique characteristics. Content tags can include style tags, domain tags, audience tags, attribute tags, category tags, and keywords, among others. For example, if the target content is an advertisement for a product, the content tags for that advertisement could be the product category (e.g., high heels), product keywords (e.g., brown, pointed-toe shoes), target audience tags (e.g., women, 18-30 years old), and style tags (e.g., British style).

[0053] As can be seen from the above, the target content may have multiple content tags. In this application, the delivery feedback records and delivery consumption records of the target content are classified according to the content tags, so as to obtain the delivery feedback record and delivery consumption record corresponding to each content tag of the target content.

[0054] In the scheme of this application, the delivery feedback record and the delivery consumption record are associated with at least one content tag of the target content. Therefore, the delivery feedback record and the delivery consumption record can be classified according to the content tag based on the associated content tag.

[0055] In some embodiments, the content tag corresponding to the user indicated by the delivery feedback record can be used as the content tag associated with the delivery feedback record. The content tag corresponding to the user refers to the content tag that matches the user tag determined to be the user tag. For example, if the user tag of user P is matched with the content tag of content Q, and it is determined that the user tag A1 of user P matches the content tag B1 of content Q, then content tag B1 is the content tag associated with the delivery feedback record generated based on the user P's interaction behavior.

[0056] In some embodiments, media platforms typically charge based on impressions, clicks, or conversions. Therefore, the cost of content delivery is determined after the target content is delivered to the user. Since the delivery cost record is related to the target content's delivery history or delivery feedback history, the content tag corresponding to the user indicated by the delivery record or the content tag corresponding to the user indicated by the delivery feedback record can be used as the content tag associated with that delivery cost record. The content tag corresponding to the user indicated by the delivery record refers to the content tag that matches the user tag indicated by the delivery record.

[0057] In some embodiments of this application, multiple paths may be set to match user tags with content tags. That is, user tags are set for each path, and the user tags set for different paths may have the same user tags. In this case, the relevant delivery feedback records (or delivery consumption records) matched with the target content tags under multiple paths can be aggregated to ensure the accuracy of the delivery feedback records (or delivery consumption records) obtained under each content tag.

[0058] Figure 3 This is a schematic diagram illustrating the matching of user tags with advertisement content tags according to one embodiment. For example... Figure 3 As shown, user tags (such as...) are set under five paths: behavioral targeting path, interest targeting path, intention targeting path, APP (Application) installation targeting path, and custom audience path. Figure 3 The system uses categories, keywords, installed lists, and DMP (Data Management Platform) audience lists to match user tags across all paths. For example, if an ad's content tag (also known as an ad tag) includes the tag "martial arts game enthusiast," and a user making a request also has that tag in their user tag, then that user is a match for the ad.

[0059] Among them, behavioral targeting path and interest targeting path refer to the use of an existing set of behavioral targeting tagging system and interest targeting tagging system to tag content according to the behavioral targeting tagging system and interest targeting tagging system, so as to recall directly matching content when requesting.

[0060] App installation targeting refers to finding other similar apps for a specific app (let's call it the target app). If a user has installed the target app, when that user requests it, the app will retrieve content from other similar apps that the user has installed (such as ads for those other apps).

[0061] Intent targeting path refers to automatically binding targeting tags with high TGI (Target Group Index) among converted users. A high TGI targeting tag means that this targeting tag has a large proportion of users among converted users. Custom audience targeting path refers to automatically binding custom audience targeting tags.

[0062] When multiple paths are set up to match user tags with content tags, the content recalled through each path (i.e., the content that matches the user tags as determined by the content tags) is merged together. Each piece of recalled content is marked with which content tags were used to recall it. Therefore, in this case, the delivery feedback record and delivery consumption record can be statistically recorded according to the content tags.

[0063] For example, when the target content is an advertisement, the delivery feedback records of each advertisement can be obtained by statistically analyzing the content tags (where Tagij refers to the content tag that plays a role in content recall under the j-th path being the i-th content tag):

[0064] Ad1 (Ad 1): Tag11 Tag12 Tag13 …

[0065] Ad2: Tag21 Tag22 Tag23 …

[0066] Ad3: Tag31 Tag32 Tag33 …

[0067] Step 220: Determine the tag quality level corresponding to each content tag of the target content based on the delivery feedback records and delivery consumption records under each content tag of the target content.

[0068] Tag quality level is used to reflect the quality of content tags. Since the tag quality level is determined based on the delivery feedback record and delivery consumption record, it can be seen that in this solution, the quality of content tags is reflected from two perspectives: content delivery effect and delivery consumption.

[0069] In some embodiments of this application, step 220 includes: calculating feedback parameters for each content tag of the target content based on the delivery feedback records for each content tag of the target content; calculating consumption parameters for each content tag of the target content based on the delivery consumption records for each content tag of the target content; and determining the tag quality level corresponding to each content tag of the target content based on the correspondence between tag quality level and feedback parameters and consumption parameters, as well as the feedback parameters and consumption parameters for each content tag of the target content.

[0070] In some embodiments, the feedback parameter can be the number of times an interactive behavior is triggered within a unit of time (e.g., within a statistical period). For example, the feedback parameter can be the number of clicks in a day, the number of conversions in a day, etc. In some embodiments, if the statistical campaign feedback records are derived from multiple interactive behaviors, the feedback parameter can be a linear weighted average of the number of all types of interactive behaviors triggered for the target content.

[0071] In some embodiments, the consumption parameter can be the cost consumed per unit time (one statistical period), the cost deviation, or the cost deviation ratio. The cost deviation is equal to the difference between the revenue generated by the targeted content and the cost incurred in delivering the targeted content; the cost deviation ratio is equal to the ratio of the cost incurred in delivering the targeted content to the total cost of delivering the targeted content. For example, if the targeted content is an advertisement for a product, the revenue generated by delivering the targeted content can be the Gross Merchandise Volume (GMV) of the products purchased based on the product advertisement.

[0072] In some embodiments, the label quality level includes high-quality labels and low-quality labels; in other embodiments, the label quality level may also include neutral labels; of course, in other embodiments, the label quality level may be further divided into finer-grained categories, which are not specifically limited here.

[0073] After determining the feedback parameters and consumption parameters under each content tag of the target content, the tag quality level of each content tag of the target content can be determined according to the pre-set correspondence between the tag quality level and the feedback parameters and consumption parameters.

[0074] In some embodiments, in the correspondence between tag quality levels and feedback parameters and consumption parameters, a range of feedback parameters and a range of consumption parameters corresponding to each tag quality level can be set. Thus, the range of consumption parameters where the consumption parameters of a content tag are located and the range of feedback parameters where the feedback parameters of a content tag are located can be determined. The tag quality level corresponding to the range of consumption parameters where the consumption parameters of the content tag are located and the range of feedback parameters where the feedback parameters of the content tag are located are determined as the tag quality level corresponding to the content tag.

[0075] Table 1 below shows the quality label levels corresponding to various consumption parameter ranges and feedback parameter ranges according to a specific embodiment. In the table, the feedback parameter is the conversion number, and the consumption parameter is the cost deviation ratio, i.e., cost / GMV.

[0076]

[0077] Table 1

[0078] Understandably, content tags identified as high-quality tags indicate that they effectively describe the characteristics of the content. In terms of effectiveness, content tagged with this quality tag is more likely to be recalled in appropriate contexts, and users are more interested in the content after recall, indicating good exposure. Conversely, content tags identified as low-quality tags indicate that they fail to adequately describe the characteristics of the content and are lacking in highlighting its features. In terms of effectiveness, adding this tag to the content generally results in only moderate recall.

[0079] Step 230: Adjust the level of each content tag according to the tag quality level corresponding to each content tag to obtain the adjusted level of each content tag.

[0080] In some embodiments of this application, a mapping relationship between tag quality levels and level adjustment strategies can be established, thereby adjusting the level of each content tag according to the level adjustment strategy mapped to the tag quality level corresponding to the content tag. The adjusted level of the content tag refers to the level of the content tag after adjustment.

[0081] In some embodiments of this application, the tag quality level includes a high-quality tag and a low-quality tag; step 230 includes: if the tag quality level corresponding to the content tag is a high-quality tag, then the level of the content tag is reduced to obtain the adjusted level of the content tag; if the tag quality level corresponding to the content tag is low-quality, then the level of the content tag is increased to obtain the adjusted level of the content tag.

[0082] As can be seen in this embodiment, when the quality level of the content tag is a high-quality tag, the level of the content tag is lowered; conversely, when the quality level of the content tag indicates that the content tag is a low-quality tag, the level of the content tag is raised.

[0083] In some embodiments, the label quality level also includes a neutral label. Table 2 below shows the adjustment strategies corresponding to each set label quality level, wherein the values ​​in the adjustment strategies in Table 2 represent the magnitude adjustment coefficient for level adjustment.

[0084]

[0085] Table 2

[0086] For example, if the level adjustment formula is set as follows:

[0087] ;(Formula 1)

[0088] This represents the level of the content tag in the previous period; D has three possible values: -1, 0, and 1, which correspond to different tag quality adjustment strategies. That is, when the content tag is a poor quality tag, D is positive, which increases the level of the content tag; when the content tag is a neutral tag, the level of the content tag remains unchanged; and when the content tag is a high quality tag, the level of the content tag decreases.

[0089] It is understood that the above is merely an exemplary example of adjusting the level according to the label quality level, and should not be considered as the scope of application of this application. In other embodiments, the magnitude adjustment coefficient may be other values, and the level may be adjusted in other non-linear ways (such as according to an exponential function).

[0090] Step 240: Determine the filtering threshold based on the adjusted levels of all content tags of the target content.

[0091] In some embodiments of this application, step 240 includes: determining the quality score corresponding to the adjusted level of each content tag based on the correspondence between the level and the quality score; determining the minimum quality score among the quality scores corresponding to the adjusted level of each content tag, and determining the minimum quality score as the filtering threshold.

[0092] In some instances, the level and quality score can be positively correlated. Understandably, when a content tag is considered high-quality, lowering the content tag's level will correspondingly lower the quality score.

[0093] Table 3 below illustrates the correspondence between grade and quality score according to a specific embodiment. Table 3 shows a positive correlation between quality score and grade.

[0094]

[0095] Table 3

[0096] In some embodiments, the quality scores of historically delivered content can be used to classify content into levels, and the determined levels can be used as optional levels for content tags. For example, sampling 1 million users and calculating the quality scores of content over a period of time for combinations like <content, tag>, then sorting these quality scores from smallest to largest and dividing them into 100 equal parts, each part can be considered a level, as shown in Table 2, including levels 1, 2, ..., 100. In specific embodiments, the number of users sampled can be set according to actual needs. Of course, in specific embodiments, the number of users sampled can be determined by combining the effectiveness and efficiency of level classification. It is understood that the larger the number of users sampled, the higher the accuracy of level classification and quality score grading, but the corresponding data processing volume also increases.

[0097] In a specific embodiment, the level corresponding to a content tag in the current period is determined. Then, the quality score corresponding to the level of the content tag in the current period can be determined according to the following formula. :

[0098] ;(Formula 2)

[0099] In the above formula, the function f(x) represents the mapping relationship between the level and the quality score, such as the mapping relationship presented in Table 3 above.

[0100] The quality score of target content reflects the probability that users will trigger interactive behavior with the target content. The higher the probability that users will trigger interactive behavior with the target content, the higher the quality score of the target content.

[0101] In some embodiments, an interaction probability prediction model can be used to predict the probability of a user triggering an interactive behavior on the target content based on the content information of the target content and the user's user information, and then the quality score of the target content can be determined based on the predicted interaction probability. The interaction probability prediction model can be used to predict the probability of a user triggering one interactive behavior on the target content (e.g., predicting the probability of triggering a click behavior), or it can be used to predict the probability of triggering two or more interactive behaviors on the target content (e.g., the probability of triggering a click behavior and the probability of triggering an order behavior, etc.).

[0102] The interaction probability prediction model can be constructed from one or more neural networks such as fully connected networks, recurrent neural networks, feedforward neural networks, and convolutional neural networks. After training, it can be used to perform interaction probability prediction. In other embodiments, the interaction probability prediction model can also be an existing neural network model that has been trained and then used as the interaction probability prediction model for probability prediction.

[0103] In some embodiments, when the interaction probability prediction model is used to predict the interaction probability of triggering two or more interaction behaviors on the target content, the quality score of the target content can be determined by combining the interaction probabilities of each predicted interaction behavior. For example, the interaction probabilities of all predicted interaction behaviors can be weighted and the weighted result can be used as the quality score of the target content.

[0104] Step 250: Based on the filtering threshold and the quality score of the target content, control the delivery of the target content. If the quality score of the target content is not lower than the filtering threshold, the target content is delivered; if the quality score of the target content is lower than the filtering threshold, the target content is filtered out.

[0105] In this application, if the content quality score of the target content is lower than the filtering threshold, the target content will be filtered out, that is, the target content will not be delivered to the user, and correspondingly, the target content will not be exposed to the user.

[0106] If the minimum quality score among the adjusted quality scores of all content tags is used as the filtering threshold, and the tag quality level indicates that the content tag is a high-quality tag, the content tag's level is lowered, and correspondingly, the quality score of the content tag decreases. Therefore, the determined filtering threshold decreases accordingly. In this case, since the target content's quality score remains essentially unchanged, the probability of the target content being displayed increases due to the lower filtering threshold, effectively increasing the target content's exposure probability. Conversely, if a content tag of the target content is determined to be a low-quality tag, the level corresponding to that content tag is raised, and the quality score of the content tag increases accordingly. The determined filtering threshold increases, and in this case, the probability of the target content being filtered out increases, effectively decreasing the target content's exposure probability.

[0107] In some implementations, steps 210-240 above can be executed periodically. This period can be set according to actual needs, such as 1 hour, 2 hours, 6 hours, or 1 day. In this case, a filtering threshold is determined based on the target content delivery feedback data and delivery consumption data from the previous period. This filtering threshold is then applied to the delivery control of the target content in the current period. This allows for periodic updates to the tag quality level of the content, and flexible adjustments to the exposure probability of the target content based on the updated tag quality level, achieving a smooth processing of the target content's exposure probability.

[0108] In some embodiments, in the first cycle, the level of the content tag can be initialized and set. Based on this, in each cycle, the level corresponding to the content tag is adjusted according to the tag quality level determined by the delivery feedback data and delivery consumption data of the previous cycle.

[0109] In this application, the tag quality level of each content tag is determined by combining the delivery feedback records and delivery consumption records of the target content under each content tag. This, in turn, determines the filtering threshold used to control the delivery of the target content. The target content is delivered when its content quality score is not lower than the filtering threshold, and filtered out when its content quality score is lower. This determines how delivering or filtering the target content directly affects its exposure probability. Therefore, this solution achieves flexible and dynamic adjustment of the target content's exposure probability by combining the tag quality level, enabling flexible control over the delivery of the target content. Furthermore, this application uses a probability smoothing method to adjust the target content's exposure probability, which avoids drastic fluctuations in content exposure.

[0110] Furthermore, the tag quality level of each content tag is determined by analyzing the delivery feedback records and delivery consumption records under each content tag. The delivery feedback records reflect the delivery effect under the content tag, and the delivery consumption records reflect the delivery consumption under the content tag. Therefore, by combining the delivery feedback records reflecting the delivery effect and the delivery consumption records reflecting the delivery consumption, the tag quality level can be determined to ensure a comprehensive reflection of the overall quality of the content tags and to ensure the accuracy and rationality of the determined tag quality level.

[0111] In some embodiments of this application, the quality score of the target content includes the quality score of the target content relative to each candidate user; such as Figure 4 As shown, step 250 includes:

[0112] Step 410: Obtain the interaction probability of the target candidate user interacting with the target content.

[0113] The target candidate user can be any candidate user determined for the target content. In some embodiments, in order to ensure that the content delivered to users is more targeted to them, user screening can be performed in advance based on user tags and content tags of the target content to determine the candidate users for the target content.

[0114] In some embodiments of this application, the method further includes: calculating the tag matching degree between the content tags of the target content and the user tags of each user in the user set; and identifying users whose tag matching degree exceeds the matching degree threshold as candidate users of the target content.

[0115] In some embodiments, the semantic similarity between the content tags of the target content and each user tag of the user can be calculated. If the semantic similarity exceeds a set semantic similarity threshold, it is determined that the content tag of the target content matches the user tag. The tag matching degree can be represented by the number of user tags that match the content tags of the target content. Correspondingly, the matching degree threshold can also be a threshold set for the number of user tags that match the content tags of the target content.

[0116] Users whose tag matching score exceeds a matching score threshold are identified as candidate users for the target content. This is equivalent to identifying the target content as the content to be delivered to that user when the tag matching score exceeds the threshold. Thus, for that user, it's essentially recalling the target content. Therefore, from this perspective, the process of matching user tags with target content tags is the target content recall process.

[0117] By identifying candidate users for the target content through the above process, the subsequent delivery control process can be carried out only for the candidate users of the target content, instead of controlling delivery to all users, which reduces the amount of data processing. Moreover, since users are pre-screened based on user tags and content tags, the matching between content and users can be guaranteed.

[0118] Furthermore, to facilitate subsequent statistical analysis of delivery feedback and consumption records based on content tags, effective content tags can be marked during the recall process of the target content. These marked effective content tags will then be used as the content tags associated with the delivery feedback records (and delivery consumption records). Specifically, the content tags marked as effective during the recall process of the target content refer to the content tags that match the user's user tags identified during the content recall process.

[0119] Step 420: Determine the quality score of the target content relative to the target candidate user based on the interaction probability.

[0120] In some embodiments, a mapping relationship between interaction probability and quality score can be pre-defined, so that the quality score of the target content relative to the target candidate user can be determined according to the obtained interaction probability.

[0121] It is understandable that since the interaction probability is relative to a specific user, and different users have different probabilities of triggering interaction behavior for user content due to differences in their required content or preferences, the quality score of the determined target content relative to the user will also be different for different users. Therefore, in this embodiment, the quality score of the target content calculated based on the interaction probability is relative to the user.

[0122] Step 430: If the quality score of the target content relative to the target candidate user is not lower than the filtering threshold, then if a content request is received from the target candidate user, the target content is sent to the target candidate user.

[0123] Step 440: If the quality score of the target content relative to the target candidate user is lower than the filtering threshold, then the target content is filtered out.

[0124] If the target content is sent to the target candidate users, it will be exposed to them; conversely, if the target content is filtered out, it will not be exposed to them. Thus, through this process, based on the determined filtering threshold and the quality score of the target content relative to the target candidate users, the delivery of target content to target candidate users is controlled, and the exposure probability of the target content is adjusted according to its content tags.

[0125] In some embodiments of this application, such as Figure 5 As shown, step 410 includes:

[0126] Step 510: Obtain user information of the target candidate user and content information of the target content.

[0127] User information can include age, gender, location, zodiac sign, preference tags, etc., without specific limitations. Content information can include content topic, content domain, content title, content category tags, etc. In some embodiments, if the target content is an advertisement, the content information may also include the target audience category set by the advertiser.

[0128] Step 520: The interaction probability prediction model outputs the interaction probability of the target candidate user triggering interactive behavior on the target content based on the user information of the target candidate user and the content information of the target content.

[0129] Interaction probability prediction models can be built using neural networks such as fully connected networks and recurrent neural networks. Understandably, to ensure the accuracy of the interaction probabilities predicted by the interaction probability prediction model, it can first be trained offline, and then the trained model can be applied online to predict interaction probabilities.

[0130] In this embodiment, an interaction probability prediction model is used to predict the interaction probability, thereby applying artificial intelligence technology to the content delivery control process.

[0131] Artificial intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0132] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0133] It is understood that in the specific implementation of this application, data related to user information (such as age, gender, location, preference tags, etc.) are involved. When the embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0134] In some embodiments of this application, the interactive behavior includes a first interactive behavior and a second interactive behavior arranged chronologically from first to last; the interaction probability includes a first interaction probability and a second interaction probability, wherein the first interaction probability refers to the probability of triggering the first interactive behavior, and the second interaction probability refers to the probability of triggering the second interactive behavior; the interaction probability prediction model includes a shared embedding layer, a first sub-neural network, and a second sub-neural network; in this embodiment, such as Figure 6 As shown, step 520 includes:

[0135] Step 610: The shared embedding layer generates user embedding features of the target candidate user and content embedding features of the target content based on the user information of the target candidate user and the content information of the target content.

[0136] Step 620: The first sub-neural network performs probability prediction based on the user embedding features of the target candidate user and the content embedding features of the target content to obtain the first interaction probability.

[0137] Step 630: The second sub-neural network performs probability prediction based on the user embedding features of the target candidate user, the content embedding features of the target content, and the first interaction probability to obtain the second interaction probability.

[0138] The second interactive action is later than the first interactive action in sequence. It can also be understood that if a user triggers a second interactive action on the content, it means that the user must have triggered a first interactive action on the content. In other words, the first interactive action is a necessary condition for triggering the second interactive action.

[0139] For example, if the target content is a product advertisement, the first interactive behavior could be clicking, and the second interactive behavior could be placing an order; if the target content is a video, the first interactive behavior could be clicking, and the second interactive behavior could be the video playback time reaching the first duration, commenting on the video, sending bullet comments, saving the video, sharing the video, etc.

[0140] The shared embedding layer is used to transform the user information of the target user and the content information of the target content into the same embedding space, thereby obtaining the feature representation of the user information of the target user in the embedding space (i.e., the user embedding feature of the target user) and the feature representation of the content information of the target content in the embedding space (i.e., the content embedding feature of the target content).

[0141] In some embodiments, the first sub-neural network and the second sub-neural network may be constructed by one or more neural networks, such as fully connected networks, recurrent neural networks, feedforward neural networks, etc., without specific limitations.

[0142] In this embodiment, since the first interactive behavior is a necessary condition for triggering the second interactive behavior, the first interactive probability that triggers the first interactive behavior is used as one of the inputs of the second sub-neural network, so that the second sub-neural network can predict the second interactive probability by referring to the first interactive probability.

[0143] In this embodiment, the first interaction probability and the second interaction probability of the target candidate user triggering the first interaction behavior and the second interaction behavior of the target content are predicted. Correspondingly, when determining the quality score of the target content relative to the target candidate user, the first interaction probability and the second interaction probability are also referenced to determine the quality score. Combining the probability of the target candidate user triggering the two interaction behaviors of the target content to determine the quality score of the target content relative to the candidate user improves the accuracy of the determined quality score.

[0144] In other embodiments of this application, the interactive behavior further includes a third interactive behavior, wherein the third interactive behavior is later than the second interactive behavior in time; the interaction probability further includes a third interactive probability, which refers to the probability of triggering the third interactive behavior; the interaction probability prediction model further includes a third sub-neural network; such as Figure 7 As shown, compared to Figure 6 In the embodiment shown, step 520 further includes:

[0145] Step 640: The third sub-neural network performs probability prediction based on the user embedding features of the target candidate user, the content embedding features of the target content, and the second interaction probability to obtain the third interaction probability.

[0146] The third interaction occurs later than the second interaction in terms of timing, indicating that the second interaction is a necessary condition for triggering the third interaction. If a user triggers a third interaction on a piece of content, it means that the user must have triggered a second interaction on that content.

[0147] For example, if the target content is an advertising link for a product, the first interaction is a click, the second interaction is a purchase, and the third interaction could be a repurchase.

[0148] For example, if the target content is a blog post, the first interactive behavior can be a click, the second interactive behavior can be a reading time of the first duration, and the third interactive behavior can be a reading time of the second duration, where the second duration is longer than the first duration.

[0149] For example, if the target content is music, the first interactive behavior can be a click, the second interactive behavior can be a playback duration of a third duration, and the third interactive behavior can be a playback duration of a fourth duration, wherein the fourth duration is longer than the third duration.

[0150] As described above, since the second interactive behavior is a necessary condition for triggering the third interactive behavior, the third sub-neural network combines the user embedding features of the target user, the content embedding features of the target content, and the second interaction probability to make probability predictions and obtain the third interaction probability of the target user triggering the third interactive behavior on the target content.

[0151] Understandably, when the interaction probability includes the first interaction probability, the second interaction probability, and the third interaction probability, step 420 corresponds to determining the quality score of the target content relative to the target candidate user by comprehensively considering the first interaction probability, the second interaction probability, and the third interaction probability of the target candidate user interacting with the target content.

[0152] In some embodiments, weighting coefficients can be pre-set for the first interaction probability, the second interaction probability, and the third interaction probability, and then the first interaction probability, the second interaction probability, and the third interaction probability of the target candidate user relative to the target content can be weighted to obtain the quality score of the target content relative to the target candidate user.

[0153] In some embodiments, a multiplicative fusion approach can be used to multiply the first interaction probability, the second interaction probability, and the third interaction probability, and use the result of the multiplication as the quality score of the target content relative to the target candidate user.

[0154] In some embodiments, the first interaction probability, the second interaction probability, and the third interaction probability may be pre-exponentially calculated, and the results of the exponential calculations may be multiplied together, with the result of the multiplication being used as the quality score of the target content relative to the target candidate user.

[0155] Figure 8A This is a framework diagram of an interaction probability prediction model specifically illustrated in this application. This interaction probability prediction model can be used to predict the probability of a user's interaction with an advertisement. Figure 8A As shown, the interaction probability prediction model includes a CTR (Click-Through-Rate) dual-tower model, a shallow CTCVR (Click-Through & Conversion Rate) dual-tower model, and a deep CTCVR dual-tower model.

[0156] The CTR dual-tower model can be regarded as the first sub-neural network used to predict the probability of the first interaction that triggers the first interactive behavior, as mentioned above. In the specific application scenario of advertising, the first interactive behavior can be a click behavior.

[0157] The shallow CTCVR dual-tower model can be viewed as the second sub-neural network used above to predict the probability of triggering the second interactive behavior. Specifically, in advertising applications, the second interactive behavior can be a conversion behavior (such as placing an order or making a purchase). In this scenario, the following relationship exists:

[0158] P(CTCVR) = P(CTR) * P(CVR); (Formula 4)

[0159] Where P(CTCVR) represents the probability of an ad being viewed and converted; P(CTR) represents the probability of an ad being viewed and clicked; and P(CVR) represents the probability of an ad being clicked and converted.

[0160] The deep CTCVR dual-tower model can be regarded as the third sub-neural network used to predict the probability of triggering the third interactive behavior, as mentioned above. In the specific application scenario of advertising, the third interactive behavior can be a repurchase behavior.

[0161] like Figure 8A As shown, the shared embedding layer is used to transform user information and advertising information into the same embedding space, obtaining the user's embedding features and the advertising embedding features. Figure 8A As shown, the interaction probability prediction model also includes an SE (Sequeze and Excitation) module, which is used to compute attention.

[0162] After calculating the attention, deep feature extraction is performed through multiple expert networks. Specifically, these include user expert network 0, user expert network 1, user expert network 2, and shared expert network 1 for deep feature extraction of user embedded features; and advertising expert network 0, advertising expert network 1, advertising expert network 2, and shared expert network 2 for deep feature extraction of advertising embedded features. In some embodiments, the expert networks may be XNNs (Explainable Neural Networks).

[0163] Furthermore, in Figure 8A In the interactive probability prediction model shown, a gate is set up for each prediction task (i.e., the probability of predicting click probability, the probability of triggering conversion behavior, and the probability of triggering repurchase behavior), so that different tasks and different data can use the deep features extracted by the expert network in a more diverse way.

[0164] Subsequently, the CTR dual-tower model predicts the click probability (first interaction probability) based on the user characteristics output by the user expert network and the advertising characteristics output by the advertising expert network; the shallow CTCVR dual-tower model predicts the second interaction probability that triggers conversion behavior based on the user characteristics output by the user expert network and the advertising characteristics output by the advertising expert network; and the deep CTCVR dual-tower model predicts the third interaction probability that triggers repurchase behavior based on the user characteristics output by the user expert network and the advertising characteristics output by the advertising expert network.

[0165] In the Figure 8ADuring the continued training of the interactive probability prediction model shown, the target loss can be a weighted sum of the first loss calculated based on the output of the CTR dual-tower model, the second loss calculated based on the output of the shallow CTCVR dual-tower model, and the third loss calculated based on the output of the deep CTCVR dual-tower model.

[0166] Figure 8A In the interactive probability prediction models shown, the CTR dual-tower model, the shallow CTCVR dual-tower model, and the deep CTCVR dual-tower model are all dual-tower models. Figure 8B This is a schematic diagram of a twin-tower model according to an embodiment of this application, as shown below. Figure 8A As shown, the dual-tower model includes a neural network that performs nonlinear transformations on user features and a neural network that performs nonlinear transformations on advertising features. The outputs of the two neural networks are then fused to obtain fused features, which can then be used for classification to output the corresponding interaction probabilities. Figure 8A The interaction probability prediction model shown can be trained using a multi-task learning approach. This interaction probability prediction model can be called Complete Conversion Chain Multi-task Model (C3MM).

[0167] exist Figure 8B In the dual-tower model shown, user characteristics can be transformed using a fully connected network (e.g., Figure 8B The Fc_1_1, Fc_1_2, ..., Fc_1_m in the model are transformed using a fully connected network (e.g., ...). Figure 8B In Fc_2_1, Fc_2_2, ..., Fc_2_n). It is understandable that... Figure 8B This is merely an illustrative example of the dual-tower model and should not be considered a limitation on the use of this application.

[0168] In some embodiments of this application, such as Figure 9 As shown, the method also includes:

[0169] Step 911: Obtain training data. The training data includes multiple training samples. The training samples include user information of the sample users, content information of the sample content, and interaction tags. The interaction tags include a first interaction tag, a second interaction tag, and a third interaction tag. The first interaction tag is used to indicate whether the sample user triggers a first interaction behavior on the sample content. The second interaction tag is used to indicate whether the sample user triggers a second interaction behavior on the sample content. The third interaction tag is used to indicate whether the sample user triggers a third interaction behavior on the sample content.

[0170] In some embodiments, if a sample user triggers a first interactive action on the sample content, the first interaction label is set to 1; otherwise, if a sample user does not trigger a first interactive action on the sample content, the first interaction label is set to 0. Similarly, the second interaction label can also be set to 0 (indicating that the user did not trigger a second interactive action) and 1 (indicating that the user did not trigger a second interactive action); the third interaction label can also be set to 0 (indicating that the user did not trigger a third interactive action) and 1 (indicating that the user did not trigger a third interactive action).

[0171] Step 912: The shared embedding layer generates user embedding features of the sample user and content embedding features of the sample content based on the user information of the sample user and the content information of the sample content.

[0172] Step 913: The first sub-neural network performs probability prediction based on the user embedding features of the sample users and the content embedding features of the sample content to obtain the first predicted interaction probability corresponding to the training sample.

[0173] Step 914: The second sub-neural network performs probability prediction based on the user embedding features of the sample user, the content embedding features of the sample content, and the first predicted interaction probability to obtain the second predicted interaction probability corresponding to the training sample.

[0174] Step 915: The third sub-neural network performs probability prediction based on the user embedding features of the sample users, the content embedding features of the sample content, and the second predicted interaction probability to obtain the third predicted interaction probability corresponding to the training sample.

[0175] Through steps 912-915 above, the first predicted interaction probability, the second predicted interaction probability, and the third predicted interaction probability corresponding to each training sample can be calculated respectively.

[0176] Step 916: Calculate the first loss based on the first predicted interaction probability and the first interaction label.

[0177] In some embodiments, the first predicted interaction probability and the first interaction label are substituted into a set loss function to calculate the first loss. The set loss function can be an absolute value loss function, a squared loss function, a cross-entropy loss function, etc., and is not specifically limited here.

[0178] In some embodiments, the loss function may be the cross-entropy loss function, and the first loss can be calculated according to the following formula.

[0179] ;(Formula 5)

[0180] Where N is the total number of training samples, Represents the i-th training sample The corresponding first interactive tag; Let i be the i-th training sample. The predicted probability of the first predicted interaction.

[0181] Step 917: Calculate the second loss based on the second predicted interaction probability and the second interaction label.

[0182] Step 918: Calculate the third loss based on the third predicted interaction probability and the third interaction label. The calculation of the second and third losses is similar to the calculation of the first loss and will not be repeated here.

[0183] Step 919: Determine the target loss based on the first loss, the second loss, and the third loss.

[0184] In some embodiments, the first loss, the second loss, and the third loss can be weighted and summed according to the following formula 5 to obtain the target loss.

[0185] ;(Formula 6)

[0186] Where Loss is the target loss. , , These are weighting coefficients, which can be preset according to actual needs. The first loss, This is the second loss. This is the third loss.

[0187] Step 920: Adjust the parameters of the interaction probability prediction model in reverse according to the target loss until the training termination condition is met.

[0188] In some embodiments, the model parameters of the interaction probability prediction model can be adjusted backward using the gradient descent algorithm based on the first prediction loss, and backward training can be stopped when the gradient is less than a threshold. The process of adjusting the parameters of the interaction probability prediction model backward based on the target loss is the backward training process of the interaction probability prediction model.

[0189] The training termination condition can be that the number of iterations of the interactive probability prediction model reaches a set threshold, or that the loss function of the interactive probability prediction model converges.

[0190] pass Figure 9 The training process shown enables the interaction probability prediction model to learn the ability to accurately predict interaction probabilities (first interaction probability, second interaction probability, and third interaction probability) based on user information and content information, thereby ensuring the accuracy of the interaction probabilities predicted by the interaction probability prediction model during online application.

[0191] In some embodiments, if the interaction probability prediction model does not include a third sub-neural network model, the training process of the interaction probability prediction model is the same as... Figure 9 The process shown is similar and will not be repeated here.

[0192] The following describes an apparatus embodiment of this application, which can be used to perform the methods described in the above embodiments of this application. For details not disclosed in the apparatus embodiments of this application, please refer to the method embodiments described in the above embodiments of this application.

[0193] Figure 10 This is a block diagram of a content delivery control device according to one embodiment, such as... Figure 10 As shown, the content delivery control device includes: an acquisition module 1010, used to acquire delivery feedback data and delivery consumption data of the target content; the delivery feedback data includes delivery feedback records under each content tag of the target content; the delivery consumption data includes delivery consumption records under each content tag of the target content; a tag quality level determination module 1020, used to determine the tag quality level corresponding to each content tag of the target content based on the delivery feedback records and delivery consumption records under each content tag of the target content; a level adjustment module 1030, used to adjust the level of each content tag according to the tag quality level corresponding to each content tag, to obtain the adjusted level of each content tag; a filtering threshold determination module 1040, used to determine the filtering threshold based on the adjusted levels of all content tags of the target content; and a delivery control module 1050, used to control the delivery of the target content based on the filtering threshold and the quality score of the target content, wherein if the content quality score of the target content is not lower than the filtering threshold, the target content is delivered; if the content quality score of the target content is lower than the filtering threshold, the target content is filtered out.

[0194] In some embodiments of this application, the tag quality level determination module 1020 includes: a feedback parameter calculation unit, used to calculate feedback parameters under each content tag of the target content based on the delivery feedback records under each content tag of the target content; a consumption parameter calculation unit, used to calculate consumption parameters under each content tag of the target content based on the delivery consumption records under each content tag of the target content; and a tag quality level determination unit, used to determine the tag quality level corresponding to each content tag of the target content based on the correspondence between tag quality level and feedback parameters and consumption parameters, as well as the feedback parameters and consumption parameters under each content tag of the target content.

[0195] In some embodiments of this application, the tag quality level includes high-quality tags and low-quality tags; the level adjustment module 1030 is further configured to: if the tag quality level corresponding to the content tag is a high-quality tag, then reduce the level of the content tag to obtain the adjusted level of the content tag; if the tag quality level corresponding to the content tag is low-quality, then increase the level of the content tag to obtain the adjusted level of the content tag.

[0196] In some embodiments of this application, the filtering threshold determination module 1040 includes: a quality score determination unit, used to determine the quality score corresponding to the adjusted level of each content tag according to the correspondence between the level and the quality score; and a filtering threshold determination unit, used to determine the minimum quality score among the quality scores corresponding to the adjusted level of each content tag, and determine the minimum quality score as the filtering threshold.

[0197] In some embodiments of this application, the quality score of the target content includes the quality score of the target content relative to each candidate user; the delivery control module 1050 includes: an interaction probability acquisition unit, used to acquire the interaction probability of the target candidate user interacting with the target content; a quality score determination unit, used to determine the quality score of the target content relative to the target candidate user based on the interaction probability; an adding unit, used to send the target content to the target candidate user if a content request initiated by the target candidate user is received, if the quality score of the target content relative to the target candidate user is not lower than the filtering threshold; and a filtering unit, used to filter out the target content if the quality score of the target content relative to the target candidate user is lower than the filtering threshold.

[0198] In some embodiments of this application, the content delivery control device further includes: a tag matching degree calculation unit, used to calculate the tag matching degree between the content tags of the target content and the user tags of each user in the user set; and a candidate user determination unit, used to determine users whose tag matching degree exceeds the matching degree threshold as candidate users of the target content.

[0199] In some embodiments of this application, the interaction probability acquisition unit includes: an acquisition unit, used to acquire user information of the target candidate user and content information of the target content; and an interaction probability determination unit, used by an interaction probability prediction model to output the interaction probability of the target candidate user triggering an interactive behavior on the target content based on the user information of the target candidate user and the content information of the target content.

[0200] In some embodiments of this application, the interactive behavior includes a first interactive behavior and a second interactive behavior arranged chronologically from first to last; the interaction probability includes a first interaction probability and a second interaction probability, wherein the first interaction probability refers to the probability of triggering the first interactive behavior, and the second interaction probability refers to the probability of triggering the second interactive behavior; the interaction probability prediction model includes a shared embedding layer, a first sub-neural network, and a second sub-neural network; the interaction probability determination unit includes: an embedding feature determination unit, used by the shared embedding layer to generate user embedding features of the target candidate user and content embedding features of the target content according to user information of the target candidate user and content information of the target content; a first prediction unit, used by the first sub-neural network to perform probability prediction based on the user embedding features of the target candidate user and the content embedding features of the target content to obtain the first interaction probability; and a second prediction unit, used by the second sub-neural network to perform probability prediction based on the user embedding features of the target candidate user, the content embedding features of the target content, and the first interaction probability to obtain the second interaction probability.

[0201] In other embodiments of this application, the interactive behavior further includes a third interactive behavior, wherein the third interactive behavior is later than the second interactive behavior in terms of timing; the interactive probability further includes a third interactive probability, which refers to the probability of triggering the third interactive behavior; the interactive probability prediction model further includes a third sub-neural network; the interactive probability determination unit further includes: a third prediction unit, used to obtain the third interactive probability by the third sub-neural network performing probability prediction based on the user embedding features of the target candidate user, the content embedding features of the target content, and the second interactive probability.

[0202] In some embodiments of this application, the content delivery control device further includes: a training data acquisition module, used to acquire training data, the training data including multiple training samples, the training samples including user information of sample users, content information of sample content, and interaction tags, the interaction tags including a first interaction tag, a second interaction tag, and a third interaction tag, the first interaction tag being used to indicate whether the sample user triggers a first interaction behavior on the sample content, the second interaction tag being used to indicate whether the sample user triggers a second interaction behavior on the sample content, and the third interaction tag being used to indicate whether the sample user triggers a third interaction behavior on the sample content; an embedding feature generation module, used by a shared embedding layer to generate user embedding features of the sample user and content embedding features of the sample content according to the user information of the sample user and the content information of the sample content; a first predicted interaction probability determination module, used by a first sub-neural network to perform probability prediction based on the user embedding features of the sample user and the content embedding features of the sample content to obtain a first predicted interaction probability corresponding to the training sample; and a second... The system comprises the following modules: a prediction interaction probability determination module, which uses a second sub-neural network to predict the second predicted interaction probability of a training sample based on the user embedding features of the sample user, the content embedding features of the sample content, and the first predicted interaction probability; a third prediction interaction probability determination module, which uses a third sub-neural network to predict the third predicted interaction probability of a training sample based on the user embedding features of the sample user, the content embedding features of the sample content, and the second predicted interaction probability; a first loss calculation module, which calculates a first loss based on the first predicted interaction probability and a first interaction label; a second loss calculation module, which calculates a second loss based on the second predicted interaction probability and the second interaction label; a third loss calculation module, which calculates a third loss based on the third predicted interaction probability and the third interaction label; a target loss determination module, which determines a target loss based on the first, second, and third losses; and a reverse adjustment module, which adjusts the parameters of the interaction probability prediction model in reverse based on the target loss until the training termination condition is met.

[0203] Figure 11 A schematic diagram of a computer system suitable for implementing the embodiments of this application is shown. It should be noted that... Figure 11 The computer system 1100 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0204] like Figure 11As shown, the computer system 1100 includes a Central Processing Unit (CPU) 1101, which can perform various appropriate actions and processes, such as executing the methods described in the above embodiments, based on programs stored in Read-Only Memory (ROM) 1102 or programs loaded from storage portion 1108 into Random Access Memory (RAM) 1103. The RAM 1103 also stores various programs and data required for system operation. The CPU 1101, ROM 1102, and RAM 1103 are interconnected via a bus 1104. An Input / Output (I / O) interface 1105 is also connected to the bus 1104.

[0205] The following components are connected to I / O interface 1105: an input section 1106 including a keyboard, mouse, etc.; an output section 1107 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 1108 including a hard disk, etc.; and a communication section 1109 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 1109 performs communication processing via a network such as the Internet. A drive 1110 is also connected to I / O interface 1105 as needed. Removable media 1111, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 1110 as needed so that computer programs read from them can be installed into storage section 1108 as needed.

[0206] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 1109, and / or installed from removable medium 1111. When the computer program is executed by central processing unit (CPU) 1101, it performs various functions defined in the system of this application.

[0207] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such transmitted data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0208] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0209] The units described in the embodiments of this application can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.

[0210] In another aspect, this application also provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiments; or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable storage medium carries computer-readable instructions that, when executed by a processor, implement the methods in any of the above embodiments.

[0211] According to one aspect of this application, an electronic device is also provided, comprising: a processor; and a memory storing computer-readable instructions that, when executed by the processor, implement the methods of any of the above embodiments.

[0212] According to one aspect of the embodiments of this application, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, causing the computer device to perform the methods of any of the above embodiments.

[0213] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to the embodiments of this application, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0214] Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, touch terminal, or network device, etc.) to execute the methods according to the embodiments of this application.

[0215] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the embodiments disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein.

[0216] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A method for controlling the delivery of content, characterized in that, include: Obtain delivery feedback data and delivery consumption data for the target content; the delivery feedback data includes delivery feedback records under each content tag of the target content; the delivery consumption data includes delivery consumption records under each content tag of the target content. Based on the delivery feedback records and delivery consumption records under each content tag of the target content, determine the tag quality level corresponding to each content tag of the target content; Based on the tag quality level corresponding to each content tag, the level of each content tag is adjusted to obtain the adjusted level of each content tag, including: if the tag quality level corresponding to the content tag is a high-quality tag, then the level of the content tag is reduced to obtain the adjusted level of the content tag; if the tag quality level corresponding to the content tag is a low-quality tag, then the level of the content tag is increased to obtain the adjusted level of the content tag. A filtering threshold is determined based on the adjusted levels of all content tags of the target content. The filtering threshold is equal to the minimum value of the quality scores corresponding to the adjusted levels of all content tags of the target content, and the levels are positively correlated with the quality scores. The delivery of the target content is controlled based on the filtering threshold and the quality score of the target content. If the quality score of the target content is not lower than the filtering threshold, the target content is delivered; if the quality score of the target content is lower than the filtering threshold, the target content is filtered out. The quality score of the target content is positively correlated with the probability that a user will trigger an interactive behavior with the target content.

2. The method according to claim 1, characterized in that, The step of determining the tag quality level corresponding to each content tag of the target content based on the delivery feedback records and delivery consumption records under each content tag of the target content includes: Based on the delivery feedback records under each content tag of the target content, calculate the feedback parameters under each content tag of the target content; Based on the delivery and consumption records under each content tag of the target content, calculate the consumption parameters under each content tag of the target content; Based on the correspondence between the tag quality level and the feedback parameters and consumption parameters, as well as the feedback parameters and consumption parameters under each content tag of the target content, the tag quality level corresponding to each content tag of the target content is determined.

3. The method according to claim 1, characterized in that, The step of determining the filtering threshold based on the adjusted levels of all content tags of the target content includes: Based on the correspondence between levels and quality scores, determine the quality score corresponding to the adjusted level of each content tag; Among the quality scores corresponding to the adjusted levels of each of the aforementioned content tags, the minimum quality score is determined, and the minimum quality score is set as the filtering threshold.

4. The method according to claim 1, characterized in that, The quality score of the target content includes the quality score of the target content relative to each candidate user; The step of controlling the delivery of the target content based on the filtering threshold and the quality score of the target content includes: Obtain the interaction probability between the target candidate user and the target content; The quality score of the target content relative to the target candidate user is determined based on the interaction probability. If the quality score of the target content relative to the target candidate user is not lower than the filtering threshold, then if a content request is received from the target candidate user, the target content is sent to the target candidate user. If the quality score of the target content relative to the target candidate user is lower than the filtering threshold, then the target content will be filtered out.

5. The method according to claim 4, characterized in that, The method further includes: Calculate the tag matching degree between the content tags of the target content and the user tags of each user in the user set; Users whose tag matching degree exceeds the matching degree threshold are identified as candidate users for the target content.

6. The method according to claim 4, characterized in that, The acquisition of the interaction probability between the target candidate user and the target content includes: Obtain the user information of the target candidate user and the content information of the target content; The interaction probability prediction model outputs the interaction probability of the target candidate user triggering an interactive behavior on the target content based on the user information of the target candidate user and the content information of the target content.

7. The method according to claim 6, characterized in that, The interactive behaviors include a first interactive behavior and a second interactive behavior arranged chronologically from first to last; the interaction probabilities include a first interaction probability and a second interaction probability, wherein the first interaction probability refers to the probability of triggering the first interactive behavior, and the second interaction probability refers to the probability of triggering the second interactive behavior. The interactive probability prediction model includes a shared embedding layer, a first sub-neural network, and a second sub-neural network; The step of the interaction probability prediction model outputting the interaction probability of the target candidate user triggering interactive behavior on the target content based on the user information of the target candidate user and the content information of the target content includes: The shared embedding layer generates user embedding features of the target candidate user and content embedding features of the target content based on the user information of the target candidate user and the content information of the target content. The first sub-neural network performs probability prediction based on the user embedding features of the target candidate user and the content embedding features of the target content to obtain the first interaction probability; The second sub-neural network performs probability prediction based on the user embedding features of the target candidate user, the content embedding features of the target content, and the first interaction probability to obtain the second interaction probability.

8. The method according to claim 7, characterized in that, The interactive behavior also includes a third interactive behavior, wherein the third interactive behavior is later than the second interactive behavior in terms of timing; the interaction probability also includes a third interaction probability, which refers to the probability of triggering the third interactive behavior; the interaction probability prediction model also includes a third sub-neural network; The step of the interaction probability prediction model outputting the interaction probability of the target candidate user triggering an interactive behavior on the target content based on the user information of the target candidate user and the content information of the target content further includes: The third interaction probability is obtained by the third sub-neural network performing probability prediction based on the user embedding features of the target candidate user, the content embedding features of the target content, and the second interaction probability.

9. The method according to claim 8, characterized in that, The method further includes: Acquire training data, which includes multiple training samples. The training samples include user information of sample users, content information of sample content, and interaction tags. The interaction tags include a first interaction tag, a second interaction tag, and a third interaction tag. The first interaction tag is used to indicate whether the sample user triggers a first interaction behavior on the sample content. The second interaction tag is used to indicate whether the sample user triggers a second interaction behavior on the sample content. The third interaction tag is used to indicate whether the sample user triggers a third interaction behavior on the sample content. The shared embedding layer generates user embedding features for the sample user and content embedding features for the sample content based on the user information of the sample user and the content information of the sample content. The first sub-neural network performs probability prediction based on the user embedding features of the sample user and the content embedding features of the sample content to obtain the first predicted interaction probability corresponding to the training sample; The second sub-neural network performs probability prediction based on the user embedding features of the sample user, the content embedding features of the sample content, and the first predicted interaction probability to obtain the second predicted interaction probability corresponding to the training sample. The third sub-neural network performs probability prediction based on the user embedding features of the sample user, the content embedding features of the sample content, and the second predicted interaction probability to obtain the third predicted interaction probability corresponding to the training sample. Calculate the first loss based on the first predicted interaction probability and the first interaction label; Calculate the second loss based on the second predicted interaction probability and the second interaction label; Calculate the third loss based on the third predicted interaction probability and the third interaction label; The target loss is determined based on the first loss, the second loss, and the third loss; The parameters of the interaction probability prediction model are adjusted in reverse based on the target loss until the training termination condition is met.

10. A content delivery control device, characterized in that, include: The acquisition module is used to acquire delivery feedback data and delivery consumption data of the target content; the delivery feedback data includes delivery feedback records under each content tag of the target content; the delivery consumption data includes delivery consumption records under each content tag of the target content. The tag quality level determination module is used to determine the tag quality level corresponding to each content tag of the target content based on the delivery feedback record and the delivery consumption record under each content tag of the target content. The level adjustment module is used to lower the level of the content tag if the tag quality level corresponding to the content tag is a high-quality tag, and to obtain the adjusted level of the content tag if the tag quality level corresponding to the content tag is a low-quality tag. The filtering threshold determination module is used to determine a filtering threshold based on the adjusted levels of all content tags of the target content; the filtering threshold is equal to the minimum value of the quality scores corresponding to the adjusted levels of all content tags of the target content, and the level is positively correlated with the quality score; The delivery control module is used to control the delivery of the target content based on the filtering threshold and the quality score of the target content. If the quality score of the target content is not lower than the filtering threshold, the target content is delivered; if the quality score of the target content is lower than the filtering threshold, the target content is filtered out. The quality score of the target content is positively correlated with the probability that a user will trigger an interactive behavior on the target content.

11. The apparatus according to claim 10, characterized in that, The label quality level determination module includes: The feedback parameter calculation unit is used to calculate the feedback parameters under each content tag of the target content based on the delivery feedback records under each content tag of the target content. The consumption parameter calculation unit is used to calculate the consumption parameters under each content tag of the target content based on the delivery consumption records under each content tag of the target content. The tag quality level determination unit is used to determine the tag quality level corresponding to each content tag of the target content based on the correspondence between the tag quality level and the feedback parameters and the consumption parameters under each content tag of the target content.

12. The apparatus according to claim 10, characterized in that, The filtering threshold determination module includes: The quality score determination unit is used to determine the quality score corresponding to the adjusted level of each content tag based on the correspondence between the level and the quality score. The filtering threshold determination unit is used to determine the minimum quality score among the quality scores corresponding to the adjusted levels of each content tag, and to determine the minimum quality score as the filtering threshold.

13. The apparatus according to claim 10, characterized in that, The quality score of the target content includes the quality score of the target content relative to each candidate user; The delivery control module includes: An interaction probability acquisition unit is used to acquire the interaction probability of a target candidate user interacting with the target content. A quality score determination unit is used to determine the quality score of the target content relative to the target candidate user based on the interaction probability. An adding unit is configured to send the target content to the target candidate user if the quality score of the target content relative to the target candidate user is not lower than the filtering threshold, and if a content request initiated by the target candidate user is received. A filtering unit is configured to filter out the target content if the quality score of the target content relative to the target candidate user is lower than the filtering threshold.

14. The apparatus according to claim 13, characterized in that, The content delivery control device further includes: The tag matching degree calculation unit is used to calculate the tag matching degree between the content tags of the target content and the user tags of each user in the user set; The candidate user determination unit is used to determine users whose tag matching degree exceeds the matching degree threshold as candidate users of the target content.

15. The apparatus according to claim 13, characterized in that, The interaction probability acquisition unit includes: The acquisition unit is used to acquire user information of the target candidate user and content information of the target content; The interaction probability determination unit is used to output the interaction probability of the target candidate user triggering an interactive behavior on the target content based on the user information of the target candidate user and the content information of the target content by the interaction probability prediction model.

16. The apparatus according to claim 15, characterized in that, The interactive behaviors include a first interactive behavior and a second interactive behavior arranged chronologically from first to last; the interaction probabilities include a first interaction probability and a second interaction probability, wherein the first interaction probability refers to the probability of triggering the first interactive behavior, and the second interaction probability refers to the probability of triggering the second interactive behavior; the interaction probability prediction model includes a shared embedding layer, a first sub-neural network, and a second sub-neural network; the interaction probability determination unit includes: An embedding feature determination unit is used by the shared embedding layer to generate user embedding features of the target candidate user and content embedding features of the target content based on the user information of the target candidate user and the content information of the target content. The first prediction unit is used to perform probability prediction by the first sub-neural network based on the user embedding features of the target candidate user and the content embedding features of the target content to obtain the first interaction probability. The second prediction unit is used to perform probability prediction by the second sub-neural network based on the user embedding features of the target candidate user, the content embedding features of the target content, and the first interaction probability to obtain the second interaction probability.

17. The apparatus according to claim 16, characterized in that, The interactive behavior also includes a third interactive behavior, wherein the third interactive behavior is later than the second interactive behavior in terms of timing; the interaction probability also includes a third interaction probability, which refers to the probability of triggering the third interactive behavior; the interaction probability prediction model also includes a third sub-neural network; The interaction probability determination unit further includes: The third prediction unit is used to perform probability prediction by the third sub-neural network based on the user embedding features of the target candidate user, the content embedding features of the target content, and the second interaction probability to obtain the third interaction probability.

18. The apparatus according to claim 17, characterized in that, The content delivery control device further includes: The training data acquisition module is used to acquire training data, which includes multiple training samples. The training samples include user information of sample users, content information of sample content, and interaction tags. The interaction tags include a first interaction tag, a second interaction tag, and a third interaction tag. The first interaction tag is used to indicate whether the sample user triggers a first interaction behavior on the sample content. The second interaction tag is used to indicate whether the sample user triggers a second interaction behavior on the sample content. The third interaction tag is used to indicate whether the sample user triggers a third interaction behavior on the sample content. The embedding feature generation module is used by the shared embedding layer to generate user embedding features of the sample user and content embedding features of the sample content based on the user information of the sample user and the content information of the sample content. The first predicted interaction probability determination module is used to perform probability prediction by the first sub-neural network based on the user embedding features of the sample user and the content embedding features of the sample content to obtain the first predicted interaction probability corresponding to the training sample. The second predicted interaction probability determination module is used to perform probability prediction by the second sub-neural network based on the user embedding features of the sample user, the content embedding features of the sample content and the first predicted interaction probability, to obtain the second predicted interaction probability corresponding to the training sample. The third predicted interaction probability determination module is used to perform probability prediction by the third sub-neural network based on the user embedding features of the sample user, the content embedding features of the sample content and the second predicted interaction probability, to obtain the third predicted interaction probability corresponding to the training sample. The first loss calculation module is used to calculate the first loss based on the first predicted interaction probability and the first interaction label; The second loss calculation module is used to calculate the second loss based on the second predicted interaction probability and the second interaction label; The third loss calculation module is used to calculate the third loss based on the third predicted interaction probability and the third interaction label; The target loss determination module is used to determine the target loss based on the first loss, the second loss, and the third loss; The reverse adjustment module is used to adjust the parameters of the interaction probability prediction model in reverse according to the target loss until the training termination condition is met.

19. An electronic device, characterized in that, include: processor; A memory storing computer-readable instructions that, when executed by the processor, implement the method as described in any one of claims 1-9.

20. A computer-readable storage medium storing computer-readable instructions thereon, characterized in that, When the computer-readable instructions are executed by a processor, the method as described in any one of claims 1-9 is implemented.

21. A computer program product comprising computer instructions, characterized in that, When the computer instructions are executed by the processor, they implement the method of any one of claims 1-9.

Images