Recommendation model training method and device, content recommendation method and device, equipment and medium

Abstract

The invention discloses a recommendation model training method and device, a content recommendation method and device, equipment and a medium, belongs to the technical field of computers, and can be applied to the technical field of artificial intelligence. The training method of the recommendation model comprises the steps of obtaining sample input information of a sample interaction object; calling an initial recommendation model to process the sample input information to obtain predicted recommendation content; obtaining a diversity measurement index of the predicted recommendation content, wherein the diversity measurement index is used for measuring the recommendation diversity of the predicted recommendation content; obtaining training loss based on diversity measurement indexes; and updating parameters of the initial recommendation model by using the training loss to obtain a target recommendation model. The training effect of the recommendation model is improved, and then the viewing rate of the content recommended by using the trained recommendation model is improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method for training a recommendation model, a method for recommending content, an apparatus, a device, and a medium. Background Technology

[0002] With the development of computer technology, more and more applications are using artificial intelligence (AI) to recommend personalized content (videos, news, articles, etc.) to users. When using AI to recommend content, a recommendation model needs to be trained first, and then the trained model is used to recommend content to the user. Summary of the Invention

[0003] This application provides a method for training a recommendation model, a method for recommending content, an apparatus, a device, and a medium, which can be used to improve the training effect of the recommendation model and thereby increase the viewing rate of the content recommended by the recommendation model.

[0004] On the one hand, embodiments of this application provide a method for training a recommendation model, the method comprising:

[0005] Obtain the sample input information of the sample interaction object;

[0006] The initial recommendation model is invoked to process the sample input information to obtain the predicted recommendation content;

[0007] Obtain a diversity metric for the predicted recommended content, which is used to measure the diversity of the predicted recommended content.

[0008] Based on the aforementioned diversity metric, the training loss is obtained;

[0009] The parameters of the initial recommendation model are updated using the training loss to obtain the target recommendation model, which is used to recommend target content based on target input information.

[0010] In one possible implementation, the sample candidate content is the content obtained after performing at least one round of filtering operations on the original content to be recommended.

[0011] A content recommendation method is also provided, the method comprising:

[0012] Obtain the target input information of the target recommendation model and the target interaction object, wherein the target recommendation model is trained using any of the above-described recommendation model training methods;

[0013] The target recommendation model is invoked to process the target input information to obtain the target recommendation content;

[0014] The target recommended content is recommended to the target interactive object.

[0015] On the other hand, embodiments of this application provide a training apparatus for a recommendation model, the apparatus comprising:

[0016] The first acquisition module is used to acquire sample input information of the sample interaction object;

[0017] The processing module is used to call the initial recommendation model to process the sample input information and obtain the predicted recommendation content;

[0018] The second acquisition module is used to acquire the diversity measurement index of the predicted recommendation content, and the diversity measurement index is used to measure the recommendation diversity of the predicted recommendation content.

[0019] The third acquisition module is used to acquire the training loss based on the diversity measurement index;

[0020] An update module is used to update the parameters of the initial recommendation model using the training loss to obtain a target recommendation model, which is used to recommend target content based on target input information.

[0021] In one possible implementation, the second acquisition module is used to acquire a training loss based on the diversity metric and the accuracy metric, wherein the accuracy metric is used to measure the recommendation accuracy of the predicted recommended content.

[0022] In one possible implementation, the processing module is used to call the initial recommendation model to process the sample input information to obtain the predicted recommendation content and the prediction evaluation index of the predicted recommendation content. The prediction evaluation index is the predicted value of the comprehensive effect of recommending the predicted recommendation content based on the sample input information.

[0023] The third acquisition module is used to determine a standard evaluation index for the predicted recommended content based on the diversity measurement index and the accuracy measurement index, wherein the standard evaluation index is a standard value of the comprehensive effect; and to acquire the training loss based on the prediction evaluation index and the standard evaluation index.

[0024] In one possible implementation, the third acquisition module is configured to: determine a first evaluation index based on the diversity measurement index and the accuracy measurement index, wherein the first evaluation index is a standard value of the direct effect of recommending the predicted recommendation content based on the sample input information; acquire updated input information of the sample interaction object; acquire a second evaluation index based on the updated input information, wherein the second evaluation index is a standard value of the indirect effect of recommending the predicted recommendation content based on the sample input information; and determine the standard evaluation index based on the first evaluation index and the second evaluation index.

[0025] In one possible implementation, the initial recommendation model includes a first initial sub-model and a second initial sub-model; the processing module is used to call the first initial sub-model to process the sample input information to obtain the predicted recommendation content; and to call the second initial sub-model to process the predicted recommendation content and the sample input information to obtain the prediction evaluation index of the predicted recommendation content.

[0026] In one possible implementation, the training loss includes a first loss and a second loss; the third acquisition module is used to acquire the first loss based on the prediction evaluation index, the first loss being used to update the parameters of the first initial sub-model in the direction of maximizing the prediction evaluation index; and to acquire the second loss based on the prediction evaluation index and the standard evaluation index, the second loss being used to update the parameters of the second initial sub-model in the direction of minimizing the difference between the prediction evaluation index and the standard evaluation index.

[0027] The update module is used to update the parameters of the first initial sub-model using the first loss and to update the parameters of the second initial sub-model using the second loss.

[0028] In one possible implementation, the second acquisition module is configured to acquire a similarity matrix based on the predicted recommended content and the historical recommended content of the sample interaction object. The element in the i-th row and j-th column of the similarity matrix is ​​used to indicate the similarity between the i-th content and the j-th content in the content set consisting of the predicted recommended content and the historical recommended content, where i and j are integers not greater than n, and n is the number of content in the content set. The module also acquires the determinant calculation result of the similarity matrix, which is used to measure the degree of difference between the content in the content set. Based on the determinant calculation result, the module acquires a diversity measurement index for the predicted recommended content.

[0029] In one possible implementation, the second acquisition module is used to acquire interaction information of the predicted recommendation content, the interaction information being used to indicate the interaction between the sample interaction object and the predicted recommendation content; and based on the interaction information, to acquire the accuracy measurement index.

[0030] In one possible implementation, the second acquisition module is used to use the measurement index corresponding to the interaction situation indicated by the interaction information as the accuracy measurement index; wherein, the interaction situation indicated by the interaction information includes a first interaction situation, a second interaction situation, a third interaction situation, or a fourth interaction situation, the measurement index corresponding to the first interaction situation is less than the measurement index corresponding to the second interaction situation, the measurement index corresponding to the second interaction situation is less than the measurement index corresponding to the third interaction situation, and the measurement index corresponding to the third interaction situation is less than the measurement index corresponding to the fourth interaction situation; the first interaction situation is that the sample interaction object does not trigger the prompt information after the prompt information of the predicted recommendation content is displayed to the sample interaction object; the second interaction situation is that... After displaying the predicted recommended content prompt to the sample interaction object, the sample interaction object triggers the prompt to enter the content display page, and the sample interaction object does not view any content on the content display page; the third interaction situation is that after displaying the predicted recommended content prompt to the sample interaction object, the sample interaction object triggers the prompt to enter the content display page, and the sample interaction object views other content besides the predicted recommended content on the content display page; the fourth interaction situation is that after displaying the predicted recommended content prompt to the sample interaction object, the sample interaction object triggers the prompt to enter the content display page, and the sample interaction object views the predicted recommended content on the content display page.

[0031] In one possible implementation, the first acquisition module is configured to acquire first characterization information and second characterization information, wherein the first characterization information is used to characterize the sample interaction object, and the second characterization information is used to characterize the sample candidate content of the sample interaction object; and acquire the sample input information based on the first characterization information and the second characterization information.

[0032] In one possible implementation, the first acquisition module is used to acquire attribute information of the sample interaction object and statistical information of the sample interaction object under at least one time window; wherein, the statistical information of the sample interaction object under any time window is determined based on the interaction between the sample interaction object and the historical recommended content under any time window; and the first characterization information is acquired based on the attribute information of the sample interaction object and the statistical information of the sample interaction object under at least one time window.

[0033] In one possible implementation, the number of sample candidate contents is at least two. The first acquisition module is used to acquire, for any sample candidate content among the at least two sample candidate contents, a representational sub-information of the any sample candidate content based on at least one of the category of the any sample candidate content or the ranking position of the any sample candidate content among the at least two sample candidate contents, and the feature vector of the any sample candidate content; and acquire the second representational information based on the representational sub-information of the at least two sample candidate contents.

[0034] In one possible implementation, the first acquisition module is configured to acquire at least one of third characterization information or time characterization information; wherein the third characterization information is used to characterize the historical recommended content of the sample interaction object; the time characterization information is determined based on the generation time of the sample input information and the latest time when the sample interaction object generates a reference interaction behavior in response to the historical recommended content; and the sample input information is acquired based on at least one of the third characterization information or the time characterization information, the first characterization information, and the second characterization information.

[0035] In one possible implementation, the sample candidate content is the content obtained after performing at least one round of filtering operations on the original content to be recommended.

[0036] A content recommendation device is also provided, the device comprising:

[0037] The acquisition module is used to acquire the target input information of the target recommendation model and the target interaction object, wherein the target recommendation model is trained using any of the above-described recommendation model training methods;

[0038] The processing module is used to call the target recommendation model to process the target input information and obtain target recommendation content;

[0039] The recommendation module is used to recommend the target content to the target interactive object.

[0040] In one possible implementation, the target recommended content is content from the target candidate content of the target interactive object, and the target candidate content is content published by information dissemination accounts followed by the target interactive object in the social application; the recommendation module is used to display prompt information of the target recommended content at the entry point of the information dissemination account, and the entry point of the information dissemination account is the entry point in the display page of the social application used to view the content published by the information dissemination account.

[0041] In one possible implementation, the recommendation module is further configured to display a prompt marker at the entry point of the information dissemination account, the prompt marker being used to indicate the existence of content to be viewed.

[0042] On the other hand, a computer device is provided, the computer device including a processor and a memory, the memory storing at least one computer program, the at least one computer program being loaded and executed by the processor, so that the computer device implements the training method or content recommendation method of any of the recommendation models described above.

[0043] On the other hand, a computer-readable storage medium is also provided, wherein at least one computer program is stored in the computer-readable storage medium, the at least one computer program being loaded and executed by a processor to enable a computer to implement the training method or content recommendation method of any of the recommendation models described above.

[0044] On the other hand, a computer program or computer program product is also provided, the computer program or computer program product including computer instructions, which are loaded and executed by a processor to enable a computer to implement the training method or content recommendation method of any of the recommendation models described above.

[0045] The technical solution provided in this application has at least the following beneficial effects:

[0046] The technical solution provided in this application update the parameters of the initial recommendation model using training loss obtained based on diversity metrics. This enables the recommendation model training process to focus on the diversity of predicted recommended content, improves the training effect of the recommendation model, increases the diversity of content recommended by the recommendation model, and thus increases the viewing rate of the recommended content. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 This is a schematic diagram of the structure of a computer system provided in an embodiment of this application;

[0049] Figure 2 This is a flowchart of a training method for a recommendation model provided in an embodiment of this application;

[0050] Figure 3 This is a schematic diagram of a recommendation link provided in an embodiment of this application;

[0051] Figure 4 This is a schematic diagram illustrating the training process of a recommendation model provided in an embodiment of this application;

[0052] Figure 5 This is a flowchart of a content recommendation method provided in an embodiment of this application;

[0053] Figure 6 This is a schematic diagram of the interface of a social application for a target interactive object provided in an embodiment of this application;

[0054] Figure 7 This is a flowchart of a content recommendation method provided in an embodiment of this application;

[0055] Figure 8 This is a schematic diagram of a training device for a recommendation model provided in an embodiment of this application;

[0056] Figure 9 This is a schematic diagram of a content recommendation device provided in an embodiment of this application;

[0057] Figure 10 This is a schematic diagram of the structure of a server provided in an embodiment of this application;

[0058] Figure 11 This is a schematic diagram of the structure of a terminal provided in an embodiment of this application. Detailed Implementation

[0059] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0060] Artificial intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0061] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, pre-trained model technology, operating / interactive systems, and mechatronics. Among these, pre-trained models, also known as large-scale models or foundational models, can be widely applied to downstream tasks across various AI fields after fine-tuning. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0062] The solutions provided in this application's embodiments relate to machine learning (ML) technology in artificial intelligence.

[0063] Machine learning is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, pre-trained learning, and unsupervised learning. Pre-trained models represent the latest development in deep learning, integrating all of these techniques.

[0064] With the research and advancement of artificial intelligence (AI) technology, AI is being studied and applied in various fields, such as smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, autonomous driving, drones, digital twins, virtual humans, robots, AI-generated content (AIGC), conversational interaction, smart healthcare, smart customer service, and game AI. It is believed that with the development of technology, AI will be applied in more fields and play an increasingly important role.

[0065] Next, we will introduce reinforcement learning in detail.

[0066] Reinforcement learning is a field within machine learning that emphasizes how to act based on the environment to maximize expected rewards. The theoretical foundation of reinforcement learning is the Markov Decision Process (MDP).

[0067] A Markov Decision Process (MDP) is a mathematical model used to describe decision problems. It includes elements such as state, action, transition probability, reward, and discount factor. The MDP assumes that the system satisfies the Markov property, meaning that the next state depends only on the current state and action, and is independent of previous states and actions.

[0068] The Markov decision process mainly consists of the following components:

[0069] 1. State: A state is a configuration or description of a system. In MDP, the set of states is usually represented by S.

[0070] 2. Action: An action is an operation that an agent can perform in a given state. In MDP, the set of actions is usually represented by A.

[0071] 3. Transition Probability: Transition probability describes the probability that the system will transition to the next state given a current state and an action. In MDP, transition probability is usually represented by P, which can be expressed as P(s'|s,a), where s and s' represent the current state and the next state, respectively, and a represents the action.

[0072] 4. Reward: The reward is the incentive an agent receives for performing an action and transitioning from one state to another. In MDP, the reward function is usually represented by R, which can be expressed as R(s,a,s'), where s and s' represent the current state and the next state, respectively, and a represents the action.

[0073] 5. Discount Factor: The discount factor is a value between 0 and 1 used to adjust the importance of future rewards. The closer the discount factor is to 1, the more the agent focuses on long-term returns; the closer the discount factor is to 0, the more the agent focuses on short-term returns.

[0074] 6. The Q-function (also known as the Action-State Value Function) is a function used in reinforcement learning to evaluate the expected reward of taking an action in a given state. The Q-function is based on state-action pairs and measures the merit of performing an action in a given state. Based on the value of the Q-function, the agent can choose the optimal action to maximize the cumulative reward. The Q-function is typically represented as Q(s,a), where s represents the state and a represents the action. The value of the Q-function represents the expected future cumulative reward after taking action a in state s. In reinforcement learning, the agent's goal is to find a policy that maximizes the value of the Q-function by taking the appropriate action in each state.

[0075] Many algorithms in reinforcement learning (such as Q-learning and Deep Q-Networks) are based on the optimization of the Q-function. The core idea of ​​these algorithms is to progressively update and optimize the Q-function through interaction with the environment and observed rewards.

[0076] Deep reinforcement learning is a subfield of reinforcement learning that uses deep learning techniques (such as deep neural networks) to represent and learn the generation of actions. Deep learning techniques enable agents to handle more complex, higher-dimensional state and action spaces, thereby solving more complex problems.

[0077] In the content recommendation scenario provided in this application embodiment, the selection of content to be recommended is viewed as a decision-making process. A specific MDP mathematical model is obtained through appropriate definitions of (state, action, policy, reward). Then, deep reinforcement learning techniques are used to train and solve this decision-making problem.

[0078] Figure 1 A schematic diagram of the structure of a computer system provided in an embodiment of this application is shown. The computer system may include: computer device 11.

[0079] The computer device 11 can acquire sample input information, and then train a target recommendation model based on the sample input information using the training method of the recommendation model provided in this application embodiment. The computer device 11 can also use the trained target recommendation model to recommend target content to the target interactive object based on the content recommendation method provided in this application embodiment.

[0080] The computer device 11 can be a terminal or a server, and this application embodiment does not limit this. For example, the terminal can be any electronic product capable of human-computer interaction with the user through one or more methods such as a keyboard, touchpad, touchscreen, remote control, voice interaction, or handwriting device, such as a PC (Personal Computer), mobile phone, smartphone, PDA (Personal Digital Assistant), wearable device, PPC (Pocket PC), tablet computer, smart car system, smart TV, smart speaker, in-vehicle terminal, etc. The server can be a single server, a server cluster consisting of multiple servers, or a cloud computing service center.

[0081] It should be noted that the computer device 11 used to execute the training method of the recommendation model provided in the embodiments of this application and the computer device 11 used to execute the content recommendation method provided in the embodiments of this application can be the same computer device or different computer devices, and the embodiments of this application do not limit this.

[0082] Taking the computer device 11 used to execute the training method of the recommendation model provided in the embodiments of this application and the computer device 11 used to execute the content recommendation method provided in the embodiments of this application as the same computer device as an example. See Figure 1 In the training phase 101 of the recommendation model, the computer device 11 acquires sample input information, inputs it into the initial recommendation model for processing, and obtains the predicted recommendation content output by the initial recommendation model. Then, the computer device 11 acquires a diversity metric based on the predicted recommendation content, acquires a training loss based on the diversity metric, and uses the training loss to update the parameters of the initial recommendation model until the target recommendation model is obtained. In the content recommendation phase 102, the computer device 11 acquires target input information, inputs it into the trained target recommendation model for processing, obtains the target recommendation content output by the target recommendation model, and then recommends the target recommendation content to the target interaction object.

[0083] Those skilled in the art should understand that the computer device 11 described above is merely an example, and other existing or future terminals or servers that are applicable to this application should also be included within the scope of protection of this application, and are hereby incorporated by reference.

[0084] This application provides a method for training a recommendation model, which can be applied to the above-mentioned... Figure 1 The computer system shown. Taking the application of this method to a computer device as an example, such as... Figure 2 As shown, the training method of the recommendation model provided in this application embodiment may include the following steps 201 to 205.

[0085] In step 201, the sample input information of the sample interaction object is obtained.

[0086] Sample input information refers to information related to sample interaction objects used to train the recommendation model. A sample interaction object is any interactive object that can view the content recommended by the recommendation model. In some embodiments, the interaction object may refer to a user. The recommendation model is a model for recommending content. It should be noted that this application embodiment does not limit the type of content recommended by the recommendation model. For example, the type of content recommended by the recommendation model may include, but is not limited to, news, videos, audio, images, and item information.

[0087] For example, obtaining sample input information of a sample interaction object may refer to extracting pre-stored sample input information or to obtaining sample input information of a sample interaction object in real time. This application embodiment does not limit this.

[0088] Taking the real-time acquisition of sample input information of a sample interaction object as an example, in one possible implementation, the process of acquiring the sample input information of the sample interaction object includes the following steps 2011 and 2013:

[0089] Step 2011: Obtain the first representation information, which is used to represent the sample interaction object.

[0090] The first representational information is information that represents the sample interaction object. In some embodiments, the first representational information may also be referred to as the basic profile of the sample interaction object.

[0091] The method for obtaining the first characterization information can be set based on experience or flexibly adjusted according to the application scenario. This application does not limit this method.

[0092] In one possible implementation, the acquisition of the first representation information includes: acquiring the attribute information of the sample interaction object and the statistical information of the sample interaction object under at least one time window. Specifically, the statistical information of the sample interaction object under any time window is determined based on the interaction between the sample interaction object and historical recommended content under any time window; the first representation information is acquired based on the attribute information of the sample interaction object and the statistical information of the sample interaction object under at least one time window.

[0093] When obtaining the first representation information, not only the attribute information of the sample interaction object is considered, but also the statistical information determined based on the interaction between the sample interaction object and the historical recommended content. The first representation information obtained in this way can comprehensively represent the sample interaction object from the perspectives of the basic attributes of the sample interaction object and the sample interaction object's preference for the recommended content. The reliability of the first representation information is high, thereby improving the reliability of the sample input information and improving the training effect of the recommendation model.

[0094] The attribute information of the sample interaction object is obtained based on the basic attributes of the sample interaction object. The types of the basic attributes of the sample interaction object can be set based on experience or flexibly adjusted according to the application scenario. This application embodiment does not limit this. For example, the attribute information of the sample interaction object may include at least one of the following: activity level and content category of interest. Activity level is used to measure the activity level of the sample interaction object in viewing historical recommended content. For example, activity level can be high, medium, or low. Content category of interest can be one or more categories from the candidate categories. Candidate categories are all categories to which the recommendable content belongs. Candidate categories can be set based on experience or flexibly adjusted according to the application scenario. This application embodiment does not limit this. For example, candidate categories may include news, entertainment, sports, education, film and television, music, etc.

[0095] For example, the sample interaction object views recommended content based on the application. Before viewing the recommended content, the sample interaction object will first register an account on the application. After the sample interaction object logs into the application using the registered account, the application can collect the interaction between the sample interaction object and the recommended content. The activity level and the categories of content of interest of the sample interaction object can be obtained by analyzing the interaction between the sample interaction object and the recommended content.

[0096] In an exemplary embodiment, the method for determining the activity level of a sample interaction object includes: determining a first quantity based on the interaction between the sample interaction object and the recommended content, wherein the first quantity is the number of recommended content viewed by the sample interaction object within a historical time period; and determining the activity level of the sample interaction object based on the first quantity. The historical time period can be set based on experience or flexibly adjusted according to the application scenario. This embodiment does not limit this; for example, the historical time period can refer to 30 days or 1 year, etc.

[0097] In some embodiments, the activity level of a sample interaction object can be determined based on a first quantity as follows: if the first quantity is greater than a first quantity threshold, the activity level of the sample interaction object is determined to be high; if the first quantity is greater than a second quantity threshold but not greater than the first quantity threshold, the activity level of the sample interaction object is determined to be medium; if the first quantity is not greater than the second quantity threshold, the activity level of the sample interaction object is determined to be low. The second quantity threshold is less than the first quantity threshold. The first and second quantity thresholds can be set empirically or flexibly adjusted according to the application scenario, and this application embodiment does not limit them. For example, the first quantity threshold can be 10 and the second quantity threshold can be 3; or, for another example, the first quantity threshold can be 30 and the second quantity threshold can be 10.

[0098] In other embodiments, the activity level of a sample interaction object is determined based on a first quantity by: determining the ratio of a first quantity to a second quantity; if the ratio is greater than a first ratio threshold, the activity level of the sample interaction object is determined to be high; if the ratio is greater than a second ratio threshold but not greater than a first ratio threshold, the activity level of the sample interaction object is determined to be medium; if the ratio is not greater than a second ratio threshold, the activity level of the sample interaction object is determined to be low. The second quantity is the total number of historical recommended content items recommended to the sample interaction object within a historical time period. The second ratio threshold is less than the first ratio threshold. The first and second ratio thresholds can be set based on experience or flexibly adjusted according to the application scenario, and this application embodiment does not limit this. For example, the first ratio threshold can be 0.5, and the second ratio threshold can be 0.1; as another example, the first ratio threshold can be 0.8, and the second ratio threshold can be 0.3.

[0099] In an exemplary embodiment, the method for determining the content categories of interest for a sample interaction object includes: determining the categories to which each of the recommended content viewed by the sample interaction object belongs within the historical recommended content recommended to the sample interaction object during a historical time period, based on the interaction between the sample interaction object and the recommended content; determining the interest value of each category in the candidate categories based on the categories to which the recommended content viewed by the sample interaction object belongs; and selecting the top H categories with the highest interest values ​​as the content categories of interest for the sample interaction object. Wherein, the interest value of any category in the candidate categories is the number of recommended content belonging to that category viewed by the sample interaction object. H is an integer not less than 1, and the value of H can be set empirically or flexibly adjusted according to the application scenario; this embodiment does not limit this.

[0100] For example, the application collects information on the interaction between the sample interaction object and the recommended content, subject to the sample interaction object's authorization. Before collecting this information, the application obtains the sample interaction object's separate permission or consent through pop-ups or redirection to a confirmation page. After explicitly obtaining the sample interaction object's separate permission or consent, the application confirms that the sample interaction object authorizes the application to collect this information. Only then does the application collect this information.

[0101] The application can be any application capable of recommending content to interactive objects. This application embodiment does not limit the type of application. For example, the application can be a social application, a video playback application, etc.

[0102] At least one time window is a time window determined based on a first time, which refers to the time when the first representation information is acquired. The number of at least one time window and the association between each time window and the first time can be set based on experience or flexibly adjusted according to the application scenario. This application embodiment does not limit this.

[0103] For example, the number of at least three time windows is: a time window with an interval of no more than 6 hours from the first time, a time window with an interval of no more than 1 day from the first time, and a time window with an interval of no more than 7 days from the first time. Of course, the at least one time window can also be other cases.

[0104] The statistical information of the sample interaction object within any given time window is determined based on the interaction between the sample interaction object and the historical recommended content within that time window. The historical recommended content within any given time window refers to the content recommended to the sample interaction object within that given time window. In an exemplary embodiment, for any historical recommended content, the interaction between the sample interaction object and that historical recommended content is related to how the application recommends that historical recommended content to the sample interaction object.

[0105] For example, the application can recommend any historical recommended content to the sample interactive object by displaying an entry point for viewing that historical recommended content. The sample interactive object can then view the historical recommended content by triggering the entry point, or it can choose not to trigger the entry point (i.e., not view the historical recommended content). In this case, the interaction between the sample interactive object and the historical recommended content can include: the sample interactive object viewing the historical recommended content; or, the sample interactive object not viewing the historical recommended content.

[0106] The format of the entry point for viewing any historical recommended content can be set based on experience or flexibly adjusted according to the application scenario; this application embodiment does not limit this. For example, the entry point for viewing any historical recommended content can be a text box.

[0107] For example, the application can recommend any historical recommended content to the sample interactive object in another way: the application displays a prompt message for the historical recommended content to the sample interactive object. The prompt message indicates to the sample interactive object that any historical recommended content can be viewed. For example, after the application displays the prompt message for the historical recommended content to the sample interactive object, the sample interactive object may choose not to trigger the prompt message, or it may trigger it. In addition to triggering the prompt message for the historical recommended content, the sample interactive object can enter a historical content display page. This page displays at least one entry point for viewing historical content, including the historical recommended content. For example, the historical content display page can be an information stream display page, where each entry point is used to view one piece of information (i.e., one piece of content) within the information stream.

[0108] After entering the historical content display page, the sample interactive object can view any historical recommended content by triggering the view entry for any historical recommended content, or view other content by triggering the view entry for other content besides any historical recommended content, or it can choose not to trigger any content view entry (i.e., not to view any content).

[0109] Based on the above analysis, it can be seen that when an application recommends any historical recommended content to a sample interactive object, it refers to the situation where the application displays a prompt message for that historical recommended content to the sample interactive object. The interaction between the sample interactive object and that historical recommended content can include the following four situations:

[0110] Scenario 1: After displaying the prompt message for any historical recommended content to the sample interactive object, the sample interactive object does not trigger the prompt message for any historical recommended content.

[0111] Scenario 2: After displaying a prompt message for any historical recommended content to the sample interactive object, the sample interactive object triggers the prompt message for any historical recommended content and enters the historical content display page, and the sample interactive object does not view any content on the historical content display page.

[0112] Scenario 3: After displaying a prompt message for any historical recommended content to the sample interaction object, the sample interaction object triggers the prompt message for any historical recommended content and enters the historical content display page, and the sample interaction object views other content besides any historical recommended content on the historical content display page.

[0113] Scenario 4: After displaying a prompt message for any historical recommended content to the sample interaction object, the sample interaction object triggers the prompt message for any historical recommended content and enters the historical content display page, and the sample interaction object views any historical recommended content on the historical content display page.

[0114] The statistical information of the sample interaction object within any time window is obtained by statistically analyzing the interaction between the sample interaction object and the historical recommended content within that time window. The type of statistical information of the sample interaction object within any time window can be set based on experience or flexibly adjusted according to the application scenario; this application embodiment does not limit this.

[0115] For example, when an application recommends any historical recommended content to a sample interactive object, it means that the application displays an entry point for viewing that historical recommended content to the sample interactive object. The statistical information of the sample interactive object within any time window may include at least one of the following: the total number of historical recommended contents viewed by the sample interactive object within that time window, and the proportion of historical recommended contents viewed by the sample interactive object within that time window. Specifically, the proportion of historical recommended contents viewed by the sample interactive object within that time window is the ratio of the number of historical recommended contents viewed by the sample interactive object within that time window to the total number of all historical recommended contents within that time window.

[0116] In some embodiments, the total number of historical recommended contents viewed by sample interaction objects within any time window can also be referred to as the total number of views of historical recommended contents within any time window; the proportion of historical recommended contents viewed by sample interaction objects within any time window can also be referred to as the viewing rate of historical recommended contents within any time window.

[0117] For example, when an application recommends any historical recommended content to a sample interactive object, it means that the application displays a prompt message for that historical recommended content to the sample interactive object. The statistical information of the sample interactive object under any time window may include at least one of the following: the total number of historical recommended content items under any time window that triggered the prompt message, the proportion of historical recommended content items under any time window that triggered the prompt message, the total number of historical recommended content items under any time window that were viewed by the sample interactive object, and the proportion of historical recommended content items under any time window that were viewed by the sample interactive object.

[0118] The proportion of content in the historical recommended content that was triggered by the sample interaction object within any time window refers to the ratio of the number of content in the historical recommended content that was triggered by the sample interaction object within any time window to the total number of content in the historical recommended content within any time window.

[0119] In some embodiments, the prompt information for historical recommended content can be displayed together with a prompt marker. The prompt marker can be any marker that serves a prompting function; for example, the prompt marker can be a red dot (referred to as a red dot). Exemplarily, triggering the prompt information can be achieved by clicking the red dot. In this case, the total number of historical recommended content items within any given time window whose prompt information is triggered by the sample interaction object can also be referred to as the total number of red dot clicks within any given time window, and the proportion of historical recommended content items within any given time window whose prompt information is triggered by the sample interaction object can also be referred to as the red dot exposure click-through rate within any given time window.

[0120] In an exemplary embodiment, the first representation information can be obtained based on the attribute information of the sample interaction object and the statistical information of the sample interaction object within at least one time window by arranging the attribute information of the sample interaction object and the statistical information of the sample interaction object within at least one time window in a certain order. The order in which the arrangement is based can be set based on experience or flexibly adjusted according to needs. In some embodiments, arranging the attribute information of the sample interaction object and the statistical information of the sample interaction object within at least one time window in a certain order can also be referred to as concatenating the attribute information of the sample interaction object and the statistical information of the sample interaction object within at least one time window.

[0121] In an exemplary embodiment, the attribute information of the sample interaction object includes the sample interaction object's age, gender, activity level, and categories of content it is interested in. The number of at least three time windows is used as an example. The statistical information of the sample interaction object under each time window includes the total number of red dot clicks and the red dot exposure click-through rate under each time window. The first characterization information includes 10-dimensional features.

[0122] The above-described methods for obtaining the first characterization information are merely illustrative examples, and the embodiments of this application are not limited thereto. In some embodiments, the method for obtaining the first characterization information may also be: using the attribute information of the sample interaction object as the first characterization information. In other embodiments, the method for obtaining the first characterization information may also be: arranging (or splicing) the statistical information of the sample interaction object in at least one time window in a certain order to obtain the first characterization information.

[0123] Step 2012: Obtain the second representation information, which is used to represent the sample candidate content of the sample interaction object.

[0124] The second representation information is information that represents the candidate content of the samples, which is the content that can be recommended to the sample interaction object. The candidate content of the sample interaction object can be set based on experience or flexibly adjusted according to the application scenario, and this application embodiment does not limit it in this way.

[0125] In an exemplary embodiment, the sample candidate content of the sample interaction object can be the original content to be recommended. The original content to be recommended can refer to content that can be recommended to the sample interaction object without any screening. For example, the original content to be recommended can refer to all content generated in the application during a historical time period that has not been recommended to the sample interaction object. The historical time period can be set based on experience or flexibly adjusted according to the application scenario. For example, the historical time period can be 5 days or 10 days, etc.

[0126] For example, in the case where the application is a dedicated application for publishing and viewing content, the content generated by the application refers to the content published by the content publishing object using the application.

[0127] For example, in the case of an application that supports both content publishing and viewing services, as well as other services, the content generated by the application refers to the content generated under the content publishing and viewing services provided by the application. For instance, the application could be a social application, which supports both content publishing and viewing services, and social services (such as chat, voice calls, video calls, etc.). The content generated by a social application could refer to the content published by information dissemination accounts followed by the sample interaction object within the social application. In some embodiments, information dissemination accounts can also be called subscription accounts, and the content published by these accounts can be called subscription account messages.

[0128] For example, the content generated by the application can be the same or different for different interactive objects.

[0129] In an exemplary embodiment, the candidate content for the sample interaction object can be the content obtained after performing at least one round of filtering operations on the original content to be recommended. In this case, the number of candidate content is relatively small, which helps to reduce the amount of data in the second representation information, reduce the amount of data in the sample input information, reduce the computational load of the recommendation model training process, and improve the training efficiency of the recommendation model. In addition, the candidate content is relatively reliable content obtained after filtering from the original content to be recommended. Training the recommendation model based on highly reliable candidate content helps to improve the training reliability of the recommendation model.

[0130] The requirement for at least one round of screening can be set based on experience or adjusted flexibly according to the application scenario; this application does not limit this.

[0131] For example, at least one round of screening operation refers to three rounds of screening operations: recall, coarse ranking, and fine ranking, in sequence. The implementation of this application does not limit the processes of recall, coarse ranking, and fine ranking, and can be flexibly set according to actual needs. In this case, the process of the initial recommendation model recommending and predicting content can be regarded as a recommendation process built upon the fine ranking stage. Of course, at least one round of screening operation can also be other cases. For example, at least one round of screening operation refers to two rounds of screening operations: recall and coarse ranking, in sequence; or, for another example, at least one round of screening operation refers to the recall round of screening operation.

[0132] For example, the sample candidate content can be all the content obtained after performing at least one round of filtering on the original recommended content, or it can be a portion of the content obtained after performing at least one round of filtering on the original recommended content. For example, a portion of the content obtained after performing at least one round of filtering on the original recommended content can refer to the top K content obtained after performing at least one round of filtering on the original recommended content, where K is an integer not greater than N, and N is the number of content obtained after performing at least one round of filtering on the original recommended content.

[0133] For example, taking a three-round selection process consisting of recall, coarse ranking, and fine ranking as an example, the sample candidate content can be the top 30 items obtained after performing recall, coarse ranking, and fine ranking on the original recommendation content. Using the top 30 items from the fine-ranked content as sample candidates has the advantage of reducing the size of the recommendation content candidate set, facilitating real-time inference during the recommendation model's learning process, and providing a safety net for the quality and effectiveness of the recommendations. For example, the fine ranking stage generates content feature vectors (embeddings). These feature vectors can be stored for training the recommendation model because they contain high-order semantic information and can be directly used for modeling content diversity. For example, the content feature vector is a numerical vector composed of multiple floating-point numbers, capable of describing various attributes and properties of the content in high-dimensional space.

[0134] For example, recommendation links can be like Figure 3 As shown. After obtaining the original content to be recommended, the original content to be recommended is recalled to obtain the recalled content; the recalled content is coarsely ranked to obtain the coarsely ranked content; the coarsely ranked content is finely ranked to obtain the finely ranked content; the top K content in the finely ranked content is used as the sample candidate content, and the initial recommendation model selects the content to recommend to the sample interaction object from the sample candidate content.

[0135] The number of recalled content items is greater than the number of coarse-ranked content items, the number of coarse-ranked content items is greater than the number of fine-ranked content items, and the number of fine-ranked content items is greater than the number of content items recommended by the initial recommendation model for the sample interaction objects. For example, see... Figure 3 The number of recalled content items was 10,000+, the number of coarse-ranked content items was 1,000+, the number of fine-ranked content items was 100+, and the number of content items recommended by the initial recommendation model based on the sample interaction object was 1.

[0136] In one possible implementation, the number of sample candidate contents is at least two, and the second representation information is obtained by: for any sample candidate content among the at least two sample candidate contents, obtaining the representation sub-information of any sample candidate content based on the feature vector of any sample candidate content; and obtaining the second representation information based on the representation sub-information of the at least two sample candidate contents.

[0137] The feature vector of any candidate sample is a vector obtained after extracting features from any candidate sample. For example, the feature vector of any candidate sample can be obtained and stored during at least one round of filtering operations on the original content to be recommended, or it can be obtained in real time.

[0138] For example, the process of extracting features from any candidate sample content can be achieved by calling a content feature extraction model. The content feature extraction model can be a pre-trained model capable of extracting feature vectors from the content. The content feature extraction model can be any type of neural network model, and this application embodiment does not limit it.

[0139] The representational information of any candidate sample content is used to represent any candidate sample content. For example, based on the feature vector of any candidate sample content, the representational information of any candidate sample content can be obtained by using the feature vector of any candidate sample content as the representational information of any candidate sample content.

[0140] For example, the way to obtain the representational information of any sample candidate content based on the feature vector of any sample candidate content can be: based on at least one of the category of any sample candidate content or the ranking position of any sample candidate content among at least two sample candidate contents, and the feature vector of any sample candidate content, the representational information of any sample candidate content can be obtained.

[0141] Based on this, the representational information of the sample candidate content not only represents the sample candidate content from the perspective of the feature vector of the sample candidate content, but also represents the sample candidate content from the perspective of the category and / or ranking position of the sample candidate content. This enables the representational information of the sample candidate content to represent the sample candidate content more comprehensively, improves the reliability of the second representational information, and thus improves the reliability of the sample input information and the training effect of the recommendation model.

[0142] The category of any candidate sample content refers to one or more categories to which the candidate sample content belongs within the candidate categories. The candidate categories are all categories to which the recommendable content belongs. The candidate categories can be set based on experience or flexibly adjusted according to the application scenario. This application embodiment does not limit this. For example, candidate categories may include news, entertainment, sports, education, film and television, music, etc.

[0143] The ranking position of any candidate sample among at least two candidate samples is determined by the arrangement order of the at least two candidate samples. The arrangement order of the at least two candidate samples can be set based on experience or flexibly adjusted according to the application scenario, and this application embodiment does not limit it in this regard. For example, the arrangement order of the at least two candidate samples may refer to the arrangement order of the at least two candidate samples after performing at least one round of screening operations.

[0144] For example, the representational information of any candidate sample content can be obtained by taking at least one of the categories of any candidate sample content or the ranking position of any candidate sample content among at least two candidate sample contents, and the feature vector of any candidate sample content.

[0145] For example, with 30 candidate sample contents, the representation information of any candidate sample contents is determined based on the category of any candidate sample contents, the ranking position of any candidate sample contents among at least two candidate sample contents, and the feature vector of any candidate sample contents. Taking an 8-dimensional vector as an example, the second representation information includes 300 (30*1+30*1+30*8=300) dimensional features.

[0146] Step 2013: Obtain sample input information based on the first and second characterization information.

[0147] The sample input information is obtained by comprehensively considering information representing the sample interaction object and information representing the sample candidate content. This comprehensive approach ensures high reliability of the sample input information, thus improving the training reliability of the recommendation model. The initial recommendation model can directly determine from which candidate content to recommend content to the sample interaction object based on this sample input information, without needing to provide separate information about the candidate content. This simplifies the training logic of the recommendation model and improves its training efficiency.

[0148] In one possible implementation, based on the first and second characterization information, the sample input information can be obtained by arranging the first and second characterization information in a certain order (or concatenating them). This method obtains the sample input information without needing to acquire other information, thus improving the efficiency of obtaining sample input information.

[0149] In another possible implementation, the sample input information can be obtained based on the first and second representation information by: obtaining at least one of the third representation information or the time representation information; or obtaining the sample input information based on at least one of the third or the time representation information, the first representation information, and the second representation information. The third representation information is used to represent the historical recommended content of the sample interaction object; the time representation information is determined based on the generation time of the sample input information and the latest time when the sample interaction object generates a reference interaction behavior in response to the historical recommended content.

[0150] Based on this, the process of obtaining sample input information not only considers information representing the sample interaction object and information representing the sample candidate content, but also considers time-related information and / or information related to historical recommendation content. The information considered is more comprehensive, which helps to improve the reliability of sample input information and thus improve the training effect of the recommendation model.

[0151] The third representation information is used to represent the historical recommended content of the sample interaction object. Historical recommended content refers to content that has been recommended to the sample interaction object. For example, historical recommended content refers to the content that has been recommended to the sample interaction object the most recent M times, where M is a positive integer. For example, for the content that has been recommended to the sample interaction object, each piece of content corresponds to a recommendation time. The content that has been recommended to the sample interaction object the most recent M times refers to the top M pieces of content with the shortest time interval among those previously recommended, where the time interval refers to the time interval between the recommendation time of the content and the time when the third representation information was obtained.

[0152] For example, the third representation information can be determined based on the feature vectors of historical recommended content. The feature vectors of historical recommended content can be obtained by extracting features from the historical recommended content. For instance, the feature vectors of historical recommended content can be used as the third representation information. Another example is determining the third representation information based on at least one of the following: the recommendation time of the historical recommended content, the category of the historical recommended content, and the feature vectors of the historical recommended content. By considering the third representation information, a basis can be provided for subsequent modeling of diversity metrics, thereby improving the training effect of the recommendation model.

[0153] Taking a historical recommendation content of 30 items as an example, and the third representation information as the feature vector of the historical recommendation content, which is an 8-dimensional vector, the third representation information can include 240 (30*8=240) dimensional features.

[0154] The generation time of sample input information refers to the time when the sample input information is generated. Sample input information can be pre-generated and stored, or it can be acquired in real time. If the sample input information is acquired in real time, the generation time can also refer to the acquisition time, i.e., the current time.

[0155] The reference interaction behavior can be set based on experience or flexibly adjusted according to the application scenario; this application embodiment does not limit it in this regard. For example, the reference interaction behavior can refer to the behavior that triggers a prompt message for historical recommended content, or it can refer to the behavior of viewing historical recommended content, etc. The latest time when the sample interaction object generates a reference interaction behavior for historical recommended content is the time interval between the time when the sample interaction object generates a reference interaction behavior for historical recommended content and the current time.

[0156] For example, the time representation information is determined based on the generation time of the sample input information, the latest time when the sample interaction object generates a reference interaction behavior in response to historical recommended content, and the time interval between the generation time and the latest time. For instance, the time representation information is obtained by arranging (or concatenating) the generation time of the sample input information, the latest time when the sample interaction object generates a reference interaction behavior in response to historical recommended content, and the time interval between the generation time and the latest time in a certain order.

[0157] For example, the time of creation and the latest time can be represented by a timestamp. Alternatively, the time of creation and the latest time can also be represented by information obtained after converting the timestamp. Converting a timestamp can refer to transforming it into information in three dimensions: the day of the year, the hour of the day (24-hour format), and the minute of the hour (60-minute format). For example, the unit of time interval can be minutes.

[0158] Taking the generation time and the latest time as examples, which are obtained by converting the timestamp into three dimensions of information, and the time interval as an example, the time representation information can include seven-dimensional features.

[0159] For example, taking sample input information based on first representation information, second representation information, third representation information, and time representation information, assuming the first representation information includes 10-dimensional features, the second representation information includes 300-dimensional features, the third representation information includes 240-dimensional features, and the time representation information includes 7-dimensional features, the sample input information is obtained by concatenating the first, second, third, and time representation information together. Therefore, the sample input information includes 557 (10+300+7+240=557)-dimensional features. For example, the second and third representation information can be considered content-related information, and the content-related information includes 540 (300+240=540)-dimensional features.

[0160] For example, for information that cannot be collected, such as the initial stage of the training method or when the sample interaction object is a new user, the corresponding information can be set to a default value. The default values ​​for different information can be the same or different. For example, the default values ​​for different information can all be set to 0.

[0161] It should be noted that the sample input information obtained in step 201 is the input information required to update the parameters of the initial recommendation model once. The number of sample input information can be one or multiple, and this embodiment does not limit this. For example, each sample interaction object has one sample input information, and obtaining multiple sample input information means obtaining the sample input information of each of the multiple sample interaction objects. The principle of obtaining sample input information of different sample interaction objects is the same.

[0162] In step 202, the initial recommendation model is invoked to process the sample input information and obtain the predicted recommendation content.

[0163] After obtaining the sample input information, the sample input information is fed into the initial recommendation model for processing. The initial recommendation model can output predicted recommendation content. For example, the predicted recommendation content is the recommended content selected by the initial recommendation model for the sample interaction object from the sample candidate content.

[0164] In one possible implementation, after the initial recommendation model processes the sample input information, it only obtains the predicted recommendation content. That is, after inputting the sample input information into the initial recommendation model, the model only outputs the predicted recommendation content. For example, after inputting the sample input information into the initial recommendation model, the model processes the sample input information to obtain the probability of the candidate content, and selects the content with the highest probability as the predicted recommendation content. This approach simplifies the internal processing logic of the initial recommendation model, reduces the computational load during recommendation model training, and improves the training efficiency of the recommendation model.

[0165] For example, in the case where only the predicted recommendation content is obtained after processing the sample input information by calling the initial recommendation model, the initial recommendation model can be a whole model, which can be any kind of neural network model.

[0166] In another possible implementation, the initial recommendation model is invoked to process the sample input information to obtain the predicted recommendation content and the prediction evaluation index of the predicted recommendation content. The prediction evaluation index is the predicted value of the comprehensive effect of recommending the predicted recommendation content based on the sample input information.

[0167] In other words, after inputting the sample information into the initial recommendation model, the initial recommendation model not only outputs the predicted recommendation content, but also the prediction evaluation index of the predicted recommendation content. In this way, the initial recommendation model outputs richer content, providing more reference information for the training process of the recommendation model and improving the training effect.

[0168] The prediction evaluation metric is the overall effect of the initial recommendation model's prediction of recommended content based on the sample input information. This overall effect is a comprehensive evaluation of the recommendation action itself, and can be used to assess the quality of this recommendation action.

[0169] For example, the overall effect of recommending and predicting content based on sample input information is a comprehensive consideration of the direct and indirect effects of this recommendation. The direct effect refers to the direct impact of the recommendation action itself, such as how the predicted content influences the interaction behavior of the sample user or the degree of difference between already recommended content.

[0170] The indirect effect of recommending content based on sample input information refers to the indirect impact of the recommendation action itself. This indirect impact arises from subsequent content recommendations made after updating the input information. Updating the input information refers to the input information of the sample's interactive object after recommending the predicted content based on the sample's input information. In other words, the indirect effect is not the influence of the predicted content itself, but rather the impact of subsequent content recommendations after the predicted content has been recommended.

[0171] The direct effect can be understood as the current effect brought about by the action of making recommendations and predicting recommended content based on the sample input information, while the indirect effect can be understood as the future effect brought about by the action of making recommendations and predicting recommended content based on the sample input information.

[0172] In an exemplary embodiment, the initial recommendation model can be a reinforcement learning model, and the prediction evaluation metric can refer to the Q-function value output by the initial recommendation model. This Q-function value represents the overall effect predicted by the initial recommendation model after taking the action of recommending content based on the sample input information. For example, the overall effect can also be referred to as cumulative reward, cumulative return, etc.; the direct effect can also be referred to as short-term reward, direct reward, short-term return, direct return, etc.; and the indirect effect can also be referred to as long-term reward, long-term reward, long-term return, long-term return, future reward, future return, etc.

[0173] In an exemplary embodiment, when the initial recommendation model is invoked to process the sample input information and obtain the predicted recommendation content and the predicted evaluation index of the predicted recommendation content, the initial recommendation model can be a holistic model, which can be any type of neural network model. After the sample input information is input into this holistic model, the holistic model outputs the predicted recommendation content and the predicted evaluation index.

[0174] In an exemplary embodiment, when the initial recommendation model is invoked to process sample input information to obtain predicted recommendation content and a prediction evaluation index for the predicted recommendation content, the initial recommendation model includes a first initial sub-model and a second initial sub-model. In this case, invoking the initial recommendation model to process sample input information to obtain predicted recommendation content and a prediction evaluation index for the predicted recommendation content includes: invoking the first initial sub-model to process the sample input information to obtain predicted recommendation content; and invoking the second initial sub-model to process the predicted recommendation content and the sample input information to obtain the prediction evaluation index for the predicted recommendation content. The initial recommendation model has two different initial sub-models to output predicted recommendation content and prediction evaluation index respectively. Based on this, during the training process of the initial recommendation model, the first initial sub-model used to output predicted recommendation content and the second initial sub-model used to output prediction evaluation index can be optimized in a targeted manner, thereby improving the training effect of the recommendation model.

[0175] The first initial sub-model can be any model capable of outputting recommended content based on input information. The structure of the first initial sub-model can be set empirically or flexibly adjusted according to the application scenario; this application does not limit this. The first initial sub-model can be any neural network model. For example, the first initial sub-model can use a simple multi-layer fully connected network as its basic component, using an activation function (e.g., a softmax function) in the output layer to output the score (or probability) of the candidate sample content, and using the content with the highest score (or probability) as the final predicted recommended content.

[0176] In some embodiments, the process by which the first initial sub-model outputs predicted recommended content based on sample input information can be referred to as an action, and the predicted recommended content is the object selected by this action.

[0177] In some embodiments, the sample input information can be represented as S, the sample candidate content can be represented as C, the first initial sub-model can be represented as ActorNet, and the action of outputting the predicted recommended content based on the sample input information can be represented as a. Then, the process of the first initial sub-model outputting the predicted recommended content based on the sample input information can be represented as a = ActorNet(S,C).

[0178] The second initial sub-model is any model capable of outputting an evaluation metric for recommended content based on input information and recommended content. The structure of the second initial sub-model can be set empirically or flexibly adjusted according to the application scenario; this application embodiment does not limit this. The second initial sub-model can be any neural network model. For example, the second initial sub-model can use a simple multi-layer fully connected network as its basic component, with the output layer being a single-node linear layer used to output the evaluation metric. In some embodiments, the second initial sub-model can be represented as CriticNet.

[0179] In some embodiments, where the initial recommendation model includes a first initial sub-model and a second initial sub-model, the initial recommendation model can be understood as an Actor-Critic structure model.

[0180] In step 203, the diversity measure of the predicted recommended content is obtained. The diversity measure is used to measure the diversity of the predicted recommended content.

[0181] The diversity metric is the metric used to obtain the training loss in the embodiments of this application.

[0182] Diversity metrics are used to measure the diversity of predicted recommended content. In other words, diversity metrics measure the degree of difference between content within a content set, where the content set refers to the collection of predicted and historical recommended content. The greater the degree of difference between content within a content set, the greater the diversity of recommended content. In other words, diversity metrics are positively correlated with the degree of difference between content within a content set.

[0183] By training the recommendation model with consideration of diversity metrics, the model can take into account the degree of difference between the content to be recommended and the historical recommended content during the recommendation process. This avoids the recommendation model recommending similar content every time, which would reduce the interaction interest of the users.

[0184] In an exemplary embodiment, the degree of difference between content within a content set can be determined by calculating the similarity between content within the content set. The similarity between content within a content set is negatively correlated with the degree of difference between content within the content set.

[0185] In one possible implementation, the process of obtaining a measure of the diversity of the predicted recommended content includes steps 2031 to 2033.

[0186] Step 2031: Based on the predicted recommended content and the historical recommended content of the sample interaction object, obtain the similarity matrix. The element in the i-th row and j-th column of the similarity matrix is ​​used to indicate the similarity between the i-th content and the j-th content in the content set composed of the predicted recommended content and the historical recommended content. i and j are integers not greater than n, and n is the number of content in the content set.

[0187] By calculating the similarity between every two items in the content set consisting of predicted and historical recommendations, a similarity matrix can be obtained. The similarity matrix has L*L elements, where L is the number of items in the content set.

[0188] This application does not limit the method of calculating the similarity between two pieces of content. For example, the similarity between two pieces of content can refer to the similarity between their feature vectors, such as the cosine similarity. For example, the feature vectors of content can be obtained by extracting features from the content.

[0189] For example, the similarity between the i-th and j-th content in a content set can be calculated using the cosine similarity based on Formula 1:

[0190]

[0191] Among them, K i,jE represents the cosine similarity between the i-th and j-th content items in the content set. o E represents the feature vector of the i-th content in the content set. j This represents the feature vector of the j-th content in the content set.

[0192] The cosine similarity between two pieces of content ranges from -1 to 1. When the feature vectors of two pieces of content are in the same direction, the cosine similarity is 1; when the feature vectors of two pieces of content are in opposite directions, the cosine similarity is -1; when the feature vectors of two pieces of content are orthogonal (i.e., the feature vectors of two pieces of content are unrelated), the cosine similarity is 0.

[0193] Step 2032: Obtain the determinant calculation result of the similarity matrix. The determinant calculation result is used to measure the degree of difference between the contents in the content set.

[0194] The determinant of a similarity matrix is ​​calculated as the value of the determinant of the similarity matrix. The determinant of a matrix is ​​a function that converts a matrix into a numerical value; its domain is the matrix, and its value is a scalar.

[0195] According to the Determinant Point Process (DPP) theory, a property of the determinant of a matrix is ​​that when rows (or columns) in a matrix are linearly dependent, the determinant value becomes smaller or even zero. If two pieces of content are very similar, their similarity value will be high, causing the corresponding rows (or columns) in the matrix to be more linearly dependent. In this case, the calculated determinant value will be small, indicating less repulsion (i.e., diversity) between these pieces of content. Conversely, if the similarity between the pieces of content is low, their repulsion is stronger, making the rows (or columns) in the similarity matrix more linearly independent. This results in a larger determinant value, reflecting a high degree of dissimilarity. In other words, the determinant calculation result of the similarity matrix can be used to measure the degree of dissimilarity between the content in a content set.

[0196] Step 2033: Based on the determinant calculation results, obtain the diversity measurement index of the predicted recommended content.

[0197] Since the determinant calculation result of the similarity matrix can be used to measure the degree of difference between content in the content set, after obtaining the determinant calculation result, a diversity measure of the predicted recommended content is obtained based on the determinant calculation result.

[0198] This application does not limit the method for obtaining the diversity measure of predicted recommended content based on the determinant calculation result, as long as the diversity measure of predicted recommended content is positively correlated with the determinant calculation result. For example, the determinant calculation result can be directly used as the diversity measure of predicted recommended content. For example, the determinant calculation result can be numerically transformed, and the transformed value can be used as the diversity measure of predicted recommended content. For example, numerical transformation of the determinant calculation result can refer to calculating the logarithm of the determinant calculation result.

[0199] For example, the diversity metric for predicting recommended content can be calculated based on Formula 2:

[0200] Diversity=logdet(K)(Formula 2)

[0201] Where Diversity represents the diversity metric for predicted recommended content; K represents the similarity matrix; det(K) represents the determinant of the similarity matrix; and logdet(K) represents the logarithm of the determinant.

[0202] Calculating the determinant of the similarity matrix to obtain a measure of the diversity of predicted recommendations is a relatively simple calculation, which improves the efficiency of obtaining this measure and thus enhances the training efficiency of the recommendation model. Furthermore, the determinant of the similarity matrix reliably measures the degree of difference between content within a content set. This degree of difference reflects the diversity of predicted recommendations, ensuring the reliability of the diversity measure and consequently improving the training reliability of the recommendation model.

[0203] In step 204, the training loss is obtained based on the diversity metric.

[0204] One possible implementation method for obtaining the training loss based on a diversity metric includes: obtaining the training loss based on both a diversity metric and an accuracy metric. The accuracy metric is used to measure the accuracy of the predicted content recommendations.

[0205] Training loss is obtained by comprehensively considering both diversity and accuracy metrics. This comprehensive approach ensures the reliability of the training loss and, consequently, improves the training reliability of the recommendation model.

[0206] Before obtaining the training loss based on diversity and accuracy metrics, it is necessary to first obtain the accuracy metric. The accuracy metric measures the accuracy of the predicted recommendations. By training the recommendation model with the accuracy metric in mind, the model can consider the accuracy of the recommended content during the recommendation process, thereby improving the accuracy of the recommendations.

[0207] In one possible implementation, the process of obtaining an accuracy metric includes steps 203a and 203b.

[0208] Step 203a: Obtain the interaction information of the predicted recommendation content. The interaction information is used to indicate the interaction between the sample interaction object and the predicted recommendation content.

[0209] For example, after the initial recommendation model outputs predicted recommended content, the predicted recommended content is recommended to the sample interaction object. After recommending the predicted recommended content to the sample interaction object, and with the authorization of the sample interaction object, interaction information is obtained based on the interaction between the sample interaction object and the predicted recommended content. This application embodiment does not limit the method of obtaining interaction information based on the interaction situation, as long as the interaction information can indicate the interaction situation.

[0210] Step 203b: Obtain accuracy metrics based on interaction information.

[0211] After obtaining the interaction information, obtain an accuracy metric for matching the interaction information.

[0212] The interaction between sample interaction objects and predicted recommendation content can reliably reflect the accuracy of the predicted recommendations. Considering this interaction to obtain an accuracy metric helps ensure its reliability. Building upon diverse metrics, further considering a highly reliable accuracy metric to obtain the training loss improves its reliability, thereby enhancing the overall training reliability of the recommendation model.

[0213] In one possible implementation, the process of obtaining an accuracy metric based on interaction information includes: using the metric corresponding to the interaction situation indicated by the interaction information as the accuracy metric. The interaction situation indicated by the interaction information includes a first interaction situation, a second interaction situation, a third interaction situation, or a fourth interaction situation, wherein the metric corresponding to the first interaction situation is less than the metric corresponding to the second interaction situation, the metric corresponding to the second interaction situation is less than the metric corresponding to the third interaction situation, and the metric corresponding to the third interaction situation is less than the metric corresponding to the fourth interaction situation.

[0214] In other words, if the interaction information indicates a first interaction situation, the first metric (the metric corresponding to the first interaction situation) is used as the accuracy metric; if the interaction information indicates a second interaction situation, the second metric (the metric corresponding to the second interaction situation) is used as the accuracy metric; if the interaction information indicates a third interaction situation, the third metric (the metric corresponding to the third interaction situation) is used as the accuracy metric; if the interaction information indicates a fourth interaction situation (the metric corresponding to the fourth interaction situation), the fourth metric is used as the accuracy metric; wherein, the first metric is less than the second metric, the second metric is less than the third metric, and the third metric is less than the fourth metric.

[0215] The first interaction scenario is that after displaying the predicted recommendation content prompt to the sample interaction object, the sample interaction object does not trigger the prompt. The second interaction scenario is that after displaying the predicted recommendation content prompt to the sample interaction object, the sample interaction object triggers the prompt and enters the content display page, but does not view any content on the content display page. The third interaction scenario is that after displaying the predicted recommendation content prompt to the sample interaction object, the sample interaction object triggers the prompt and enters the content display page, and views content other than the predicted recommendation content on the content display page. The fourth interaction scenario is that after displaying the predicted recommendation content prompt to the sample interaction object, the sample interaction object triggers the prompt and enters the content display page, and views the predicted recommendation content on the content display page.

[0216] Detailed descriptions of the first, second, third, and fourth interaction scenarios can be found in step 201 regarding the four interaction scenarios between the sample interaction object and any historical recommended content; they will not be repeated here. The first, second, third, and fourth interaction scenarios represent four levels of interaction behavior, with the interaction intensity increasing sequentially. The greater the interaction intensity indicated by the interaction information, the higher the degree of matching between the predicted recommended content and the sample interaction object; a larger metric will be used as the accuracy metric. In some embodiments, the accuracy metric may also be called a matching degree metric, a degree of matching metric, or a matching accuracy metric, etc.

[0217] The specific values ​​of the first, second, third, and fourth metrics can be set based on experience or adjusted flexibly according to the application scenario. This application does not limit this, as long as the first metric is less than the second metric, the second metric is less than the third metric, and the third metric is less than the fourth metric.

[0218] For example, the first metric has a value of 0, the second metric has a value of 1, the third metric has a value of 2, and the fourth metric has a value of 3. For example, the accuracy metric can be represented as Relevance, and the value of the accuracy metric can be represented as Relevance∈{0,1,2,3}.

[0219] Using different metrics as accuracy measures under different interaction scenarios between sample interaction objects and predicted recommendation content can improve the matching between accuracy measures and specific interaction scenarios. Furthermore, the interaction intensity increases sequentially for the first, second, third, and fourth interaction scenarios, and the corresponding metrics also increase sequentially. Training the recommendation model based on accuracy measures determined in this way encourages the model to recommend content with higher interaction intensity for the sample interaction objects, improving the training effect of the recommendation model. In addition, by considering accuracy measures with high matching with specific interaction scenarios on top of diversity measures, the training loss can be obtained. This helps to improve the matching between the training loss and specific interaction scenarios while ensuring the matching between the training loss and the diversity of predicted recommendation content, thus improving the reliability of the training loss and ultimately enhancing the training effect of the recommendation model.

[0220] For example, the prompt information for the predicted recommended content can be displayed together with a prompt marker. The prompt marker can be any marker that serves a prompting function; for example, the prompt marker can be a red dot (referred to as a red dot). For example, displaying the red dot to the sample interaction object can be called exposing the red dot, triggering the prompt information can be called clicking the red dot, and entering the content display page can be called entering the box. The first interaction scenario mentioned above can refer to the case of exposing the red dot without clicking it; the second interaction scenario mentioned above can refer to the case of exposing and clicking the red dot, entering the box, but exiting directly without clicking any content; the third interaction scenario mentioned above can refer to the case of exposing and clicking the red dot, entering the box, and clicking on non-red dot prompt content (i.e., content other than the predicted recommended content); the fourth interaction scenario mentioned above can refer to the case of exposing and clicking the red dot, entering the box, and clicking on the red dot prompt content (i.e., the predicted recommended content).

[0221] For example, the aforementioned box may refer to a content viewing box provided by an application that can recommend content. Users can see an information stream composed of multiple text and image content, video content, etc. in the box. In other words, the box can be understood as a box containing content.

[0222] For example, taking a social application as an example, a social application supports both content publishing and viewing services, as well as social services (such as chat, voice calls, video calls, etc.). The "box" can refer to the page that the sample interaction object enters after triggering the information delivery account entry point in the social application. Here, the information delivery account entry point is the entry point on the sample interaction object's social application display page used to view content published by the information delivery account. In some embodiments, the information delivery account can also be called a subscription account, the content published by the information delivery account can also be called a subscription account message, and the information delivery account entry point can also be called a subscription account message entry point. Clicking on this entry point leads to a box containing an information stream consisting of multiple text and image content, video content, etc.

[0223] Once inside the box, you are inside; otherwise, you are outside. For social applications, "outside the box" can refer to the main social interface outside of content posted by informational accounts. "Inside the box" can refer to the page displaying content posted by informational accounts.

[0224] Red dot exposure refers to off-box content exposure. When the sample interaction object is within a social application or outside of a news dissemination account, the system generates a red dot and a prompt message (e.g., a short title) that is seen by the sample interaction object. One off-box exposure corresponds to only one red dot and one prompt message.

[0225] The content viewed by the sample interaction object on the content display page can refer to the content exposed within the box. In some embodiments, the length of the exposed content within the box can be obtained by statistically analyzing the longest exposed content list generated by the system and viewed by the sample interaction object after a single entry into the content display page (i.e., the box). For example, if the content from the first to the fifth position is exposed to the sample interaction object, and then the sample interaction object exits the box, and the sixth position content is not exposed, then the length of the exposed content within the box is 5. Exemplarily, the content viewed by the sample interaction object can refer to the content viewed by the sample interaction object itself, or it can refer to the content viewed by the sample interaction object through the viewing entry point.

[0226] In the exemplary embodiment, the acquisition of diversity metrics and accuracy metrics includes, but is not limited to, the following three cases:

[0227] Case a: After the initial recommendation model outputs predicted recommendations, diversity and accuracy metrics are obtained directly based on the predicted recommendations.

[0228] In this case a, the embodiments of this application do not limit the order in which diversity metrics and accuracy metrics are obtained. For example, diversity metrics can be obtained first based on the predicted recommendation content, and then accuracy metrics can be obtained based on the predicted recommendation content; or accuracy metrics can be obtained first based on the predicted recommendation content, and then diversity metrics can be obtained based on the predicted recommendation content; or diversity metrics and accuracy metrics can be obtained simultaneously based on the predicted recommendation content.

[0229] In this scenario (a), the order in which diversity and accuracy metrics are obtained is relatively flexible and requires no additional computational logic. This helps reduce the computational load required to obtain diversity and accuracy metrics and improves the efficiency of obtaining them.

[0230] Case b: After the initial recommendation model outputs predicted recommendation content, an accuracy metric is obtained based on the predicted recommendation content; if the accuracy metric is greater than the first metric threshold, a diversity metric is obtained based on the predicted recommendation content.

[0231] The first indicator threshold can be set based on experience or adjusted flexibly according to the application scenario. This application does not limit this.

[0232] Obtaining a diversity metric when the accuracy metric is high (greater than the first metric threshold) ensures that the diversity of predicted recommendations is considered only after the accuracy of the predicted recommendations is high. Training the recommendation model using the accuracy and diversity metrics obtained under this condition (b) helps to improve the diversity of the recommended content while ensuring the accuracy of the recommended content, thus further improving the training effect of the recommendation model.

[0233] Case c: After the initial recommendation model outputs predicted recommendation content, a diversity metric is obtained based on the predicted recommendation content; if the diversity metric is greater than the second metric threshold, an accuracy metric is obtained based on the predicted recommendation content.

[0234] The threshold for the second indicator can be set based on experience or adjusted flexibly according to the application scenario. This application does not limit this.

[0235] Obtaining an accuracy metric when the diversity metric is high (greater than the second metric threshold) ensures that the accuracy of the predicted recommendations is considered only when the diversity of the predicted recommendations is high. Training the recommendation model using the accuracy and diversity metrics obtained under this condition (c) helps to improve the accuracy of the recommended recommendations while ensuring the diversity of the recommended content, thus further improving the training effect of the recommendation model.

[0236] In one possible implementation, for the case where the predicted recommended content and the prediction evaluation index of the predicted recommended content are obtained after the initial recommendation model is called to process the sample input information, the process of obtaining the training loss based on the diversity measure index and the accuracy measure index includes the following steps 2041 and 2042.

[0237] Step 2041: Based on diversity and accuracy metrics, determine the standard evaluation metrics for the predicted and recommended content. The standard evaluation metrics are the standard values ​​of the overall effect.

[0238] Standard evaluation metrics are used as supervisory information for the prediction evaluation metrics of the initial recommendation model output, and can be considered as a label that is relied upon when training the recommendation model.

[0239] In one possible implementation, the process of determining standard evaluation metrics for predictive recommendation content based on diversity and accuracy metrics includes steps A through C.

[0240] Step A: Based on diversity and accuracy metrics, determine the first evaluation metric, which is the standard value of the direct effect of recommending predicted content based on sample input information.

[0241] Since diversity and accuracy measures are obtained for the predicted and recommended content, they can reliably characterize the direct effect of recommending predicted and recommended content based on sample input information. Therefore, based on diversity and accuracy measures, a standard value (i.e., the first evaluation indicator) is determined for the direct effect of recommending predicted and recommended content based on sample input information to ensure the reliability of the first evaluation indicator.

[0242] In an exemplary embodiment, the first evaluation metric can be determined based on the diversity metric and the accuracy metric by taking the weighted sum of the diversity metric and the accuracy metric as the first evaluation metric. The weights of the diversity metric and the accuracy metric can be set based on experience or adjusted flexibly according to the application scenario; this embodiment does not limit this.

[0243] For example, the first evaluation index can be calculated based on Formula 3:

[0244] R=Relevance+α*Diversity (Formula 3)

[0245] Where R represents the first evaluation metric, Relevance represents the accuracy metric, Diversity represents the diversity metric, and α represents the weight of the diversity metric. α can be a hyperparameter that controls the business's preference for content diversity and the relative importance of the diversity metric to the accuracy metric when determining the first evaluation metric.

[0246] In an exemplary embodiment, the sample input information can be understood as a state (s) in reinforcement learning, the predicted recommendation content can be understood as an action (a) in reinforcement learning, and the first evaluation metric can be understood as the reward value (R) for taking action (a) in state (s). This reward value consists of two parts: a diversity metric and an accuracy metric. The diversity metric measures the diversity between the predicted recommendation content and the historical recommendation content, while the accuracy metric is determined by the behavioral feedback of the sample interaction object after the predicted recommendation content has been recommended.

[0247] Step B: Obtain updated input information of the sample interaction object; based on the updated input information, obtain the second evaluation index, which is the standard value of the indirect effect brought by the recommended content based on the sample input information.

[0248] Updating input information refers to the input information of the sample interaction object obtained after recommending and predicting content based on the sample input information. In other words, the updated input information and the sample input information are the input information of the sample interaction object obtained at different times. For example, the updated input information can be obtained by updating the sample input information or by completely re-obtaining it; this application embodiment does not limit this approach.

[0249] Taking the complete re-acquisition of updated input information as an example, since the principle of obtaining updated input information is the same as that of obtaining sample input information, after recommending and predicting recommended content based on sample input information, the input information of the sample interaction object can be obtained again according to the same principle as obtaining sample input information to obtain updated input information.

[0250] The input information of the sample interaction objects acquired at different times is related. Next, we will take the sample input information as the input information of the initially acquired sample interaction object, which is determined based on the first representation information, the second representation information, the third representation information, and the time representation information, as an example for explanation:

[0251] 1) The attribute information of the sample interaction object on which the first representation information depends remains unchanged. The statistical information of the sample interaction object on which the first representation information depends, within at least one time window (e.g., the total number of red dot clicks and the red dot exposure click-through rate in the past 6 hours, 1 day, and 7 days), is updated based on the real-time recommendation situation and the interaction between the sample interaction object and the real-time recommended content. In other words, the first representation information is updated at different times based on the real-time recommendation situation and the interaction between the sample interaction object and the real-time recommended content.

[0252] 2) If the sample interaction object generates a reference interaction behavior towards the recommended content (e.g., clicking the red dot corresponding to the content to enter the box), the latest time on which the time representation information is based is updated to the time when the reference interaction behavior occurred; if the sample interaction object does not generate a reference interaction behavior towards the recommended content (e.g., clicking the red dot corresponding to the content to enter the box), the latest time on which the time representation information is based remains unchanged. The generation time on which the time representation information is based is updated to the system real-time time. The time interval on which the time representation information is based is updated synchronously. That is to say, the time representation information will be updated in real time at different times.

[0253] 3) If new sample candidate content appears, the information on which the second representation information depends (e.g., the category of the sample candidate content, the feature vector of the sample candidate content, and the ranking of the sample candidate content) is updated according to the new sample candidate; if no new sample candidate appears, the second representation information remains unchanged. That is to say, the second representation information will be updated according to the new sample candidate content at different times.

[0254] 4) Feature vectors of historical recommended content (e.g., content recommended in the last 30 times) are maintained using a queue with a constant storage capacity. This means that as time progresses, the feature vectors of the oldest recommended content are continuously deleted, and the feature vectors of new recommended content are added. In other words, the third-party representation information is updated based on real-time recommendations at different times.

[0255] The second evaluation metric is used to evaluate the indirect effects of recommending and predicting content based on the sample input information. Since updating the input information is the input information of the sample interaction object obtained after recommending and predicting content based on the sample input information, and the second evaluation metric is obtained based on the updated input information, it can be understood that the second evaluation metric is used to evaluate the future effects (or future benefits, future returns, future rewards) of recommending and predicting content based on the sample input information.

[0256] In an exemplary embodiment, the process of obtaining the second evaluation index based on the updated input information can be as follows: calling the initial recommendation model to process the updated input information, obtaining the updated recommendation content and the predicted evaluation index corresponding to the updated recommendation content, and using the predicted evaluation index corresponding to the updated recommendation content as the second evaluation index.

[0257] In an exemplary embodiment, the method for obtaining the second evaluation metric based on the updated input information can also be as follows: call the reference recommendation model to process the updated input information, obtain the updated recommended content and the predicted evaluation metric corresponding to the updated recommended content, and use the predicted evaluation metric corresponding to the updated recommended content as the second evaluation metric. The reference recommendation model has the same structure as the initial recommendation model. The parameters of the reference recommendation model can be set empirically or determined based on the parameters of the initial recommendation model and its training status. For example, the initial values ​​of the parameters of the reference recommendation model are the same as the initial values ​​of the parameters of the initial recommendation model. After updating the parameters of the initial recommendation model a certain number of times, the current values ​​of the parameters of the initial recommendation model are synchronized to the parameters of the reference recommendation model.

[0258] Step C: Determine the standard evaluation indicators based on the first and second evaluation indicators.

[0259] For example, the standard evaluation index can be determined based on the first evaluation index and the second evaluation index by taking the weighted sum of the first evaluation index and the second evaluation index as the standard evaluation index. The weights of the first evaluation index and the second evaluation index can be set based on experience or flexibly adjusted according to the application scenario, and this application embodiment does not limit this.

[0260] For example, the standard evaluation index can be calculated based on the following formula 4:

[0261] Standard evaluation index = R + γ * Q(s) ′ ,a′)(Formula 4)

[0262] Where R represents the first evaluation index, Q(s) ′ ,a′) represents the second evaluation indicator; s ′ γ represents updating the input information; a′ represents updating the recommended content. For example, a′ can be the result output by the first initial sub-model based on the updated input information; γ represents the weight of the second evaluation metric, which can be understood as a discount factor used to measure the importance of future effects (future rewards).

[0263] The standard evaluation index is determined by comprehensively considering the direct and indirect effects of the recommended content based on the sample input information. The information considered in determining the standard evaluation index is relatively rich, which helps to improve the reliability of the determined standard evaluation index, thereby improving the reliability of the training loss obtained later and improving the training reliability of the recommendation model.

[0264] Step 2042: Obtain the training loss based on the prediction evaluation metric and the standard evaluation metric.

[0265] The method of obtaining training loss based on steps 2041 and 2042 fully considers the two aspects of information from the initial recommendation model output, which helps to improve the matching between training loss and recommendation model and improve the training effect of recommendation model.

[0266] In one possible implementation, an overall training loss is obtained based on the predicted evaluation metric and the standard evaluation metric. This overall training loss is used to update the parameters of the initial recommendation model in the direction of minimizing the difference between the predicted evaluation metric and the standard evaluation metric.

[0267] In one possible implementation, for the initial recommendation model comprising a first initial sub-model and a second initial sub-model, the training loss includes a first loss and a second loss. The process of obtaining the training loss based on the predicted evaluation metric and the standard evaluation metric includes: obtaining a first loss based on the predicted evaluation metric, which is used to update the parameters of the first initial sub-model in a direction that maximizes the predicted evaluation metric; and obtaining a second loss based on the predicted evaluation metric and the standard evaluation metric, which is used to update the parameters of the second initial sub-model in a direction that minimizes the difference between the predicted evaluation metric and the standard evaluation metric.

[0268] When obtaining the training loss, a first loss applicable to the first initial sub-model and a second loss applicable to the second initial sub-model are obtained respectively. The parameters of the first initial sub-model and the second initial sub-model are updated using the first loss and the second loss respectively, which enables the first initial sub-model and the second initial sub-model to optimize towards their respective optimization goals, improves the targeting of optimization, improves the convergence efficiency of model training, and improves the training efficiency of the recommendation model.

[0269] In some embodiments, the first loss can be represented as Actor Loss, and for example, the first loss can be calculated based on the following formula 5:

[0270]

[0271] Here, Actor Loss represents the first loss; (Predicted Recommended Content) represents the gradient of the predicted recommended content with respect to the parameters of the first initial sub-model.

[0272] In some embodiments, the gradient can be logarithmically represented to facilitate gradient calculation, and the first loss can then be calculated based on the following formula 6:

[0273]

[0274] Where E[] represents the expected value; Q(s,a) represents the logarithm of the gradient of the predicted recommended content 'a' with respect to the parameters of the first initial sub-model; Q(s,a) represents the prediction evaluation metric; s represents the sample input information; and a represents the predicted recommended content.

[0275] By minimizing the Actor Loss, the final recommended content is a = argmax{Q(s,a)}. That is, the first loss is used to update the parameters of the first initial sub-model in the direction of maximizing the prediction evaluation index (also known as the sum of long-term multi-step rewards).

[0276] In some embodiments, the second loss may represent the Critic Loss. Exemplarily, the second loss may be calculated based on the squared difference between the predicted evaluation metric and the standard evaluation metric. For example, the second loss may be calculated based on the following formula 7:

[0277] Critic Loss=E[(Q(s,a)-(R+γ*Q(s′,a′))) 2 ] (Formula 7)

[0278] Wherein, Critic Loss represents the second loss; E[] represents the expected value, used to calculate the average loss of a batch of sample input information; Q(s,a) represents the prediction evaluation index; R+γ*Q(s) ′ ,a′) represents the standard evaluation index; R represents the first evaluation index; Q(s) ′ ,a′) represents the second evaluation index; γ represents the discount factor, which also represents the weight of the second evaluation index.

[0279] By minimizing the Critic Loss, the second initial sub-model becomes increasingly accurate in predicting the evaluation metric. In other words, the second loss is used to update the parameters of the second initial sub-model in the direction of minimizing the difference between the predicted evaluation metric and the standard evaluation metric.

[0280] It should be noted that the above description only illustrates the process of obtaining training loss by taking the case of calling the initial recommendation model to process the sample input information to obtain the predicted recommendation content and the prediction evaluation index of the predicted recommendation content. The embodiments of this application are not limited to this.

[0281] In an exemplary embodiment, for the case where only predicted recommended content is obtained after the initial recommendation model processes the sample input information, the process of obtaining the training loss based on the diversity measure and the accuracy measure can be as follows: determine the first evaluation metric based on the diversity measure and the accuracy measure; determine the training loss based on the first evaluation metric, and the training loss is used to update the parameters of the initial recommendation model in the direction of maximizing the first evaluation metric.

[0282] In an exemplary embodiment, for the case where only predicted recommended content is obtained after processing the sample input information using the initial recommendation model, the process of obtaining the training loss based on the diversity and accuracy metrics can be as follows: First, determine a first evaluation metric based on the diversity and accuracy metrics; obtain updated input information of the sample interaction objects; obtain a second evaluation metric based on the updated input information; determine a standard evaluation metric based on the first and second evaluation metric; and obtain the training loss based on the standard evaluation metric. The training loss is used to update the parameters of the initial recommendation model in a direction that maximizes the standard evaluation metric.

[0283] The above description only illustrates the acquisition of training loss based on diversity and accuracy metrics, and the embodiments of this application are not limited thereto. In some embodiments, training loss can also be acquired solely based on diversity metrics. The embodiments of this application do not limit the method of acquiring training loss solely based on diversity metrics, as long as the acquired training loss ensures that the parameters of the initial recommendation model are updated in the direction that maximizes the diversity metrics. For example, the negative value of the diversity metrics can be used as the training loss. When acquiring training loss solely based on diversity metrics, only the diversity metrics need to be acquired to obtain the training loss, eliminating the need for other metrics, thereby reducing the computational load required to acquire the training loss, improving the efficiency of acquiring the training loss, and ultimately improving the training efficiency of the recommendation model.

[0284] In step 205, the parameters of the initial recommendation model are updated using the training loss to obtain the target recommendation model, which is used to recommend target content based on the target input information.

[0285] In one possible implementation, for the case where only a single overall training loss is obtained, the overall parameters of the initial recommendation model are updated using that overall training loss.

[0286] In one possible implementation, for cases where the training loss includes a first loss and a second loss, the process of updating the parameters of the initial recommendation model using the training loss includes: updating the parameters of the first initial sub-model using the first loss, and updating the parameters of the second initial sub-model using the second loss.

[0287] In an exemplary embodiment, after updating the parameters of the initial recommendation model using the training loss, one iteration is completed to obtain the updated recommendation model. If the updated recommendation model satisfies the training termination condition, the updated recommendation model is used as the target recommendation model.

[0288] If the updated recommendation model does not meet the training termination condition, new input information of the sample interaction object is obtained. Based on the new input information and the updated recommendation model, a new training loss is obtained. The parameters of the updated recommendation model are updated using the new training loss to obtain the updated recommendation model again. This process is repeated until a recommendation model that meets the training termination condition is obtained. The recommendation model that meets the training termination condition is taken as the target recommendation model.

[0289] For example, the training termination conditions can be set based on experience or flexibly adjusted according to the application scenario, and this application embodiment does not limit them. For example, the training termination conditions may include, but are not limited to: the number of times the model parameters are updated reaches a threshold; the training loss converges; the training loss is less than a loss threshold.

[0290] In some embodiments, when the training loss is determined based on diversity and accuracy metrics, and the acquisition of diversity and accuracy metrics is as described in step 204 (case b), the process of obtaining the target recommendation model when the accuracy metric is not greater than the first metric threshold can be as follows: First reference loss is obtained based solely on the accuracy metric; the parameters of the initial recommendation model are updated using the first reference loss until the accuracy metric corresponding to the predicted recommendation content output by the recommendation model is greater than the first metric threshold, thus obtaining the first pre-trained recommendation model; second reference loss is obtained based on the accuracy and diversity metrics corresponding to the predicted recommendation content output by the first pre-trained recommendation model; the parameters of the first pre-trained recommendation model are updated using the second reference loss, thus obtaining the target recommendation model.

[0291] For example, the process of updating the parameters of the initial recommendation model using the first pre-training loss can be called the process of pre-training the initial recommendation model. By pre-training the initial recommendation model, it can be ensured that the content recommended by the initial recommendation model has high accuracy.

[0292] This application does not limit the method of obtaining the first reference loss based solely on the accuracy metric, as long as the obtained first reference loss ensures that the parameters of the initial recommendation model are updated in the direction of maximizing the accuracy metric. For example, the negative value of the accuracy metric can be used as the first reference loss. The principle of obtaining the second reference loss based on the accuracy metric and diversity metric corresponding to the predicted recommendation content output by the first pre-trained recommendation model is the same as the principle of obtaining the training loss based on the accuracy metric and diversity metric corresponding to the predicted recommendation content output by the initial recommendation model, as described in step 204, and will not be repeated here.

[0293] In some embodiments, when the training loss is determined based on diversity and accuracy metrics, and the acquisition of diversity and accuracy metrics is as described in step 204 (case c), the process of obtaining the target recommendation model when the diversity metric is not greater than the second metric threshold can be as follows: A third reference loss is obtained based solely on the diversity metric; the parameters of the initial recommendation model are updated using the third reference loss until the diversity metric corresponding to the predicted recommendation content output by the recommendation model is greater than the second metric threshold, thus obtaining the second pre-trained recommendation model; a fourth reference loss is obtained based on the accuracy and diversity metrics corresponding to the predicted recommendation model output by the second pre-trained recommendation model; the parameters of the second pre-trained recommendation model are updated using the fourth reference loss, thus obtaining the target recommendation model.

[0294] For example, the process of updating the parameters of the initial recommendation model using the third reference loss can be called the process of pre-training the initial recommendation model. By pre-training the initial recommendation model, it can be ensured that the content recommended by the initial recommendation model has a high degree of diversity.

[0295] This application does not limit the method of obtaining the third reference loss based solely on the diversity metric, as long as the obtained third reference loss ensures that the parameters of the initial recommendation model are updated in the direction of maximizing the diversity metric. For example, the negative value of the diversity metric can be used as the third reference loss. The principle of obtaining the fourth reference loss based on the accuracy metric and diversity metric corresponding to the predicted recommendation model output by the second pre-trained recommendation model is the same as the principle of obtaining the training loss based on the accuracy metric and diversity metric corresponding to the predicted recommendation content output by the initial recommendation model, as described in step 204, and will not be repeated here.

[0296] The technical solution provided in this application update the parameters of the initial recommendation model using training loss obtained based on diversity metrics. This enables the recommendation model training process to focus on the diversity of predicted recommended content, improves the training effect of the recommendation model, increases the diversity of content recommended by the recommendation model, and thus increases the viewing rate of the recommended content.

[0297] For example, the training process of a recommendation model can be as follows: Figure 4 As shown. Multiple sample input information s are obtained. For example, the number of sample input information is BatchSize, and the dimension of each sample input information is 557. The sample input information s is input into the first initial sub-model to obtain the predicted recommendation content a. The sample input information s and the predicted recommendation content a are input into the second initial sub-model to obtain the prediction evaluation index Q(s,a).

[0298] Calculate a diversity metric based on the predicted recommended content a and the historical recommended content; calculate an accuracy metric based on the interaction between the sample interaction objects and the predicted recommended content a; and obtain the first evaluation metric R based on the diversity metric and the accuracy metric.

[0299] Based on the predicted recommended content a and the top 30 candidate content, obtain the updated input information s'; based on the updated input information s', obtain the second evaluation index Q(s',a'), where a' refers to the content recommended based on the updated input information.

[0300] Based on the prediction and evaluation metric Q(s,a), the first loss is obtained. Based on the first evaluation index R, the second evaluation index Q(s',a'), and the predicted evaluation index Q(s,a), the second loss is obtained.

[0301] The parameters of the first initial sub-model are updated using the first loss, and the parameters of the second initial sub-model are updated using the second loss, until the training termination condition is met, thus obtaining a target recommendation model that includes the first target sub-model and the second target sub-model.

[0302] The training method for the recommendation model provided in this application is a training method based on a reinforcement learning algorithm. The recommendation model to be trained is an agent in the reinforcement learning algorithm, the input information of the recommendation model is the state in the reinforcement learning algorithm, the recommended content output by the recommendation model is the action in the reinforcement learning algorithm, and the first evaluation index corresponding to the recommended content output by the recommendation model is the Q function value in the reinforcement learning algorithm.

[0303] Content recommendation is a multi-step iterative process. After a new action occurs, the state changes, the reward is evaluated and accumulated, and a new action is generated based on the new state. In this process, let the current step number be t. The states St and St+1 before and after an action, the action itself at, and the reward Rt for the current action are combined into a quadruple (St, at, Rt, St+1). This quadruple can serve as the basic data unit for reinforcement learning algorithms.

[0304] After the action *at* is generated, new recommended content is selected, and the state *St* changes to state *St+1*. After the action *at* is generated, the diversity of the new recommended content compared to historical recommended content can be immediately calculated, yielding *Diversity(t)*. Once the content recommendation is complete, the system collects feedback from the sample interaction objects, yielding *Relevance(t)*, and the corresponding reward *Rt* = *Relevance(t)* + α*Diversity(t). For ease of expression, when the number of iterations is not emphasized, the quadruple (*St*, *at*, *Rt*, *St+1*) can also be written as (*s*, *a*, *r*, *s'*).

[0305] During the training of the recommendation model, given a batch of user state data (equal to BatchSize) s, ActorNet (the first initial sub-model) is first fed in to generate action a. Then, CriticNet (the second initial sub-model) receives s and a, and outputs the Q-function prediction value (prediction evaluation metric). After action a occurs, predicted recommendation content is selected from the refined candidate set (sample candidate content) and pushed to the user. The state s is updated to s', and a diversity metric is calculated. The system waits for user feedback, calculates an accuracy metric, and finally combines these to obtain a reward value R. A standard evaluation metric is then calculated based on the reward value R.

[0306] The losses for the ActorNet and CriticNet neural network models are calculated separately based on the prediction evaluation metric and the standard evaluation metric. Based on the obtained losses, the parameters of the recommendation model are updated using any reinforcement learning training algorithm. For example, the reinforcement learning training algorithm may include, but is not limited to, DQN (Deep Q-Learning), DDPG (Deep Deterministic Policy Gradient), and TD3 (Twin Delayed Deep Deterministic Policy Gradient).

[0307] The above describes the training and inference process of using a reinforcement learning algorithm for one iteration (i.e., one-step decision, selecting and recommending one piece of content). Because the approach is based on real-time feedback and online learning, this process iterates continuously. After multiple decision steps, the diversity and accuracy of the recommended content will be improved, and the model parameters will tend to converge.

[0308] The method provided in this application can be applied to red dot recommendation for "subscription account messages" information flow products. This method aims to comprehensively optimize the click-through rate and diversity of red dots over a long period, and uses reinforcement learning techniques for modeling and algorithm implementation. It can achieve the following effects;

[0309] 1) Perform diversity modeling on content recommended at different times, and perform multi-objective optimization in conjunction with click-through rate (i.e. accuracy) to balance the diversity and accuracy of content delivery.

[0310] 2) A Markov decision process model for matching scenarios and a reinforcement learning quadruple are defined to use reinforcement learning methods to select the optimal content as the recommended content, thereby optimizing the long-term effect.

[0311] Among these, reinforcement learning techniques can be used to conveniently model the following key points:

[0312] 1) Enhanced diversity: Design a reward function that can comprehensively optimize diversity and click-through rate.

[0313] 2) Context awareness: By leveraging users' historical recommended content and statistical interaction behavior to form a state, more relevant and diverse content is recommended.

[0314] 3) User feedback loop: Based on user interaction with recommended content, update status and rewards in real time to form a feedback loop that enhances the push effect.

[0315] 4) Long-term effect optimization: The reinforcement learning process takes the reward function accumulated from multi-step action decisions as the ultimate optimization goal, and pays more attention to long-term effects.

[0316] This application provides a content recommendation method, which can be applied to the above-mentioned... Figure 1 The computer system shown. Taking the application of this method to a computer device as an example, such as... Figure 5 As shown, the content recommendation method provided in this application embodiment may include the following steps 501 to 503.

[0317] In step 501, the target recommendation model and the target input information of the target interaction object are obtained.

[0318] Among them, the target recommendation model utilizes Figure 2 The training method for any of the recommendation models in the illustrated embodiments is used for training.

[0319] In an exemplary embodiment, the method for obtaining target input information of a target interactive object includes: obtaining fourth characterization information and fifth characterization information, wherein the fourth characterization information is used to characterize the target interactive object and the fifth characterization information is used to characterize the target candidate content of the target interactive object; and obtaining target input information based on the fourth characterization information and the fifth characterization information.

[0320] In an exemplary embodiment, the process of obtaining the fourth characterization information includes: obtaining attribute information of the target interactive object and statistical information of the target interactive object under at least one time window; wherein, the statistical information of the target interactive object under any time window is determined based on the interaction between the target interactive object and the historical recommended content under any time window; and obtaining the fourth characterization information based on the attribute information of the target interactive object and the statistical information of the target interactive object under at least one time window.

[0321] In an exemplary embodiment, the number of target candidate contents is at least two, and the method for obtaining the fifth characterization information includes: for any target candidate content among the at least two target candidate contents, obtaining characterization sub-information of any target candidate content based on at least one of the category of any target candidate content or the ranking position of any target candidate content among the at least two target candidate contents, and the feature vector of any target candidate content; and obtaining the fifth characterization information based on the characterization sub-information of the at least two target candidate contents.

[0322] In an exemplary embodiment, obtaining target input information based on fourth and fifth characterization information includes: obtaining at least one of sixth characterization information or time characterization information; wherein the sixth characterization information is used to characterize the historical recommended content of the target interactive object; the target time characterization information is determined based on the generation time of the target input information and the latest time when the target interactive object generates a reference interaction behavior in response to the historical recommended content; and obtaining target input information based on at least one of the sixth characterization information or target time characterization information, the fourth characterization information, and the fifth characterization information.

[0323] In an exemplary embodiment, the target candidate content is the content obtained after performing at least one round of filtering operations on the original content to be recommended of the target interactive object.

[0324] The principle of obtaining target input information and Figure 2 The principle of obtaining sample input information in the illustrated embodiments is the same, and can be found in [reference needed]. Figure 2 The relevant content in step 201 of the illustrated embodiment will not be repeated here.

[0325] In step 502, the target recommendation model is invoked to process the target input information and obtain the target recommendation content.

[0326] The implementation principle of step 502 is... Figure 2 The implementation principle of step 202 in the illustrated embodiment is the same, and will not be repeated here.

[0327] It should be noted that, for the case where the initial recommendation model includes a first initial sub-model and a second initial sub-model, and the target recommendation model includes a first target sub-model and a second target sub-model, in this case, calling the target recommendation model to process the target input information and obtain the target recommendation content means calling the first target sub-model to process the target input information and obtain the target recommendation content.

[0328] In step 503, target recommended content is recommended to the target interactive object.

[0329] For example, the target recommended content is the content in the target candidate content of the target interaction object.

[0330] In an exemplary embodiment, recommending target content to a target interactive object may mean directly displaying the target content on the target interactive object's screen.

[0331] In an exemplary embodiment, recommending target content to a target interactive object may refer to displaying a viewing entry point for the target recommended content on the target interactive object's screen. In this case, the target recommended content is displayed in response to a triggering operation of the viewing entry point.

[0332] In an exemplary embodiment, recommending target content to a target interactive object can also refer to displaying a prompt message indicating the existence of the target recommended content on the target interactive object's screen. The prompt message contains less information than the target recommended content itself. For example, the prompt message may be the title of the target recommended content, a summary of the target recommended content, etc. In an exemplary embodiment, a prompt marker may also be displayed along with the prompt message. The form of the prompt marker can be set based on experience or flexibly adjusted according to the application scenario; for example, the prompt marker may be a red dot.

[0333] When displaying a prompt message indicating the target recommended content on the screen of the target interactive object, the content display page is shown in response to the prompt message's trigger action. The content display page then shows the entry point to view the target recommended content, and in response to the trigger action of this entry point, the target recommended content is displayed. It should be noted that, in addition to displaying the entry point to the target recommended content, the content display page can also display entry points to other content, and the target interactive object can also view other content by triggering these entry points.

[0334] In one possible implementation, the target candidate content is the content posted by informational accounts followed by the target user within the social application. In other words, the target recommended content is the content recommended to the target user from among the posts published by the informational accounts they follow. In this case, the process of recommending the target content to the target user includes: displaying a prompt message about the target recommended content at the informational account's entry point, where the informational account's entry point is the entry point on the social application's display page used to view the content posted by that account.

[0335] The display position of the information delivery account entry on the display page of the social application can be a fixed position set by the social application, a position set by the target interaction object, or a position that is adjusted in real time according to the social situation of the target interaction object, etc. This application embodiment does not limit this.

[0336] By displaying a prompt message indicating the target recommended content at the entry point of information delivery accounts, the target audience can intuitively understand that the social application has recommended their content. This encourages them to view the recommended content by triggering the prompt, thus increasing the view rate. Furthermore, compared to directly displaying the recommended content, displaying a prompt message consumes fewer resources, saving display resources that would otherwise be wasted if the target audience does not view the recommended content.

[0337] In an exemplary embodiment, after displaying the prompt message for the target recommended content at the entry point of the information delivery account, the method further includes: in response to a triggering operation of the information delivery account entry, displaying a content display page, which displays an entry point for viewing the target recommended content; and in response to the entry point for viewing the target recommended content, displaying the target recommended content. For example, the triggering operation of the information delivery account entry can refer to a triggering operation at any location within the information delivery account entry. For example, since the prompt message for the target recommended content is displayed at the information delivery account entry, if the triggering operation of the information delivery account entry is obtained, it can be considered that the target interactive object has triggered the prompt message for the target recommended content.

[0338] In one possible implementation, the method further includes displaying a notification marker at the entry point of an information delivery account, the notification marker indicating that there is content to be viewed.

[0339] The form of the notification marker can be set based on experience or flexibly adjusted according to the application scenario. For example, the notification marker can be a red dot. For instance, when a red dot is displayed at the entry point of an information dissemination account, the triggering operation for the notification message can also be understood as a click operation on the red dot.

[0340] When target recommended content exists, in addition to displaying a prompt message at the entry point of information dissemination accounts, a prompt marker is also displayed. This allows the target audience to be more intuitively informed that the social application has recommended target content, improving the visual effect and thus increasing the human-computer interaction rate.

[0341] In an exemplary embodiment, the entry point for information delivery accounts may also display the number of recommended content items that have been recommended to the target interaction object but have not been viewed by the target interaction object.

[0342] For example, the interface of a social application targeting an interactive object can be like... Figure 6 As shown, the information delivery account entry 601 refers to the entry corresponding to the title "Subscription Account Message". The information delivery account entry 601 displays a prompt message 602 with the target recommended content "xx News: There will be a severe rainstorm today", and also displays a prompt mark 603.

[0343] The content recommendation method provided in this application can be applied to red dot recommendation scenarios. Red dot recommendation is a common strategy in recommendation systems, mainly used to remind users of new information or content updates. The purpose of red dot recommendation is to attract users' attention and encourage them to click or view the recommended content, thereby increasing user activity and engagement. Red dot recommendation is usually implemented by displaying a red dot at a certain location on the application interface or website. For example, a red dot can be displayed at the entrance of an information flow product to indicate that information content has been updated. Red dot recommendation typically appears in places easily visible to users, such as application icons, menu items, and list items. When new information or content is updated, a red dot or a red dot with a number will be displayed. After the user sees the red dot, if they click on the red dot to view the new content, the red dot disappears.

[0344] The Red Dot recommendation system has several potential problems that need to be addressed. Long-term experiments have shown that while solely focusing on optimizing Red Dot click-through rates can maximize overall consumer metrics in the short term, the growth in users entering the box and making clicks tends to decline or even disappear over the long term. Analysis suggests that over-optimizing the Red Dot recommendation system and relying too heavily on historical user behavior data will lead to increasingly narrow and undiverse recommended content. This ignores users' demand for novel and diverse content, resulting in a decline in user experience and even churn. If the Red Dot recommended content is mismatched or focuses on a limited number of items for an extended period, it may cause user dissatisfaction and reduce user satisfaction.

[0345] The method provided in this application provides a new red dot recommendation system, which can improve the red dot entry rate and click volume of text and video content, and also increase the proportion of users who consume the box.

[0346] In this embodiment, the target recommendation model is invoked to obtain the target recommendation content to the target interaction object. The target recommendation model is trained by considering diversity metrics, which enables the target recommendation model to recommend target content by considering diversity, thereby improving the recommendation diversity of target content and increasing the likelihood that the target recommendation content will be viewed by the target interaction object, resulting in better content recommendation performance.

[0347] For example, the process by which a computer device performs a content recommendation method can be as follows: Figure 7 As shown, the process includes steps 701 to 714.

[0348] Step 701: Obtain first representation information, second representation information, third representation information and time representation information; wherein, the first representation information is used to represent the sample interaction object, the second representation information is used to represent the sample candidate content of the sample interaction object, and the third representation information is used to represent the historical recommendation content of the sample interaction object.

[0349] For example, the first characterization information is obtained based on the attribute information of the sample interaction object and the statistical information of the sample interaction object in at least one time window; the statistical information of the sample interaction object in any time window is determined based on the interaction between the sample interaction object and the historical recommended content in any time window.

[0350] For example, the number of sample candidate contents is at least two, the second representation information is obtained based on the representation sub-information of at least two sample candidate contents, and the representation sub-information of any sample candidate content is obtained based on at least one of the category of any sample candidate content or the ranking position of any sample candidate content among at least two sample candidate contents, as well as the feature vector of any sample candidate content.

[0351] For example, the sample candidate content is the content obtained after performing at least one round of filtering operations on the original content to be recommended.

[0352] For example, the time representation information is determined based on the generation time of the sample input information and the latest time when the sample interaction object generates reference interaction behavior in response to historical recommended content.

[0353] Step 702: Based on the first representation information, the second representation information, the third representation information, and the time representation information, obtain the sample input information of the sample interaction object.

[0354] Step 703: Call the first initial sub-model to process the sample input information and obtain the predicted recommendation content.

[0355] Step 704: Call the second initial sub-model to process the predicted recommendation content and sample input information to obtain the prediction evaluation index of the predicted recommendation content.

[0356] Step 705: Based on the predicted recommended content and the historical recommended content of the sample interaction objects, obtain the similarity matrix; obtain the determinant calculation result of the similarity matrix; based on the determinant calculation result, obtain the diversity measurement index of the predicted recommended content.

[0357] Step 706: Obtain the interaction information of the predicted recommended content; use the metrics corresponding to the interaction status indicated by the interaction information as the accuracy metrics.

[0358] Step 707: Determine the first evaluation indicator based on the diversity and accuracy metrics; obtain the updated input information of the sample interaction objects; obtain the second evaluation indicator based on the updated input information; determine the standard evaluation indicator based on the first and second evaluation indicators.

[0359] Step 708: Obtain the first loss based on the predictive evaluation indicators; obtain the second loss based on the predictive evaluation indicators and the standard evaluation indicators.

[0360] Step 709: Update the parameters of the first initial sub-model using the first loss, and update the parameters of the second initial sub-model using the second loss, to obtain a target recommendation model that includes the first target sub-model and the second target sub-model.

[0361] Step 710: Obtain the fourth representation information, the fifth representation information, the sixth representation information, and the target time representation information; wherein, the fourth representation information is used to represent the target interactive object, the fifth representation information is used to represent the target candidate content of the target interactive object, and the sixth representation information is used to represent the historical recommended content of the target interactive object.

[0362] Step 711: Based on the fourth representation information, the fifth representation information, the sixth representation information, and the target time representation information, obtain the target input information of the target interactive object.

[0363] Step 712: Call the first target sub-model in the target recommendation model to process the target input information and obtain the target recommendation content.

[0364] Step 713: Display prompts and markers for the target recommended content at the entry point of information dissemination accounts.

[0365] Step 714: In response to the triggering action of the information dissemination account entry, display the content display page, which shows the entry point for viewing the target recommended content; in response to the triggering action of the entry point for viewing the target recommended content, display the target recommended content.

[0366] See Figure 8 This application provides a training apparatus for a recommendation model, the apparatus comprising:

[0367] The first acquisition module 801 is used to acquire sample input information of the sample interaction object;

[0368] Processing module 802 is used to call the initial recommendation model to process the sample input information and obtain the predicted recommendation content;

[0369] The second acquisition module 803 is used to acquire the diversity measurement index of the predicted recommendation content. The diversity measurement index is used to measure the recommendation diversity of the predicted recommendation content.

[0370] The third acquisition module 804 is used to acquire the training loss based on diversity metrics;

[0371] The update module 805 is used to update the parameters of the initial recommendation model using the training loss to obtain the target recommendation model. The target recommendation model is used to recommend target content based on the target input information.

[0372] In one possible implementation, the second acquisition module 803 is used to acquire the training loss based on a diversity metric and an accuracy metric, wherein the accuracy metric is used to measure the accuracy of the predicted recommended content.

[0373] In one possible implementation, the processing module 802 is used to call the initial recommendation model to process the sample input information, and obtain the predicted recommendation content and the prediction evaluation index of the predicted recommendation content. The prediction evaluation index is the predicted value of the comprehensive effect brought by recommending the predicted recommendation content based on the sample input information.

[0374] The third acquisition module 804 is used to determine the standard evaluation index of the predicted and recommended content based on the diversity measurement index and the accuracy measurement index. The standard evaluation index is the standard value of the comprehensive effect; and to obtain the training loss based on the prediction evaluation index and the standard evaluation index.

[0375] In one possible implementation, the third acquisition module 804 is used to determine a first evaluation index based on diversity and accuracy metrics, wherein the first evaluation index is a standard value of the direct effect of recommending and predicting content based on sample input information; acquire updated input information of sample interaction objects; acquire a second evaluation index based on the updated input information, wherein the second evaluation index is a standard value of the indirect effect of recommending and predicting content based on sample input information; and determine a standard evaluation index based on the first and second evaluation indicators.

[0376] In one possible implementation, the initial recommendation model includes a first initial sub-model and a second initial sub-model; the processing module 802 is used to call the first initial sub-model to process the sample input information to obtain the predicted recommendation content; and to call the second initial sub-model to process the predicted recommendation content and the sample input information to obtain the prediction evaluation index of the predicted recommendation content.

[0377] In one possible implementation, the training loss includes a first loss and a second loss; a third acquisition module 804 is used to acquire the first loss based on the predicted evaluation metric, the first loss being used to update the parameters of the first initial sub-model in the direction of maximizing the predicted evaluation metric; and to acquire the second loss based on the predicted evaluation metric and the standard evaluation metric, the second loss being used to update the parameters of the second initial sub-model in the direction of minimizing the difference between the predicted evaluation metric and the standard evaluation metric.

[0378] The update module 805 is used to update the parameters of the first initial sub-model using the first loss and to update the parameters of the second initial sub-model using the second loss.

[0379] In one possible implementation, the second acquisition module 803 is used to acquire a similarity matrix based on the predicted recommended content and the historical recommended content of the sample interaction object. The element in the i-th row and j-th column of the similarity matrix is ​​used to indicate the similarity between the i-th content and the j-th content in the content set consisting of the predicted recommended content and the historical recommended content, where i and j are integers not greater than n, and n is the number of content in the content set. The determinant calculation result of the similarity matrix is ​​acquired, and the determinant calculation result is used to measure the degree of difference between the content in the content set. Based on the determinant calculation result, a diversity measure index of the predicted recommended content is acquired.

[0380] In one possible implementation, the second acquisition module 803 is used to acquire interaction information of the predicted recommendation content, the interaction information being used to indicate the interaction between the sample interaction object and the predicted recommendation content; based on the interaction information, an accuracy measurement index is acquired.

[0381] In one possible implementation, the second acquisition module 803 is used to use the measurement index corresponding to the interaction situation indicated by the interaction information as an accuracy measurement index; wherein, the interaction situation indicated by the interaction information includes a first interaction situation, a second interaction situation, a third interaction situation, or a fourth interaction situation, the measurement index corresponding to the first interaction situation is less than the measurement index corresponding to the second interaction situation, the measurement index corresponding to the second interaction situation is less than the measurement index corresponding to the third interaction situation, and the measurement index corresponding to the third interaction situation is less than the measurement index corresponding to the fourth interaction situation; the first interaction situation is that the sample interaction object does not trigger the prompt information after the prompt information of the predicted recommendation content is displayed to the sample interaction object; the second... The second interaction scenario involves the sample interacting object triggering a prompt message about predicted recommended content after being shown the message, and then entering the content display page without viewing any content. The third interaction scenario involves the sample interacting object triggering a prompt message about predicted recommended content after being shown the message, and then viewing content other than the predicted recommended content on the content display page. The fourth interaction scenario involves the sample interacting object triggering a prompt message about predicted recommended content after being shown the message about predicted recommended content, and then viewing the predicted recommended content on the content display page.

[0382] In one possible implementation, the first acquisition module 801 is used to acquire first representation information and second representation information, the first representation information being used to represent the sample interaction object, and the second representation information being used to represent the sample candidate content of the sample interaction object; and to acquire sample input information based on the first representation information and the second representation information.

[0383] In one possible implementation, the first acquisition module 801 is used to acquire attribute information of the sample interaction object and statistical information of the sample interaction object under at least one time window; wherein, the statistical information of the sample interaction object under any time window is determined based on the interaction between the sample interaction object and the historical recommended content under any time window; and based on the attribute information of the sample interaction object and the statistical information of the sample interaction object under at least one time window, first characterization information is acquired.

[0384] In one possible implementation, the number of sample candidate contents is at least two. The first acquisition module 801 is used to acquire the representation sub-information of any sample candidate content based on at least one of the category of any sample candidate content or the ranking position of any sample candidate content in the at least two sample candidate contents, and the feature vector of any sample candidate content; and acquire the second representation information based on the representation sub-information of the at least two sample candidate contents.

[0385] In one possible implementation, the first acquisition module 801 is used to acquire at least one of third characterization information or time characterization information; wherein, the third characterization information is used to characterize the historical recommended content of the sample interaction object; the time characterization information is determined based on the generation time of the sample input information and the latest time when the sample interaction object generates reference interaction behavior in response to the historical recommended content; and the sample input information is acquired based on at least one of the third characterization information or time characterization information, the first characterization information and the second characterization information.

[0386] In one possible implementation, the sample candidate content is the content obtained after performing at least one round of filtering operations on the original content to be recommended.

[0387] See Figure 9 This application provides a content recommendation device, which includes:

[0388] Module 901 is used to acquire the target input information of the target recommendation model and the target interaction object. The target recommendation model utilizes... Figure 2 The recommendation model in any of the embodiments shown is trained using the training method described above.

[0389] Processing module 902 is used to call the target recommendation model to process the target input information and obtain the target recommendation content;

[0390] Recommendation module 903 is used to recommend target content to the target interaction object.

[0391] In one possible implementation, the target recommended content is the content in the target candidate content of the target interactive object, and the target candidate content is the content published by the information dissemination account that the target interactive object follows in the social application; the recommendation module 903 is used to display the prompt information of the target recommended content at the entry point of the information dissemination account, and the entry point of the information dissemination account is the entry point in the display page of the social application used to view the content published by the information dissemination account.

[0392] In one possible implementation, the recommendation module 903 is also used to display a prompt marker at the entry point of information dissemination accounts, the prompt marker being used to indicate that there is content to be viewed.

[0393] It should be noted that the apparatus provided in the above embodiments is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process and beneficial effects are detailed in the method embodiments, which will not be repeated here.

[0394] In an exemplary embodiment, a computer device is also provided, comprising a processor and a memory storing at least one computer program. The at least one computer program is loaded and executed by one or more processors to enable the computer device to implement any of the above-described recommendation model training methods or content recommendation methods. The computer device can be a server or a terminal. The structures of the server and the terminal will be described below.

[0395] Figure 10 This is a schematic diagram of a server structure provided in an embodiment of this application. The server can vary significantly due to differences in configuration or performance. It may include one or more Central Processing Units (CPUs) 1001 and one or more memories 1002. The one or more memories 1002 store at least one computer program, which is loaded and executed by the one or more processors 1001 to enable the server to implement the training method or content recommendation method of the recommendation model provided in the various method embodiments described above. Of course, the server may also have wired or wireless network interfaces, a keyboard, and input / output interfaces for input and output. The server may also include other components for implementing device functions, which will not be elaborated upon here.

[0396] Figure 11 This is a schematic diagram of the structure of a terminal provided in an embodiment of this application. The terminal can be: a PC, mobile phone, smartphone, PDA, wearable device, PPC, tablet computer, smart car infotainment system, smart TV, smart speaker, or in-vehicle terminal. The terminal may also be referred to as user equipment, portable terminal, laptop terminal, desktop terminal, or other names.

[0397] Typically, a terminal includes a processor 1101 and a memory 1102.

[0398] Processor 1101 may include one or more processing cores, such as a quad-core processor, an octa-core processor, etc. Processor 1101 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). Processor 1101 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 1101 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content required to be displayed on the screen. In some embodiments, processor 1101 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0399] The memory 1102 may include one or more computer-readable storage media, which may be non-transitory. The memory 1102 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 1102 is used to store at least one instruction, which is executed by the processor 1101 to enable the terminal to implement the training method or content recommendation method of the recommendation model provided in the method embodiments of this application.

[0400] In some embodiments, the terminal may also optionally include: a peripheral device interface 1103 and at least one peripheral device. The processor 1101, memory 1102, and peripheral device interface 1103 can be connected via a bus or signal line. Each peripheral device can be connected to the peripheral device interface 1103 via a bus, signal line, or circuit board. Specifically, the peripheral device includes at least one of: a radio frequency circuit 1104, a display screen 1105, a camera assembly 1106, an audio circuit 1107, and a power supply 1108.

[0401] Peripheral device interface 1103 can be used to connect at least one I / O (Input / Output) related peripheral device to processor 1101 and memory 1102. In some embodiments, processor 1101, memory 1102 and peripheral device interface 1103 are integrated on the same chip or circuit board; in some other embodiments, any one or two of processor 1101, memory 1102 and peripheral device interface 1103 can be implemented on separate chips or circuit boards, which is not limited in this embodiment.

[0402] The radio frequency (RF) circuit 1104 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The RF circuit 1104 communicates with communication networks and other communication devices via electromagnetic signals. The RF circuit 1104 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals back into electrical signals. Optionally, the RF circuit 1104 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, etc. The RF circuit 1104 can communicate with other terminals through at least one wireless communication protocol. This wireless communication protocol includes, but is not limited to: metropolitan area networks (MANs), various generations of mobile communication networks (2G, 3G, 4G, and 5G), wireless local area networks (WLANs), and / or WiFi (Wireless Fidelity) networks. In some embodiments, the RF circuit 1104 may also include circuitry related to NFC (Near Field Communication), which is not limited in this application.

[0403] Display screen 1105 is used to display a UI (User Interface). This UI may include graphics, text, icons, videos, and any combination thereof. When display screen 1105 is a touch display screen, it also has the ability to collect touch signals on or above its surface. These touch signals can be input as control signals to processor 1101 for processing. In this case, display screen 1105 can also be used to provide virtual buttons and / or a virtual keyboard, also known as soft buttons and / or a soft keyboard. In some embodiments, display screen 1105 can be a single screen, located on the front panel of the terminal; in other embodiments, display screen 1105 can be at least two screens, respectively located on different surfaces of the terminal or in a folded design; in other embodiments, display screen 1105 can be a flexible display screen, located on a curved or folded surface of the terminal. Furthermore, display screen 1105 can be configured as a non-rectangular, irregular shape, i.e., a non-rectangular screen. Display screen 1105 can be made of materials such as LCD (Liquid Crystal Display) or OLED (Organic Light-Emitting Diode).

[0404] The camera assembly 1106 is used to acquire images or videos. Optionally, the camera assembly 1106 includes a front-facing camera and a rear-facing camera. Typically, the front-facing camera is located on the front panel of the terminal, and the rear-facing camera is located on the back of the terminal. In some embodiments, there are at least two rear-facing cameras, which are any one of a main camera, a depth-sensing camera, a wide-angle camera, and a telephoto camera, to achieve background blurring by fusion of the main camera and the depth-sensing camera, panoramic shooting by fusion of the main camera and the wide-angle camera, VR (Virtual Reality) shooting, or other fusion shooting functions. In some embodiments, the camera assembly 1106 may also include a flash. The flash can be a single-color temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm-light flash and a cool-light flash, which can be used for light compensation at different color temperatures.

[0405] The audio circuit 1107 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, converting the sound waves into electrical signals that are input to the processor 1101 for processing, or input to the radio frequency circuit 1104 to achieve voice communication. For stereo sound acquisition or noise reduction purposes, multiple microphones may be used, each positioned at a different location on the terminal. The microphone may also be an array microphone or an omnidirectional microphone. The speaker is used to convert electrical signals from the processor 1101 or the radio frequency circuit 1104 into sound waves. The speaker may be a conventional diaphragm speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can convert electrical signals not only into audible sound waves but also into inaudible sound waves for purposes such as distance measurement. In some embodiments, the audio circuit 1107 may also include a headphone jack.

[0406] Power supply 1108 is used to power the various components in the terminal. Power supply 1108 can be AC ​​power, DC power, a disposable battery, or a rechargeable battery. When power supply 1108 includes a rechargeable battery, the rechargeable battery can support wired charging or wireless charging. The rechargeable battery can also be used to support fast charging technology.

[0407] In some embodiments, the terminal further includes one or more sensors 1109. The one or more sensors 1109 include, but are not limited to: an acceleration sensor 1110, a gyroscope sensor 1111, a pressure sensor 1112, an optical sensor 1113, and a proximity sensor 1114.

[0408] Accelerometer 1110 can detect the magnitude of acceleration along the three coordinate axes of a coordinate system established by the terminal. For example, accelerometer 1110 can be used to detect the components of gravitational acceleration along the three coordinate axes. Processor 1101 can control display screen 1105 to display the user interface in either a landscape or portrait view based on the gravitational acceleration signal acquired by accelerometer 1110. Accelerometer 1110 can also be used for collecting game or user motion data.

[0409] The gyroscope sensor 1111 can detect the terminal's orientation and rotation angle. The gyroscope sensor 1111 can work in conjunction with the accelerometer sensor 1110 to collect the user's 3D movements on the terminal. Based on the data collected by the gyroscope sensor 1111, the processor 1101 can perform the following functions: motion sensing (e.g., changing the UI based on the user's tilt), image stabilization during shooting, game control, and inertial navigation.

[0410] The pressure sensor 1112 can be disposed on the side bezel of the terminal and / or the lower layer of the display screen 1105. When the pressure sensor 1112 is disposed on the side bezel of the terminal, it can detect the user's grip signal on the terminal, and the processor 1101 can perform left / right hand recognition or quick operation based on the grip signal collected by the pressure sensor 1112. When the pressure sensor 1112 is disposed on the lower layer of the display screen 1105, the processor 1101 can control the operable controls on the UI interface based on the user's pressure operation on the display screen 1105. The operable controls include at least one of button controls, scroll bar controls, icon controls, and menu controls.

[0411] An optical sensor 1113 is used to collect ambient light intensity. In one embodiment, the processor 1101 can control the display brightness of the display screen 1105 based on the ambient light intensity collected by the optical sensor 1113. Specifically, when the ambient light intensity is high, the display brightness of the display screen 1105 is increased; when the ambient light intensity is low, the display brightness of the display screen 1105 is decreased. In another embodiment, the processor 1101 can also dynamically adjust the shooting parameters of the camera assembly 1106 based on the ambient light intensity collected by the optical sensor 1113.

[0412] The proximity sensor 1114, also known as a distance sensor, is typically installed on the front panel of the terminal. The proximity sensor 1114 is used to detect the distance between the user and the front of the terminal. In one embodiment, when the proximity sensor 1114 detects that the distance between the user and the front of the terminal is gradually decreasing, the processor 1101 controls the display screen 1105 to switch from a screen-on state to a screen-off state; when the proximity sensor 1114 detects that the distance between the user and the front of the terminal is gradually increasing, the processor 1101 controls the display screen 1105 to switch from a screen-off state to a screen-on state.

[0413] Those skilled in the art will understand that Figure 11 The structure shown does not constitute a limitation on the terminal and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0414] In an exemplary embodiment, a computer-readable storage medium is also provided, which stores at least one computer program, which is loaded and executed by a processor of a computer device to enable the computer to implement the training method or content recommendation method of any of the above-described recommendation models.

[0415] In one possible implementation, the aforementioned computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), magnetic tape, floppy disk, and optical data storage device, etc.

[0416] In an exemplary embodiment, a computer program or computer program product is also provided, which includes computer instructions loaded and executed by a processor to enable a computer to implement the training method or content recommendation method of any of the above-described recommendation models.

[0417] It should be noted that all information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.), and signals involved in this application have been authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the interaction between the sample interaction objects and the predicted recommendation content, as well as the attribute information of the sample interaction objects involved in this application, were obtained with full authorization.

[0418] The terms "first," "second," etc., used in this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. The implementations described in the above exemplary embodiments do not represent all implementations consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application.

[0419] It should be understood that "multiple" as used in this article refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0420] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. Optionally, the program is stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0421] The above description is merely an exemplary embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the principles of this application should be included within the protection scope of this application.

Claims

1. A method for training a recommendation model, characterized in that, The method comprises: obtaining sample input information of a sample interactive object; calling an initial recommendation model to process the sample input information to obtain predicted recommendation content; obtaining a diversity measurement index of the predicted recommendation content, the diversity measurement index being used to measure recommendation diversity of the predicted recommendation content; based on the diversity measurement index, obtaining a training loss; updating parameters of the initial recommendation model using the training loss to obtain a target recommendation model, the target recommendation model being used to recommend target recommendation content based on target input information.

2. The method of claim 1, wherein, The method comprises: based on the diversity measurement index and an accuracy measurement index, obtaining a training loss, the accuracy measurement index being used to measure recommendation accuracy of the predicted recommendation content.

3. The method of claim 2, wherein, The method comprises: calling the initial recommendation model to process the sample input information to obtain the predicted recommendation content and a predicted evaluation index of the predicted recommendation content, the predicted evaluation index being a predicted value of a comprehensive effect brought by recommending the predicted recommendation content based on the sample input information; The method comprises: based on the diversity measurement index and the accuracy measurement index, determining a standard evaluation index of the predicted recommendation content, the standard evaluation index being a standard value of the comprehensive effect; based on the predicted evaluation index and the standard evaluation index, obtaining the training loss.

4. The method of claim 3, wherein, The method comprises: based on the diversity measurement index and the accuracy measurement index, determining a first evaluation index, the first evaluation index being a standard value of a direct effect brought by recommending the predicted recommendation content based on the sample input information; obtaining updated input information of the sample interactive object; based on the updated input information, obtaining a second evaluation index, the second evaluation index being a standard value of an indirect effect brought by recommending the predicted recommendation content based on the sample input information; based on the first evaluation index and the second evaluation index, determining the standard evaluation index.

5. The method of claim 3, wherein, The initial recommendation model comprises a first initial sub-model and a second initial sub-model; the method comprises: calling the first initial sub-model to process the sample input information to obtain the predicted recommendation content; calling the second initial sub-model to process the predicted recommendation content and the sample input information to obtain the predicted evaluation index of the predicted recommendation content.

6. The method of claim 5, wherein, The training loss comprises a first loss and a second loss; the method comprises: Based on the prediction evaluation index, a first loss is obtained, the first loss being used to update parameters of the first initial sub-model in a direction of maximizing the prediction evaluation index; Based on the prediction evaluation index and the standard evaluation index, a second loss is obtained, the second loss being used to update parameters of the second initial sub-model in a direction of minimizing a difference between the prediction evaluation index and the standard evaluation index; The updating of the parameters of the initial recommendation model by using the training loss comprises: The parameters of the first initial sub-model are updated by using the first loss, and the parameters of the second initial sub-model are updated by using the second loss.

7. The method according to any of claims 1 to 6, characterized in that The obtaining of the diversity measurement index of the predicted recommendation content comprises: Based on the predicted recommendation content and historical recommendation content of the sample interaction object, a similarity matrix is obtained, an element at an i-th row and a j-th column in the similarity matrix being used to indicate a similarity between an i-th content and a j-th content in a content set constituted by the predicted recommendation content and the historical recommendation content, i and j being integers not greater than n, n being a quantity of contents in the content set; A determinant calculation result of the similarity matrix is obtained, the determinant calculation result being used to measure a difference degree between the contents in the content set; Based on the determinant calculation result, the diversity measurement index of the predicted recommendation content is obtained.

8. The method according to any of claims 2-6, characterized by, The obtaining method of the accuracy measurement index comprises: Interaction information of the predicted recommendation content is obtained, the interaction information being used to indicate an interaction between the sample interaction object and the predicted recommendation content; Based on the interaction information, the accuracy measurement index is obtained.

9. The method of claim 8, wherein, The obtaining of the accuracy measurement index based on the interaction information comprises: A measurement index corresponding to an interaction indicated by the interaction information is taken as the accuracy measurement index; The interaction indicated by the interaction information comprises a first interaction, a second interaction, a third interaction or a fourth interaction, a measurement index corresponding to the first interaction being less than a measurement index corresponding to the second interaction, the measurement index corresponding to the second interaction being less than a measurement index corresponding to the third interaction, the measurement index corresponding to the third interaction being less than a measurement index corresponding to the fourth interaction. The first interaction condition is that the sample interaction object does not trigger the prompt information after the sample interaction object is displayed with the prompt information of the predicted recommended content; the second interaction condition is that the sample interaction object triggers the prompt information to enter a content display page after the sample interaction object is displayed with the prompt information of the predicted recommended content, and the sample interaction object does not view any content in the content display page; the third interaction condition is that the sample interaction object triggers the prompt information to enter a content display page after the sample interaction object is displayed with the prompt information of the predicted recommended content, and the sample interaction object views content other than the predicted recommended content in the content display page; and the fourth interaction condition is that the sample interaction object triggers the prompt information to enter a content display page after the sample interaction object is displayed with the prompt information of the predicted recommended content, and the sample interaction object views the predicted recommended content in the content display page.

10. The method of any one of claims 1-6, wherein, The sample input information of the sample interaction object is obtained, including: The first representation information is obtained, and the second representation information is obtained, the first representation information being used for representing the sample interaction object, and the second representation information being used for representing sample candidate content of the sample interaction object; The sample input information is obtained based on the first representation information and the second representation information.

11. The method of claim 10, wherein, The first representation information is obtained, including: Attribute information of the sample interaction object and statistical information of the sample interaction object in at least one time window are obtained; wherein the statistical information of the sample interaction object in any time window is determined based on an interaction condition of the sample interaction object and historical recommended content in the any time window; The first representation information is obtained based on the attribute information of the sample interaction object and the statistical information of the sample interaction object in at least one time window.

12. The method of claim 10, wherein, The number of the sample candidate content is at least two, and the second representation information is obtained, including: For any sample candidate content of the at least two sample candidate contents, representation sub-information of the any sample candidate content is obtained based on at least one of a category of the any sample candidate content or a ranking position of the any sample candidate content in the at least two sample candidate contents, and a feature vector of the any sample candidate content; The second representation information is obtained based on the representation sub-information of the at least two sample candidate contents.

13. The method of claim 10, wherein, The sample input information is obtained based on the first representation information and the second representation information, including: At least one of third representation information or time representation information is obtained; wherein the third representation information is used for representing historical recommended content of the sample interaction object; and the time representation information is determined based on a generation time of the sample input information and a latest time at which the sample interaction object generates a reference interaction behavior for the historical recommended content; The sample input information is obtained based on at least one of the third representation information or the time representation information, the first representation information and the second representation information.

14. A content recommendation method characterized by, The method includes: The target recommendation model is trained by using the training method of the recommendation model in any one of claims 1-13. The target recommendation model is called to process the target input information to obtain target recommendation content. The target recommendation content is recommended to the target interactive object.

15. The method of claim 14, wherein, The target recommendation content is content in target candidate content of the target interactive object, and the target candidate content is content published by an information communication account followed by the target interactive object in a social application program. The target recommendation content is recommended to the target interactive object by: Displaying prompt information of the target recommendation content at an information communication account portal, and the information communication account portal is an entrance for viewing content published by an information communication account in a display page of the social application program.

16. The method of claim 15, wherein, The method further includes: Displaying a prompt mark at the information communication account portal, and the prompt mark is used to prompt that there is content to be viewed.

17. A training device for a recommendation model, characterized in that, The device includes: A first obtaining module is configured to obtain sample input information of a sample interactive object. A processing module is configured to call an initial recommendation model to process the sample input information to obtain predicted recommendation content. A second obtaining module is configured to obtain a diversity measurement index of the predicted recommendation content, and the diversity measurement index is used to measure recommendation diversity of the predicted recommendation content. A third obtaining module is configured to obtain a training loss based on the diversity measurement index. An updating module is configured to update parameters of the initial recommendation model by using the training loss to obtain a target recommendation model, and the target recommendation model is used to recommend target recommendation content based on target input information.

18. A content recommendation apparatus characterized by comprising: The device includes: An obtaining module is configured to obtain a target recommendation model and target input information of a target interactive object, and the target recommendation model is obtained by using the training method of the recommendation model in any one of claims 1-13. A processing module is configured to call the target recommendation model to process the target input information to obtain target recommendation content. A recommendation module is configured to recommend the target recommendation content to the target interactive object.

19. A computer device, comprising: The computer device includes a processor and a memory, and the memory stores at least one computer program, which is loaded and executed by the processor to enable the computer device to implement the training method of the recommendation model in any one of claims 1-13 or the content recommendation method in any one of claims 14-16.

20. A computer-readable storage medium, characterized in that, The computer readable storage medium stores at least one computer program, which is loaded and executed by the processor to enable the computer to implement the training method of the recommendation model in any one of claims 1-13 or the content recommendation method in any one of claims 14-16.

Images