Method for training resource recommendation model and resource recommendation method

Abstract

The present disclosure relates to a resource recommendation model training method and a resource recommendation method. The method comprises: obtaining a first sample data set and a second sample data set; the first sample data set contains basic user information and user behavior data of a first type of user, and the second sample data set contains basic user information of a second type of user; the first type of user has more user behavior data than the second type of user; using the basic user information of the first type of user as input, using the predicted user portrait and predicted resources of the first type of user as output, and using the user behavior data of the first type of user as supervision information, the pre-trained resource recommendation model is supervised fine-tuned to obtain a fine-tuned resource recommendation model; using the basic user information of the second type of user as input, using the predicted user portrait and predicted resources of the second type of user as output, and using the fine-tuned resource recommendation model to perform reinforcement learning training to obtain a resource recommendation model. The present scheme can improve the accuracy of resource recommendation.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, and in particular to a training method, resource recommendation method, apparatus, server, storage medium and program product for a resource recommendation model. Background Technology

[0002] When recommending resources to users, accurate recommendations generally rely on user behavior data. Insufficient user behavior data makes it difficult to guarantee the accuracy of recommended resources. For example, for short video content creation platforms, some users have installed the short video app and have a large amount of interactive behavior data, allowing for accurate resource recommendations. However, some users have only interacted with resources recommended by the short video platform's resource alliance but haven't installed the app. This leads to a persistent data sparsity bottleneck in profiling these users, requiring recommendations based only on their static basic attributes and minimal resource conversion behavior, severely impacting recommendation accuracy. Summary of the Invention

[0003] This disclosure provides a training method, a resource recommendation method, an apparatus, a server, a storage medium, and a program product for a resource recommendation model, to at least address the problem of insufficient accuracy in resource recommendation in related technologies. The technical solution of this disclosure is as follows:

[0004] According to a first aspect of the present disclosure, a method for training a resource recommendation model is provided, comprising:

[0005] Obtain a first sample dataset and a second sample dataset; the first sample dataset contains basic user information and user behavior data of a first type of user, and the second sample dataset contains basic user information of a second type of user, wherein the first type of user has more user behavior data than the second type of user; wherein, the basic user information is used to characterize the user's static attribute information, and the user behavior data is used to characterize the user's interaction behavior data with resources in the application of the content creation platform;

[0006] Using the basic user information of the first type of users as input, the predicted user profile and predicted resources of the first type of users as output, and the user behavior data of the first type of users as supervision information, the pre-trained resource recommendation model is supervised and fine-tuned to obtain the fine-tuned resource recommendation model.

[0007] Using the basic user information of the second type of users as input, and the predicted user profile and predicted resources of the second type of users as output, the fine-tuned resource recommendation model is trained by reinforcement learning to obtain the resource recommendation model.

[0008] In an exemplary embodiment, the step of supervising the fine-tuning of a pre-trained resource recommendation model, using basic user information of the first type of users as input, predicted user profiles and predicted resources of the first type of users as output, and user behavior data of the first type of users as supervision information, to obtain a fine-tuned resource recommendation model, includes:

[0009] Based on the basic user information and user behavior data of the first type of users, user profile tags and resource tags for the first type of users are constructed.

[0010] The pre-trained resource recommendation model processes the basic user information of the first type of users and outputs the predicted user profile and predicted resources for the first type of users.

[0011] Based on the difference information between the predicted user profile and the user profile label, and the difference information between the predicted resource and the resource label, the pre-trained resource recommendation model is subjected to supervised fine-tuning to obtain the fine-tuned resource recommendation model.

[0012] In one exemplary embodiment, the output of the pre-trained resource recommendation model further includes at least one of the model's internal thought process and the reasons for predicting the predicted resource;

[0013] The method further includes:

[0014] Based on the basic user information and user behavior data of the first type of users, construct thinking process tags and reason tags corresponding to the resource tags;

[0015] The pre-trained resource recommendation model processes the basic user information of the first type of users and outputs the predicted user profile, thought process, predicted resources, and prediction reasons for the first type of users.

[0016] Based on the differences between the predicted user profile and the user profile label, the differences between the thought process and the thought process label, the differences between the predicted resource and the resource label, and the differences between the predicted reason and the reason label, the pre-trained resource recommendation model is subjected to supervised fine-tuning to obtain the fine-tuned resource recommendation model.

[0017] In an exemplary embodiment, the step of training the fine-tuned resource recommendation model using basic user information of the second type of users as input and predicted user profiles and predicted resources of the second type of users as output through reinforcement learning to obtain the resource recommendation model further includes:

[0018] Construct resource tags for the second type of users, and construct a baseline output format for predicting user profiles and resources;

[0019] The basic user information of the second type of users is processed by the finely tuned resource recommendation model to output the predicted user profile and predicted resources of the second type of users.

[0020] The resource recommendation model is trained by reinforcement learning using at least one of the following rewards: the length of the text corresponding to the predicted user profile is reasonable, the predicted resource is consistent with the resource tag, and the output format of the predicted user profile and the predicted resource matches the benchmark output format.

[0021] In one exemplary embodiment, the output of the fine-tuned resource recommendation model further includes at least one of the model's internal thought process and the reasons for predicting the predicted resource;

[0022] The baseline output format for predicting user profiles and resources includes:

[0023] Construct a baseline output format for the predicted user profile, the thought process, the predicted resources, and the predicted reasons.

[0024] In one exemplary embodiment, resource recommendation is performed through a target content creation platform;

[0025] The construction of resource tags for the second type of user includes:

[0026] Obtain the successful and failed recommendation data of the target content creation platform for the second type of user in the resource recommendation competition;

[0027] The first type of resources successfully recommended by the target content creation platform are obtained from the successful recommendation data, and the second type of resources successfully recommended by other content creation platforms are obtained from the failed recommendation data; the second type of resources are resources whose recommendation index values ​​meet the conditions.

[0028] The first type of resource and the second type of resource are identified as resource tags.

[0029] According to a second aspect of the present disclosure, a resource recommendation method is provided, comprising:

[0030] Obtain basic user information of the target user;

[0031] The basic user information is processed by calling a resource recommendation model, which outputs a predicted user profile and predicted resources for the target user; the resource recommendation model is trained using the training method for the resource recommendation model described in any of the above embodiments.

[0032] Based on the predicted user profile and the predicted resources, recommended resources are determined for the target user.

[0033] According to a third aspect of the present disclosure, a training apparatus for a resource recommendation model is provided, comprising:

[0034] The sample acquisition unit is configured to acquire a first sample dataset and a second sample dataset; the first sample dataset contains basic user information and user behavior data of a first type of user, and the second sample dataset contains basic user information of a second type of user, wherein the first type of user has more user behavior data than the second type of user; wherein the basic user information is used to characterize the user's static attribute information, and the user behavior data is used to characterize the user's interaction behavior data with resources in the application of the content creation platform;

[0035] The first training unit is configured to perform supervised fine-tuning of the pre-trained resource recommendation model, taking the basic user information of the first type of users as input, the predicted user profile and predicted resources of the first type of users as output, and the user behavior data as supervision information, to obtain the fine-tuned resource recommendation model.

[0036] The second training unit is configured to perform reinforcement learning training on the fine-tuned resource recommendation model, taking the basic user information of the second type of users as input and the predicted user profile and predicted resources of the second type of users as output, to obtain the resource recommendation model.

[0037] In an exemplary embodiment, the first training unit is configured to: construct user profile tags and resource tags for the first type of users based on basic user information and user behavior data of the first type of users; process the basic user information of the first type of users through the pre-trained resource recommendation model to output predicted user profiles and predicted resources for the first type of users; and perform supervised fine-tuning of the pre-trained resource recommendation model based on the difference information between the predicted user profile and the user profile tags, and the difference information between the predicted resources and the resource tags, to obtain a fine-tuned resource recommendation model.

[0038] In an exemplary embodiment, the output of the pre-trained resource recommendation model further includes at least one of the model's internal thought process and the prediction reason for the predicted resource; a first training unit is configured to perform operations based on the basic user information and user behavior data of the first type of user, constructing thought process tags and reason tags corresponding to the resource tags; process the basic user information of the first type of user through the pre-trained resource recommendation model, and output the predicted user profile, thought process, predicted resource, and prediction reason for the first type of user; and perform supervised fine-tuning of the pre-trained resource recommendation model based on the difference information between the predicted user profile and the user profile tags, the difference information between the thought process and the thought process tags, the difference information between the predicted resource and the resource tags, and the difference information between the predicted reason and the reason tags, to obtain a fine-tuned resource recommendation model.

[0039] In one exemplary embodiment, the second training unit is configured to construct resource tags for the second type of user and construct a baseline output format for predicted user profiles and predicted resources; process the basic user information of the second type of user through the fine-tuned resource recommendation model, and output predicted user profiles and predicted resources for the second type of user; and perform reinforcement learning training on the fine-tuned resource recommendation model with a reward constructed from at least one of the following: the length of the text corresponding to the predicted user profile is reasonable, the predicted resources are consistent with the resource tags, and the output format of the predicted user profile and the predicted resources matches the baseline output format, to obtain a resource recommendation model.

[0040] In one exemplary embodiment, the output of the fine-tuned resource recommendation model further includes at least one of the model's internal thought process and the prediction reason for the predicted resource; the second training unit is configured to execute a baseline output format for constructing the predicted user profile, the thought process, the predicted resource, and the prediction reason.

[0041] In one exemplary embodiment, resource recommendation is performed through a target content creation platform; a second training unit is configured to perform the following operations: acquire successful recommendation data and failed recommendation data of the target content creation platform for the second type of users in a resource recommendation competition; acquire a first type of resource successfully recommended by the target content creation platform from the successful recommendation data, and acquire a second type of resource successfully recommended by other content creation platforms from the failed recommendation data; the second type of resource is a resource whose recommendation index value meets the conditions; and determine the first type of resource and the second type of resource as resource tags.

[0042] According to a fourth aspect of the present disclosure, a resource recommendation apparatus is provided, comprising:

[0043] The information acquisition unit is configured to acquire basic user information of the target user.

[0044] The model processing unit is configured to execute the resource recommendation model to process the basic user information and output a predicted user profile and predicted resources for the target user; the resource recommendation model is trained by the resource recommendation model training method described in any of the above embodiments;

[0045] The resource recommendation unit is configured to determine recommended resources for the target user based on the predicted user profile and the predicted resources.

[0046] According to a fifth aspect of the present disclosure, a server is provided, comprising:

[0047] processor;

[0048] Memory used to store the processor's executable instructions;

[0049] The processor is configured to execute the instructions to implement the method as described in any of the preceding methods.

[0050] According to a sixth aspect of the present disclosure, a computer-readable storage medium is provided that, when instructions in the computer-readable storage medium are executed by a processor of a server, enables the server to perform the method as described in any of the preceding claims.

[0051] According to a seventh aspect of the present disclosure, a computer program product is provided, the computer program product including instructions that, when executed by a processor of a server, enable the server to perform the method as described in any of the preceding claims.

[0052] The technical solutions provided by the embodiments of this disclosure have at least the following beneficial effects:

[0053] The acquired training data includes a first sample dataset containing basic user information and user behavior data of the first type of users, and a second sample dataset containing basic user information of the second type of users. During training, a two-stage training method is set up. In the first stage, the basic user information of the first type of users is used as input, and the predicted user profile and predicted resources of the first type of users are used as output. The user behavior data of the first type of users is used as supervision information to perform supervised fine-tuning on the pre-trained resource recommendation model, resulting in a fine-tuned resource recommendation model. In the second stage, the basic user information of the second type of users is used as input, and the predicted user profile and predicted resources of the second type of users are used as output. The fine-tuned resource recommendation model is then trained using reinforcement learning to obtain the resource recommendation model. In this method, the first type of users has more user behavior data than the second type of users. Therefore, through two-stage training of the resource recommendation model, the first stage learns user behavior patterns from the first type of users with massive amounts of real user behavior data, and the second stage transfers the learned user behavior pattern knowledge to the scenario of the second type of users. By making full use of the generation and reasoning capabilities of the resource recommendation model, the bottleneck of data sparsity for the second type of users is overcome, and an enhanced, inferential behavioral profile is constructed for them, thereby significantly improving the accuracy of resource conversion prediction.

[0054] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0055] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure, and are not intended to unduly limit this disclosure.

[0056] Figure 1 This is a flowchart illustrating a training method for a resource recommendation model according to an exemplary embodiment.

[0057] Figure 2 This is a schematic diagram illustrating a method for constructing user profile tags based on basic user information and user behavior data, according to an exemplary embodiment.

[0058] Figure 3 This is a flowchart illustrating a resource recommendation method according to an exemplary embodiment.

[0059] Figure 4 This is a schematic diagram illustrating the principle of a training method for a resource recommendation model according to an exemplary embodiment.

[0060] Figure 5 This is a structural block diagram illustrating a training apparatus for a resource recommendation model according to an exemplary embodiment.

[0061] Figure 6 This is a structural block diagram illustrating a resource recommendation device according to an exemplary embodiment.

[0062] Figure 7 This is a block diagram illustrating a server according to an exemplary embodiment. Detailed Implementation

[0063] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0064] It should be noted that the embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0065] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. It should also be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for display, data used for analysis, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties.

[0066] The core technical problem addressed in this application is that for single users who only utilize resource alliances on content creation platforms, the available user behavior data dimensions are extremely limited (only basic attributes and some alliance-side interaction data), leading to a severe representational bottleneck in traditional user profiling methods. Under this data sparsity constraint, existing technologies struggle to fully understand and accurately characterize these users, thus limiting the upper limit of the effectiveness of resource recommendation systems and further exploration of their commercial value.

[0067] Based on this, this application proposes an innovative technical solution that can achieve probabilistic inference and dynamic modeling of the deep intentions of individual users under limited data conditions, providing key technical support for the deployment and upgrading of resource recommendation systems.

[0068] refer to Figure 1This is a flowchart illustrating a training method for a resource recommendation model provided in an exemplary embodiment. This embodiment uses the application of this method to a server as an example for illustration. It is understood that this method can also be applied to a terminal, and further to a system including both a terminal and a server, and is implemented through interaction between the terminal and the server. The server can be a standalone server or a server cluster composed of multiple servers. In this embodiment, the method includes the following steps:

[0069] In step S110, a first sample dataset and a second sample dataset are obtained; the first sample dataset contains basic user information and user behavior data of the first type of users, and the second sample dataset contains basic user information of the second type of users. The first type of users have more user behavior data than the second type of users; wherein, the basic user information is used to characterize the static attribute information of users, and the user behavior data is used to characterize the interaction behavior data of users with resources in the application of the content creation platform.

[0070] In this step, the first type of user refers to users with abundant behavioral data, while the second type refers to users with sparse behavioral data. For example, regarding a content creation platform, the first type of user consists of those who have the corresponding application installed and are active on the platform's resource alliance SDK (Software Development Kit), which can be understood as "dual-users." The second type of user consists of those who have not installed the corresponding application and only interact with resources recommended by the platform's resource alliance, which can be understood as "single-users." For instance, if the content creation platform is Platform A, then the first type of user has the Platform A app installed and is active on both the Platform A app and the Platform A resource alliance, while the second type of user does not have the Platform A app installed and only interacts with the Platform A resource alliance.

[0071] Therefore, the first sample dataset obtained for the first type of users not only contains basic user information but also rich user behavior data of the first type of users within the content creation platform application. The second sample dataset obtained for the second type of users, however, mainly contains basic user information and lacks user behavior data of the second type of users within the application.

[0072] Basic user information refers to a user's relatively static basic attributes, such as gender, age, occupation, place of residence, information about the mobile device used (such as phone model, price, etc.), installed applications, and frequently used applications. User behavior data refers to the massive, continuous, and multi-dimensional interactive data generated in applications, such as video production and consumption, e-commerce consumption, and commercialization conversion data.

[0073] It's important to note that the basic user information fields for the first type of users are aligned with those for the second type of users. This means that the basic user information for the first type of users input into the model will not include any basic user information that the second type of users lack, and vice versa. For example, if the basic user information fields for the second type of users include gender, age, occupation, and place of residence, then the basic user information fields for the first type of users will also include these fields. This ensures that the user behavior patterns learned by the model from the first type of users can be transferred to the scenarios involving the second type of users.

[0074] In step S120, the pre-trained resource recommendation model is fine-tuned by taking the basic user information of the first type of users as input, the predicted user profile and predicted resources of the first type of users as output, and the user behavior data of the first type of users as supervision information, to obtain the fine-tuned resource recommendation model.

[0075] The pre-trained resource recommendation model is a large language model, which can be trained using real user profiles of several sample users. For example, the real user profiles of the sample users can be used as input to perform unsupervised training on the initially constructed large language model, enabling the large language model to learn the relevant capabilities for describing user profiles.

[0076] The user profile is a description of the user in natural language, also known as a user biography. For example, a user profile might be: "This is a male user aged xx, who frequently uses a Y-type mobile phone with a xx system costing xx yuan. His current residence is xx. He has these apps installed on his phone: xx. He recently posted different categories of videos on the platform, including xx; he searched for these contents: xx…". The predicted user profile output by the large language model includes the user's possible behavioral trajectories within the content creation platform inferred from the user's basic user information, including various behavioral sequences such as video behavior and consumption behavior. For example, "This user may watch videos of the xx category, including xx; he may search for the following contents: xx;…".

[0077] Among them, predicted resources refer to resources that users may be interested in.

[0078] The first type of users has basic user information and user behavior data. Therefore, in this stage, user behavior data can be used as supervised information, and a supervised fine-tuning training method can be used to train the pre-trained resource recommendation model.

[0079] To enable the resource recommendation model to learn the basic ability to rewrite user behavior profiles and associate them with resource predictions based on limited basic user information, this stage of training uses the basic user information of the first type of users as input, the predicted user profile and predicted resources of the first type of users as output, and the user behavior data of the first type of users as supervision information for supervised fine-tuning training.

[0080] The training in this step enables the crucial "knowledge transfer" function: by supervising and fine-tuning on massive amounts of real user behavior data from the first type of users, the resource recommendation model can gain a deep understanding of user behavior patterns within the content creation platform, and internalize the mapping relationship between different attribute groups (such as "50-year-old male living in xx province") and their typical behavioral sequences into the model parameters.

[0081] Therefore, when the resource recommendation model encounters a second type of users with similar attributes, it can infer and generate a matching, reasonable and coherent simulated behavior sequence (i.e., "user profile") based on the learned group behavior patterns, thus providing richer inference basis for resource conversion prediction.

[0082] In step S130, the basic user information of the second type of users is used as input, and the predicted user profile and predicted resources of the second type of users are used as output. The fine-tuned resource recommendation model is trained by reinforcement learning to obtain the resource recommendation model.

[0083] The second type of user has basic user information but lacks user behavior data, making supervision through user behavior data impossible. Therefore, this stage employs reinforcement learning to train the fine-tuned resource recommendation model. Reinforcement learning is a goal-oriented learning method that operates through interaction. It does not provide a pre-given "correct" answer (label) but instead explores and optimizes strategies in a continuous decision sequence through feedback from reward signals. Therefore, during training, the basic user information of the second type of user is used as input, and the predicted user profile and predicted resources for the second type of user are used as outputs. Reward settings can be configured to train the fine-tuned resource recommendation model using reinforcement learning. The reward signal can be set from one or more dimensions, such as the predicted user profile, predicted resources, and the format of the model's output results. For example, the reward set from the predicted user profile dimension could be the reasonableness of the text length corresponding to the predicted user profile; the reward set from the predicted resources dimension could be the accuracy of the predicted resources; and the reward set from the format of the model's output results dimension could be the standardization of the output format.

[0084] Through the reinforcement learning mechanism and reward settings described above, the resource recommendation model gradually adjusts its parameters during training, making its output more inclined to generate user profile predictions and resource predictions that can obtain high advantage scores, thereby systematically improving the reliability, richness, and accuracy of the final resource recommendation.

[0085] It should be noted that there may be one or more predicted user profiles output in steps S120 and S130 above. Similarly, there may be one or more predicted resources.

[0086] In the training method of the above resource recommendation model, the acquired training data includes a first sample dataset containing basic user information and user behavior data of the first type of users, and a second sample dataset containing basic user information of the second type of users. During training, a two-stage training is set up. In the first stage, the basic user information of the first type of users is used as input, and the predicted user profile and predicted resources of the first type of users are used as output. User behavior data is used as supervision information to perform supervised fine-tuning on the pre-trained resource recommendation model to obtain the fine-tuned resource recommendation model. In the second stage, the basic user information of the second type of users is used as input, and the predicted user profile and predicted resources of the second type of users are used as output to perform reinforcement learning training on the fine-tuned resource recommendation model to obtain the resource recommendation model. In this method, the first type of users has more user behavior data than the second type of users. Therefore, through two-stage training of the resource recommendation model, the first stage learns user behavior patterns from the first type of users with massive amounts of real user behavior data, and the second stage transfers the learned user behavior pattern knowledge to the scenario of the second type of users. By making full use of the generation and reasoning capabilities of the resource recommendation model, the bottleneck of data sparsity for the second type of users is overcome, and an enhanced, inferential behavioral profile is constructed for them, thereby significantly improving the accuracy of resource conversion prediction.

[0087] As mentioned above, the model training in this application mainly includes two training phases, and the training process of the two training phases will be described below.

[0088] In the first stage, in an exemplary embodiment, step S120 takes the basic user information of the first type of users as input, the predicted user profile and predicted resources of the first type of users as output, and the user behavior data of the first type of users as supervision information to perform supervised fine-tuning on the pre-trained resource recommendation model to obtain the fine-tuned resource recommendation model. This includes: constructing user profile labels and resource labels for the first type of users based on the basic user information and user behavior data of the first type of users; processing the basic user information of the first type of users through the pre-trained resource recommendation model to output the predicted user profile and predicted resources of the first type of users; and performing supervised fine-tuning on the pre-trained resource recommendation model based on the difference information between the predicted user profile and the user profile labels, as well as the difference information between the predicted resources and the resource labels, to obtain the fine-tuned resource recommendation model.

[0089] Specifically, before conducting the first stage of model training, supervisory information can be pre-constructed based on the basic user information and user behavior data of the first type of users. This includes user profile tags and resource tags. The user profile tags serve as supervisory information for the predicted user interactions in the model output, while the resource tags serve as supervisory information for the predicted resources in the model output.

[0090] For example, such as Figure 2 As shown, the basic user information for the first type of user includes gender, age, occupation, etc. User behavior data includes video behavior (such as following bloggers, recent searches, etc.), e-commerce behavior (such as browsing, purchasing, etc.), and commercial behavior (such as browsing, clicking, etc.). The user profile tags constructed by integrating the basic user information and user behavior data can be: "This is a male user aged xx, childless. He frequently uses a Y-type mobile phone with a xx system costing 1299 yuan. His current residence is xx. He has these apps installed on his phone: xx. He recently posted different types of videos on the platform, including xx; he searched for these contents: xx, ...". The constructed resource tags can be resources associated with the positive feedback behaviors of the first type of user (such as conversion behaviors like purchases).

[0091] During supervised fine-tuning training, basic user information of the first type of users is input into the pre-trained resource recommendation model for processing, outputting predicted user profiles and predicted resources for the first type of users. A first loss value (i.e., difference information) between the predicted user profile and user profile labels, and a second loss value (i.e., difference information) between the predicted resources and resource labels are calculated using a loss function. The training objective is to reduce the total loss of the first and second loss values ​​to perform supervised fine-tuning of the pre-trained resource recommendation model. In some embodiments, weights can be assigned to user profiles and resources respectively, and the first and second loss values ​​can be weighted and summed to obtain the total loss. The training objective is to reduce the total loss.

[0092] In this embodiment, the resource recommendation model is supervised and fine-tuned using real user behavior data of the first type of users. This allows the resource recommendation model to gain a deeper understanding of user behavior patterns within the content creation platform and internalize the mapping relationship between different attribute groups and their typical behavior sequences into the model parameters. This gives the resource recommendation model the basic ability to continue writing user behavior profiles and associate resource predictions based on limited information.

[0093] Furthermore, in an exemplary embodiment, the output of the pre-trained resource recommendation model further includes at least one of the model's internal thought process and the prediction reason for the predicted resource; the method further includes: constructing thought process labels and reason labels corresponding to resource labels based on the basic user information and user behavior data of the first type of users; processing the basic user information of the first type of users through the pre-trained resource recommendation model to output the predicted user profile, thought process, predicted resource, and prediction reason for the first type of users; and performing supervised fine-tuning of the pre-trained resource recommendation model based on the difference information between the predicted user profile and user profile labels, the difference information between the thought process and the thought process labels, the difference information between the predicted resource and the resource labels, and the difference information between the predicted reason and the reason labels to obtain the fine-tuned resource recommendation model.

[0094] The internal thought process of the model is used to demonstrate the reasoning logic of the model from the user's basic user information to the user's behavior.

[0095] In this context, the prediction reason refers to the rationale for inferring that a predicted resource is one that a user might be interested in. Therefore, the prediction reason corresponds to the predicted resource, and similarly, the resource tag corresponds to the reason tag.

[0096] In this embodiment, the output of the pre-trained resource recommendation model includes not only predicted user profiles and predicted resources, but also at least one of the model's internal thought process and the reasons for predicting the resources. Therefore, before training the pre-trained resource recommendation model, it is also necessary to construct thought process labels and reason labels corresponding to resource labels, which serve as supervision information for the model's internal thought process and prediction reasons, respectively. For example, thought process labels and reason labels corresponding to resource labels can be constructed based on the basic user information and user behavior data of the first type of user. In one implementation, an open-source large model can be used to construct thought process labels and reason labels corresponding to resource labels based on the basic user information and user behavior data of the first type of user.

[0097] Furthermore, during actual training, the basic user information of the first type of users is input into the pre-trained resource recommendation model for processing. The model outputs a predicted user profile, thought process, predicted resource, and predicted reason for the first type of user. A loss function is used to calculate the first loss value between the predicted user profile and user profile labels, the second loss value between the predicted resource and resource labels, the third loss value between the thought process and thought process labels, and the fourth loss value between the predicted reason and reason labels. The training objective is to reduce the total loss of the first, second, third, and fourth loss values, and to perform supervised fine-tuning of the pre-trained resource recommendation model. In some embodiments, weights can be assigned to the four outputs—user profile, resource, thought process, and reason—to obtain a weighted sum of the individual loss values, and training can then be performed with the goal of reducing this total loss.

[0098] In this embodiment, based on the predicted resources and predicted user profiles, the output of the resource recommendation model also includes the model's internal thought process and the reasons for predicting the resources. By introducing two dimensions of supervisory information—the model's internal thought process and the reasons for prediction—to conduct supervised fine-tuning of the resource recommendation model, the accuracy of the prediction results of the resource recommendation model can be further improved.

[0099] The second stage: In an exemplary embodiment, step S130 takes the basic user information of the second type of user as input and the predicted user profile and predicted resources of the second type of user as output, and performs reinforcement learning training on the fine-tuned resource recommendation model to obtain the resource recommendation model. It also includes: constructing resource tags for the second type of user and constructing a baseline output format for the predicted user profile and predicted resources; processing the basic user information of the second type of user through the fine-tuned resource recommendation model to output the predicted user profile and predicted resources of the second type of user; and performing reinforcement learning training on the fine-tuned resource recommendation model using at least one of the following rewards: reasonable length of the text corresponding to the predicted user profile, consistency between the predicted resources and resource tags, and matching of the output format of the predicted user profile and predicted resources with the baseline output format, to obtain the resource recommendation model.

[0100] The baseline output format for predicting user profiles and resources can be a structured format, for example, outputting the predicted user profile first, followed by the predicted resource. In an exemplary embodiment, if the output of the fine-tuned resource recommendation model also includes at least one of the model's internal thought process and the prediction rationale for the predicted resource, then constructing the baseline output format for predicting user profiles and resources includes: constructing a baseline output format for predicting user profiles, thought process, predicted resources, and prediction rationale. Accordingly, the basic output format can be: outputting the predicted user profile first, then the thought process, then the predicted resource and prediction rationale. The setting of the baseline output format guides the resource recommendation model to output each output task sequentially.

[0101] The reinforcement learning method used in this embodiment can be the GRPO (Group Relative Policy Optimization) algorithm, which optimizes the model policy by comparing the relative rewards of samples within the group, avoiding the dependence of traditional methods on value models and significantly improving training efficiency.

[0102] This stage primarily utilizes reinforcement learning to retrain the resource recommendation model trained in the first stage. The input consists of basic user information for the second type of users, and the core task is to autonomously generate user profiles and predict resources. Unlike supervised learning, reinforcement learning does not rely on explicitly labeled training data; instead, it optimizes policies through reward signals, thereby training the model. Therefore, this embodiment mainly focuses on a detailed description of the rewards set in the second stage.

[0103] The reward signals in the second phase mainly involve the following three types:

[0104] Reasonable reward for user profile text length: This reward guides the resource recommendation model to generate user profile texts of moderate length and reasonable information density. This reward ensures that the generated user profiles are both rich and realistic. Without this constraint, the text of user profiles generated during training tends to be shorter, resulting in insufficient information; however, it is also not advisable to encourage the model to generate excessively long texts, so as to avoid model illusion and endless generation of long texts.

[0105] Accuracy Reward for Predicted Resources: This reward is used to improve the accuracy of resource recommendation models in predicting resources. A positive reward is given if the predicted resources of the resource recommendation model match the constructed true targets (resource labels, winning resources from resource recommendation competitions, or losing resources with high recommendation metric values).

[0106] Model output format conformance bonus: This bonus ensures that the model output format conforms to the required baseline output format. A positive bonus is awarded for correct output format.

[0107] After setting the reward signals, before proceeding to the second stage of model training, the necessary content for each reward signal is pre-constructed, including: resource labels required for the reward based on predicted resource accuracy, and a baseline output format required for the reward based on the standardization of the model output format. Further, during actual training, the basic user information of the second type of users is input into the fine-tuned resource recommendation model obtained from the first stage of training for processing, outputting predicted user profiles and predicted resources for the second type of users. Reward evaluation is performed based on the output results, and the resource recommendation model is assigned a positive or negative reward for at least one reward signal based on the evaluation results to guide the training of the larger model. This achieves reinforcement learning training of the fine-tuned resource recommendation model, resulting in the final resource recommendation model.

[0108] This embodiment uses the reinforcement learning mechanism and reward settings described above to enable the resource recommendation model to gradually adjust its parameters during training, making its output more inclined to generate profile inferences and resource predictions that can obtain high advantage scores, thereby systematically improving the reliability, richness and accuracy of the generated content and the final resource recommendation.

[0109] In one exemplary embodiment, resource recommendation is performed through a target content creation platform; constructing resource tags for a second type of user includes: obtaining successful and failed recommendation data of the target content creation platform for the second type of user in a resource recommendation competition; obtaining first-type resources successfully recommended by the target content creation platform from the successful recommendation data, and obtaining second-type resources successfully recommended by other content creation platforms from the failed recommendation data; the second-type resources are resources whose recommendation index values ​​meet the conditions; and determining the first-type resources and the second-type resources as resource tags.

[0110] It is understood that the resource tags for the second type of users in this embodiment are used to evaluate the accuracy of the resource recommendation model's predicted resources, in order to obtain a reward for the accuracy of the predicted resources and guide the model training. Therefore, the construction of resource tags is crucial for the reinforcement learning training of the resource recommendation model in the second stage. Based on this, this embodiment introduces the "competition feedback signal" unique to the resource alliance of the content creation platform to construct resource tags that are both authentic (from this platform) and forward-looking (borrowed from external platforms). The competition feedback signal can be understood as the result data after participating in the resource recommendation competition, containing two types of key data:

[0111] Recommended Success Data (Winning Data): This represents instances where resources recommended by the target content creation platform (this platform) have won and been successfully displayed (i.e., the first type of resources). Their subsequent conversion directly reflects the real resource interests of the second type of users and is used to anchor the model's prediction baseline to ensure that it does not deviate from the actual business situation.

[0112] Recommendation failure data (competition failure data): This represents situations where the resources recommended by the target content creation platform (this platform) failed to win, and the second type of user turned to see resources recommended by other content creation platforms. Although the final conversion results of the resources recommended by other content creation platforms in the recommendation failure data are unknown, it reflects the judgment made by other content creation platforms on the value of the second type of user based on potentially richer user data (such as user behavior data on other content creation platform applications), and contains important reference information.

[0113] Therefore, not only can direct resource tags for the second type of users be constructed from successful recommendation data, but indirect resource tags can also be constructed from failed recommendation data. Furthermore, the first type of resources obtained from successful recommendation data directly reflects users' true resource interests, and thus can be directly used as resource tags. However, the conversion results of resources obtained from failed recommendation data are unknown. Therefore, to improve the quality of resource tags constructed based on failed recommendation data, a recommendation metric can be introduced as a measure. Resources whose recommendation metric values ​​meet the criteria are selected from the failed recommendation data as the second type of resources, and the first type of resources and the selected second type of resources are used as resource tags. The criteria can be that the recommendation metric value ranks in the top N (e.g., top 30%).

[0114] This embodiment constructs resource tags for the second type of users based on the successful and failed recommendation data of the target content creation platform in the resource recommendation competition. This approach draws on the high-confidence recommendation results of external platforms to make up for the lack of behavioral data of the second type of users on its own platform, thereby constructing a reinforcement learning signal that has both internal authenticity and external reference for reward evaluation.

[0115] The resource recommendation model trained in the above two stages has the ability to generate inferential behavioral profiles and resource predictions for "users similar to the second type of users" (it should be noted that similar users here refer to users when the model is applied, and are not the same group of users used for model training in the aforementioned embodiments, but all belong to single users with sparse user behavior data. To avoid confusion with the second type of users in the training stage, they are collectively referred to as target users below). To realize the scalable and efficient application of this capability in the resource recommendation system and to ensure the timeliness of user profiles, this application also designs an inference and deployment architecture with periodic full updates as its core.

[0116] First, the trained resource recommendation model is deployed as a high-performance service. This service receives anonymized basic user information from target users, and the model generates a structured output containing "the model's internal thought process," "user interactions," "predicted resources," and "prediction reasons" through forward computation.

[0117] Secondly, periodically update the full target user profile. To address the issue of dynamically changing user information and prevent the inferred profile from becoming outdated over time, a periodic full update strategy can be adopted. The update cycle can be set according to business needs and computing resources; for example, a fixed update cycle can be set. At the beginning of each cycle, the system triggers a full update task. For example, since the number of target users (i.e., individual users) is large, but the information for each user is relatively small and changes infrequently, the update cycle can be set to fifteen days or one month.

[0118] In some embodiments, to support periodic full updates, a dedicated asynchronous batch inference pipeline can be established, which can process massive amounts of target user data in parallel and efficiently generate inference profiles of all users.

[0119] Each data processing step includes: 1. Input Data: Extracting the latest basic user information for the "target user" from the database to form the input for this inference. 2. Batch Inference: The input data is fed into the established inference pipeline, where the deployed resource model generates the latest "user profile" and "prediction resources" for each target user. 3. Result Storage and Publishing: The generated structured results are written to the feature platform. New user profiles will overwrite the old profiles from the previous period, ensuring that downstream systems always use the latest version of the data.

[0120] Finally, when the resource recommendation engine selects recommended resources for target users, in addition to using their sparse real behavioral features, it will also query and incorporate the "predicted resource" list generated by the resource recommendation model. Furthermore, the "predicted user profile" output by the resource recommendation model greatly enriches the user profile dimensions and can be added to the resource recommendation system as additional features, thereby effectively improving the accuracy of resource matching. After the recommended resources are launched, the system continuously monitors core metrics such as resource conversion rate and click-through rate. Real conversion feedback data from target users guided by the new profile will be collected and used as new reward signals, flowing back to the reinforcement learning training stage for subsequent model iteration and optimization, forming a continuous reinforcement loop of "data-training-deployment-feedback".

[0121] In one exemplary embodiment, such as Figure 3 The diagram shown is a flowchart illustrating a resource recommendation method applying the above-described resource recommendation model according to an exemplary embodiment. In this embodiment, the method includes the following steps:

[0122] Step S310: Obtain the target user's basic user information;

[0123] Step S320: Call the resource recommendation model to process the basic user information and output the predicted user profile and predicted resources for the target user;

[0124] Step S330: Based on the predicted user profile and predicted resources, determine the recommended resources for the target user.

[0125] It is understandable that users similar to the first type of users have abundant user behavior data, so resource recommendations can be made by combining their user behavior data. However, users similar to the second type of users have sparse user behavior data, so only their limited information can be used for resource recommendations. Therefore, this embodiment is mainly applied to single users similar to the second type of users, that is, the target users are mainly single users similar to the second type of users.

[0126] When recommending resources to target users similar to the second type of users, the basic user information is first obtained. This basic user information is then input into the resource recommendation model for processing, outputting a predicted user profile and predicted resources for the target user. In practical applications, the model's internal reasoning process and the rationale for predicting the resources can also be output. Further combining the predicted user profile and predicted resources, the recommended resources for the target user are determined.

[0127] The resource recommendation method provided in this embodiment, when recommending resources to target users with sparse user behavior data, not only uses their sparse real behavior features, but also combines the predicted user profile obtained by the resource recommendation model based on their basic user information, which greatly enriches the dimensions of user characterization. In addition, the resource recommendation model also outputs predicted resources, so the predicted resources can be further verified by combining the sparse real behavior features of the target user and the predicted user profile, the recommendation degree can be determined, and resource recommendation can be carried out, which greatly improves the accuracy of resource recommendation.

[0128] In an exemplary embodiment, specific examples will be described below in conjunction with the accompanying drawings to facilitate understanding of the embodiments of this disclosure by those skilled in the art.

[0129] This application addresses the limitation of sparse user behavior data on the accuracy of resource recommendation by proposing a joint training framework based on a resource recommendation model and reinforcement learning. The guiding model dynamically infers and generates a reasonable behavioral trajectory (i.e., a "user profile") of a user within the platform based on limited basic user information, thereby constructing a high-quality simulated profile. Resource prediction based on this enhanced profile can significantly compensate for the lack of real user behavior data, achieving more accurate conversion prediction.

[0130] refer to Figure 4 This is a schematic diagram illustrating the principle of the training method for a resource recommendation model in an application example, which includes two training phases.

[0131] Phase 1: Supervised fine-tuning (cold start).

[0132] This phase utilizes complete user behavior data from "dual-users" (i.e., the first type of user, users who simultaneously use both the content creation platform's applications and resource alliances) for training. The model input consists of basic user information aligned between dual-users and single-users (such as demographic attributes, installed application lists, etc.), and the supervision objective is to teach the model to output three parts:

[0133] 1. A coherent "user profile" to reasonably deduce the user's possible behavioral trajectory within the platform.

[0134] 2. A "thinking process" within the model, demonstrating its reasoning logic from basic user information to behavioral deduction.

[0135] 3. The predicted resources that the user may convert and the rationale for the prediction.

[0136] For example, the model input might be: "This is a male user aged xx, who frequently uses a Y-type mobile phone with a xx system costing xx yuan. His current residence is xx. He has these apps installed on his phone: xx…". The output user profile might be: "He may post different categories of videos on the platform, including xx; he may search for these contents: xx,…".

[0137] Phase 2: Fine-tuning of reinforcement learning based on feedback signals from participants.

[0138] This stage utilizes basic user information from "single-user users" (i.e., the second type of user, users who only use the resource alliance of the content creation platform) for reinforcement learning training. The model trained in the first stage learns to continue writing the user profile of single-user users and predict resources. Its output can also include the three parts mentioned above: user profile, thought process, predicted resources, and prediction rationale. The entire training process in the second stage mainly involves two core aspects: the construction and alignment of training objectives, and optimization based on a composite reward function.

[0139] Construction and alignment of training objectives: The construction of training objectives is mainly based on the "competition feedback signal" unique to the resource alliance of the content creation platform. This signal is both realistic (from this platform) and forward-looking (by reference to external platforms). The signal mainly includes two types of key data: winning data (i.e., successful recommendation data) and losing data (i.e., failed recommendation data), as detailed in the above embodiments, and will not be repeated here.

[0140] Model Optimization Based on a Composite Reward Function: Based on the aforementioned objectives, a composite reward function was designed to optimize the model's generation results. Its core mechanism is as follows: the model generates multiple possible behavioral predictions and profile descriptions for the same user each time (i.e., multiple candidate users and their corresponding prediction resources). Through an advantage function evaluation module built based on the aforementioned real conversion feedback and competition failure signals, the system can quantify the advantage of each prediction within the group relative to the actual benchmark, thereby identifying which predictions perform better.

[0141] The advantage function evaluation module, as an important part of this reinforcement learning training, evaluates the following three rewards: the reasonableness of the user profile text length, the accuracy of the prediction resources, and the standardization of the model output format.

[0142] The two-stage training process essentially transfers the "behavioral pattern knowledge" learned from dual-user scenarios to single-user scenarios, and further utilizes the winning and losing signals of the resource alliance for target alignment and reinforcement. This fully leverages the generation and reasoning capabilities of the resource recommendation model, overcomes the bottleneck of sparse single-user data, and constructs an enhanced, inferential behavioral profile for them, thereby significantly improving the accuracy of resource conversion prediction and providing an effective technical path for optimizing resource recommendation strategies.

[0143] This method offers the following advantages: 1. **User Profile Generation Paradigm:** It proposes generating simulated behavioral sequence descriptions (user profiles) for single-user scenarios with sparse data, serving as a core intermediate representation to enhance these profiles and transforming limited information into inferable semantic behavioral trajectories. 2. **Two-Stage Training Transfer:** First, it utilizes supervised fine-tuning with complete data from "dual-users" (i.e., the first type of user), enabling the resource recommendation model to learn to continue writing user profiles and predict resources. Then, it optimizes for "single-users" (i.e., the second type of user) through reinforcement learning, achieving knowledge transfer from rich data to sparse scenarios. 3. **Compound Reward-Driven Optimization:** A reinforcement learning reward function is designed that integrates the accuracy of predicted resources, the standardization of output format, and the reasonableness of the text length of user profiles. This ensures accurate resource prediction and a standardized model output format, while generating user profiles with appropriate information density. 4. **End-to-End Inference Paradigm:** The resource recommendation model can simultaneously output user profiles and predicted resources in a single process, tightly coupling behavioral inference with the final goal, providing a complete solution for understanding single-user scenarios and accurate recommendations.

[0144] This solution continuously provides information-enhancing understanding for single users on content creation platforms, expanding their behavioral sequences while offering more precise capture of resource interests. This solution primarily applies to single users within resource alliances, and its effectiveness has been demonstrated through online A / B testing.

[0145] 1. Alliance as a whole:

[0146] a. Remove shallow inner loop: Expected cost (ROI consumption) increases by 0.760%, consumption increases by 0.382%, and exposure increases by 0.186%.

[0147] b. KACE (Key-Action-Conversion-Engagement of User's Relationship to Resources, a composite feature of key behaviors, attitudes, and conversions) (0.8 confidence level):

[0148] i. Posterior expected consumption increased by 0.656% [0.21%, 1.11%].

[0149] ii. Corrected cost consumption increased by 0.649% [0.23%, 1.07%].

[0150] 2. Single user: Expected spending (ROI) increased by 1.579%, spending increased by 1.613%, and exposure increased by 0.633%.

[0151] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0152] It is understood that the same / similar parts between the various embodiments of the methods described above in this specification can be referred to each other. Each embodiment focuses on the differences from other embodiments, and relevant parts can be referred to the description of other method embodiments.

[0153] Based on the same inventive concept, this disclosure also provides a training apparatus for a resource recommendation model for implementing the training method of the resource recommendation model mentioned above, and a resource recommendation apparatus for implementing the resource recommendation method mentioned above.

[0154] Figure 5 This is a structural block diagram illustrating a training apparatus for a resource recommendation model according to an exemplary embodiment. (Refer to...) Figure 5 The device includes:

[0155] The sample acquisition unit 510 is configured to acquire a first sample dataset and a second sample dataset; the first sample dataset contains basic user information and user behavior data of a first type of user, and the second sample dataset contains basic user information of a second type of user, with the first type of user having more user behavior data than the second type of user.

[0156] The first training unit 520 is configured to perform supervised fine-tuning of the pre-trained resource recommendation model, taking the basic user information of the first type of users as input, the predicted user profile and predicted resources of the first type of users as output, and user behavior data as supervision information, to obtain the fine-tuned resource recommendation model.

[0157] The second training unit 530 is configured to perform reinforcement learning training on the fine-tuned resource recommendation model, taking the basic user information of the second type of users as input and the predicted user profile and predicted resources of the second type of users as output, to obtain the resource recommendation model.

[0158] In an exemplary embodiment, the first training unit 520 is configured to perform the following operations: construct user profile tags and resource tags for the first type of users based on basic user information and user behavior data of the first type of users; process the basic user information of the first type of users through a pre-trained resource recommendation model to output predicted user profiles and predicted resources for the first type of users; and perform supervised fine-tuning of the pre-trained resource recommendation model based on the difference information between the predicted user profile and the user profile tags, as well as the difference information between the predicted resources and the resource tags, to obtain a fine-tuned resource recommendation model.

[0159] In an exemplary embodiment, the output of the pre-trained resource recommendation model further includes at least one of the model's internal thought process and the prediction reason for the predicted resource; the first training unit 520 is configured to perform operations based on the basic user information and user behavior data of the first type of user, constructing thought process labels and reason labels corresponding to resource labels; process the basic user information of the first type of user through the pre-trained resource recommendation model, and output the predicted user profile, thought process, predicted resource, and prediction reason for the first type of user; based on the difference information between the predicted user profile and user profile labels, the difference information between the thought process and the thought process labels, the difference information between the predicted resource and the resource labels, and the difference information between the predicted reason and the reason labels, the pre-trained resource recommendation model is subjected to supervised fine-tuning to obtain the fine-tuned resource recommendation model.

[0160] In an exemplary embodiment, the second training unit 530 is configured to perform tasks such as constructing resource tags for a second type of user and constructing a baseline output format for predicting user profiles and predicting resources; processing the basic user information of the second type of user through a fine-tuned resource recommendation model to output the predicted user profile and predicted resources for the second type of user; and training the fine-tuned resource recommendation model with reinforcement learning based on at least one of the following: the length of the text corresponding to the predicted user profile is reasonable, the predicted resources are consistent with the resource tags, and the output format of the predicted user profile and predicted resources matches the baseline output format, thereby obtaining the resource recommendation model.

[0161] In one exemplary embodiment, the output of the fine-tuned resource recommendation model also includes at least one of the model’s internal thought process and the reason for predicting the predicted resource; the second training unit 530 is configured to perform a baseline output format for constructing the predicted user profile, thought process, predicted resource and prediction reason.

[0162] In one exemplary embodiment, resource recommendation is performed through a target content creation platform; the second training unit 530 is configured to perform the following operations: acquire successful recommendation data and failed recommendation data of the target content creation platform for a second type of user in a resource recommendation competition; acquire first-type resources successfully recommended by the target content creation platform from the successful recommendation data, and acquire second-type resources successfully recommended by other content creation platforms from the failed recommendation data; the second-type resources are resources whose recommendation index values ​​meet the conditions; and determine the first-type resources and the second-type resources as resource tags.

[0163] Figure 6 This is a structural block diagram illustrating a resource recommendation device according to an exemplary embodiment. (Refer to...) Figure 6 The device includes:

[0164] Information acquisition unit 610 is configured to acquire basic user information of the target user;

[0165] The model processing unit 620 is configured to execute the resource recommendation model to process basic user information and output a predicted user profile and predicted resources for the target user; the resource recommendation model is trained by the method of any one of claims 1 to 6;

[0166] Resource recommendation unit 630 is configured to determine recommended resources to target users based on predicted user profiles and predicted resources.

[0167] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0168] Figure 7This is a block diagram illustrating an electronic device 700 for implementing a training method or resource recommendation method for a resource recommendation model, according to an exemplary embodiment. For example, the electronic device 700 can be a server. (Refer to...) Figure 7 The electronic device 700 includes a processing component 720, which further includes one or more processors, and memory resources represented by memory 722 for storing instructions, such as application programs, that can be executed by the processing component 720. The application programs stored in memory 722 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 720 is configured to execute instructions to perform the methods described above.

[0169] The electronic device 700 may further include: a power supply component 724 configured to perform power management of the electronic device 700, a wired or wireless network interface 726 configured to connect the electronic device 700 to a network, and an input / output (I / O) interface 728. The electronic device 700 can operate on an operating system stored in memory 722, such as Windows 7 Server, Mac OS X, Unix, Linux, FreeB7D, or similar.

[0170] In an exemplary embodiment, a computer-readable storage medium including instructions is also provided, such as a memory 722 including instructions, which can be executed by a processor of an electronic device 700 to perform the above-described method. The storage medium may be a computer-readable storage medium, such as a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device.

[0171] In an exemplary embodiment, a computer program product is also provided, the computer program product including instructions that can be executed by a processor of an electronic device 700 to perform the above-described method.

[0172] It should be noted that the above-mentioned apparatus, server, computer-readable storage medium, computer program product, etc., may also include other implementation methods according to the description of the method embodiments. For specific implementation methods, please refer to the description of the relevant method embodiments, which will not be elaborated here.

[0173] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the claims.

[0174] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A training method for a resource recommendation model, characterized in that, include: Obtain a first sample dataset and a second sample dataset; the first sample dataset contains basic user information and user behavior data of a first type of user, and the second sample dataset contains basic user information of a second type of user, wherein the first type of user has more user behavior data than the second type of user; wherein, the basic user information is used to characterize the user's static attribute information, and the user behavior data is used to characterize the user's interaction behavior data with resources in the application of the content creation platform; Using the basic user information of the first type of users as input, the predicted user profile and predicted resources of the first type of users as output, and the user behavior data of the first type of users as supervision information, the pre-trained resource recommendation model is supervised and fine-tuned to obtain the fine-tuned resource recommendation model. Using the basic user information of the second type of users as input, and the predicted user profile and predicted resources of the second type of users as output, the fine-tuned resource recommendation model is trained by reinforcement learning to obtain the resource recommendation model.

2. The method according to claim 1, characterized in that, The process of taking basic user information of the first type of users as input, predicting user profiles and predicted resources of the first type of users as output, and using user behavior data of the first type of users as supervision information to perform supervised fine-tuning on the pre-trained resource recommendation model, to obtain the fine-tuned resource recommendation model, includes: Based on the basic user information and user behavior data of the first type of users, user profile tags and resource tags for the first type of users are constructed. The pre-trained resource recommendation model processes the basic user information of the first type of users and outputs the predicted user profile and predicted resources for the first type of users. Based on the difference information between the predicted user profile and the user profile label, and the difference information between the predicted resource and the resource label, the pre-trained resource recommendation model is subjected to supervised fine-tuning to obtain the fine-tuned resource recommendation model.

3. The method according to claim 2, characterized in that, The output of the pre-trained resource recommendation model also includes at least one of the model's internal thought process and the reasons for predicting the predicted resource. The method further includes: Based on the basic user information and user behavior data of the first type of users, construct thinking process tags and reason tags corresponding to the resource tags; The pre-trained resource recommendation model processes the basic user information of the first type of users and outputs the predicted user profile, thought process, predicted resources, and prediction reasons for the first type of users. Based on the differences between the predicted user profile and the user profile label, the differences between the thought process and the thought process label, the differences between the predicted resource and the resource label, and the differences between the predicted reason and the reason label, the pre-trained resource recommendation model is subjected to supervised fine-tuning to obtain the fine-tuned resource recommendation model.

4. The method according to claim 1, characterized in that, The step of using the basic user information of the second type of users as input and the predicted user profile and predicted resources of the second type of users as output, and performing reinforcement learning training on the fine-tuned resource recommendation model to obtain the resource recommendation model, further includes: Construct resource tags for the second type of users, and construct a baseline output format for predicting user profiles and resources; The basic user information of the second type of users is processed by the finely tuned resource recommendation model to output the predicted user profile and predicted resources of the second type of users. The resource recommendation model is trained by reinforcement learning using at least one of the following rewards: the predicted user profile has a reasonable text length, the predicted resource matches the resource tag, and the output format of the predicted user profile and the predicted resource matches the baseline output format.

5. The method according to claim 4, characterized in that, The output of the fine-tuned resource recommendation model also includes the model's internal thought process and at least one of the reasons for predicting the predicted resource; The baseline output format for predicting user profiles and resources includes: Construct a baseline output format for the predicted user profile, the thought process, the predicted resources, and the predicted reasons.

6. The method according to claim 5, characterized in that, Recommend resources through target content creation platforms; The construction of resource tags for the second type of user includes: Obtain the successful and failed recommendation data of the target content creation platform for the second type of user in the resource recommendation competition; The first type of resources successfully recommended by the target content creation platform are obtained from the successful recommendation data, and the second type of resources successfully recommended by other content creation platforms are obtained from the failed recommendation data; the second type of resources are resources whose recommendation index values ​​meet the conditions. The first type of resource and the second type of resource are identified as resource tags.

7. A resource recommendation method, characterized in that, include: Obtain basic user information of the target user; The resource recommendation model is invoked to process the basic user information, and outputs a predicted user profile and predicted resources for the target user; the resource recommendation model is trained by the method described in any one of claims 1 to 6. Based on the predicted user profile and the predicted resources, recommended resources are determined for the target user.

8. A training device for a resource recommendation model, characterized in that, include: The sample acquisition unit is configured to acquire a first sample dataset and a second sample dataset. The first sample dataset contains basic user information and user behavior data of a first type of user, and the second sample dataset contains basic user information of a second type of user. The first type of user has more user behavior data than the second type of user. The basic user information is used to characterize the user's static attribute information, and the user behavior data is used to characterize the user's interaction behavior with resources in the application of the content creation platform. The first training unit is configured to perform supervised fine-tuning of the pre-trained resource recommendation model, taking the basic user information of the first type of users as input, the predicted user profile and predicted resources of the first type of users as output, and the user behavior data as supervision information, to obtain the fine-tuned resource recommendation model. The second training unit is configured to perform reinforcement learning training on the fine-tuned resource recommendation model, taking the basic user information of the second type of users as input and the predicted user profile and predicted resources of the second type of users as output, to obtain the resource recommendation model.

9. A resource recommendation device, characterized in that, include: The information acquisition unit is configured to acquire basic user information of the target user. The model processing unit is configured to execute a resource recommendation model to process the basic user information and output a predicted user profile and predicted resources for the target user; the resource recommendation model is trained by the method described in any one of claims 1 to 6. The resource recommendation unit is configured to determine recommended resources for the target user based on the predicted user profile and the predicted resources.

10. A server, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the training method of the resource recommendation model as described in any one of claims 1 to 6 or the resource recommendation method as described in claim 7.

11. A computer-readable storage medium, characterized in that, When the instructions in the computer-readable storage medium are executed by the processor of the server, the server is able to perform the training method of the resource recommendation model as described in any one of claims 1 to 6 or the resource recommendation method as described in claim 7.

12. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the training method of the resource recommendation model according to any one of claims 1 to 6 or the resource recommendation method according to claim 7.

Images