Recommended methods, apparatuses, electronic devices, computer storage media, and program products

By training student models and combining self-supervised pre-training and supervised training, the objective function is optimized, and explicit semantic interest circle prediction results are output. This solves the problems of poor interpretability and cross-domain reusability of generative recommendation methods, and decouples user intent planning from underlying item matching, thereby improving recommendation accuracy and content diversity.

CN123115509APending Publication Date: 2026-11-17BEIJING DAJIA INTERNET INFORMATION TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202610921107.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-06-24
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing generative recommendation methods suffer from weak interpretability, poor cross-domain reusability, and unstable inference and prediction performance.

Method used

By training the student model, user profile features and historical behavior sequences are obtained. Self-supervised pre-training and supervised training are combined with multi-dimensional reward functions, interest circle credit weight allocation, word-level advantage value adaptive correction and parameter upper limit truncation constraint mechanism to construct and maximize the optimization objective function, complete the model reinforcement learning iterative optimization, and output explicit semantic inference chain and interest circle prediction results.

Benefits of technology

It achieves user modeling in plaintext semantic form, adapts to multiple business scenarios, and provides clear logical attribution through explicit reasoning chains, improving recommendation accuracy and content diversity, and solving the semantic black box and prediction instability problems in traditional methods.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123115509A_ABST
    Figure CN123115509A_ABST
Patent Text Reader

Abstract

This disclosure relates to a recommendation method, apparatus, electronic device, computer storage medium, and computer program product. The method includes: analyzing and processing user profile features and user historical behavior sequences using a trained student model to obtain inference chains and interest circle prediction results; the interest circles in the interest circle prediction results are plaintext semantic expression units of a user's single interest topic, used to determine recommendation objects. The student model training process includes: completing self-supervised pre-training based on a teacher model and a pre-set interest circle corpus to learn the semantic interpretation of interest circles; then completing supervised training based on user profile features, the inference chain output by the teacher model, and a reference interest circle set; and constructing and maximizing the objective function by combining a multi-dimensional reward function, interest circle credit weight allocation, adaptive correction of word-level advantage values, and parameter upper limit truncation constraint mechanism to complete model reinforcement learning iterative optimization. This improves the accuracy of recommendations.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, and in particular to a recommendation method, apparatus, electronic device, computer storage medium, and computer program product. Background Technology

[0002] With the development of online ecosystems such as short videos, e-commerce, and social media, information overload has become a core pain point in the industry. Sequence recommendation technology has emerged to address this issue. This technology can dynamically predict users' future preferences by modeling their historical interaction sequences, making it a key engine for solving the information overload problem.

[0003] In related technologies, current generative recommendation methods refactor the recommendation task into a sequence-to-sequence language modeling task. Leveraging the contextual understanding and world knowledge capabilities of large language models, they directly generate target items or their identifiers, thus achieving a paradigm shift from "matching" to "generation." Specifically, these include: Method 1, generative recommendation based on Semantic IDs (SIDs): This method maps a user's historical interaction item sequences or user behaviors to SIDs (Semantic IDs) through hierarchical quantization / clustering. After converting these sequences into SID sequences, the model is trained to predict the next SID via autoregression. The predicted SID is then used to retrieve specific items from the index and used as recommended items. Method 2, user multi-interest recommendation based on implicit embeddings: This method uses capsule networks or attention mechanisms to map user historical behavior sequences to implicit vectors (LatentInterest Embeddings). Items are retrieved from the item database based on vector similarity, and the retrieved items are used as recommended items.

[0004] However, generative recommendation methods based on semantic ID suffer from problems such as semantic black box, poor cross-domain reusability, and unstable prediction targets; methods based on implicit embedding lack plaintext semantics and interpretability, and have no explicit reasoning ability. Therefore, related technologies suffer from weak interpretability, poor cross-domain reusability, and unstable reasoning and prediction effects. Summary of the Invention

[0005] This disclosure provides a recommendation method, apparatus, electronic device, computer storage medium, and computer program product to at least solve the problems of weak interpretability, poor cross-domain reusability, and unstable inference and prediction performance in related technologies. The technical solution of this disclosure is as follows:

[0006] According to a first aspect of the present disclosure, a recommendation method is provided, the method comprising:

[0007] The system acquires user profile features and user historical behavior sequences. The user profile features are used to characterize the statistical distribution features of each user's interactive behavior within the video platform in each level of interest circle under a predefined semantic space. The user historical behavior sequence is a sequence formed by mapping each user's interactive behavior within a preset time period to the predefined semantic space and sorting it by time.

[0008] The trained student model analyzes and processes the user profile features and the user's historical behavior sequence to obtain inference chain and interest circle prediction results. The inference chain is an explicit semantic expression of the logical reasoning process of the student model. The interest circle prediction results are plain text semantic expression units of a user's single interest topic in the predefined semantic space, used to determine the recommendation object.

[0009] The student model is trained in a phased manner: it is pre-trained by relying on the teacher model and a pre-set interest circle corpus to learn the semantic interpretation of interest circles, and then supervised training is completed based on user profile features, the inference chain output by the teacher model and the reference interest circle set. The objective function is constructed and maximized by combining multi-dimensional reward function, interest circle credit weight allocation, word level advantage value adaptive correction and parameter upper limit truncation constraint mechanism to complete the model reinforcement learning iterative optimization.

[0010] In one embodiment, the step of completing self-supervised pre-training based on a teacher model and a pre-set interest circle corpus to learn the semantic interpretation of interest circles includes:

[0011] Input the interest circle name, basic interest tags and target video text metadata into the teacher model, output the semantic explanation corresponding to each interest circle, and construct a corpus containing the interest circle and the semantic explanation;

[0012] Based on the interest circles and corresponding semantic interpretations contained in the corpus, the student model is pre-trained under self-supervised supervision to obtain the pre-trained student model.

[0013] In one embodiment, based on the interest circles and corresponding semantic interpretations contained in the corpus, the student model is self-supervised pre-trained to obtain a pre-trained student model, including:

[0014] The student model is then input into each interest circle in the corpus, triggering the student model to output the predicted semantic explanation corresponding to each interest circle.

[0015] Based on the predicted semantic interpretation output by the student model and the actual semantic interpretation corresponding to the interest circle in the corpus, the semantic loss is calculated to obtain the semantic loss result.

[0016] The parameters of the student model are iteratively updated based on the semantic loss result until the semantic loss meets the preset pre-training conditions, thus obtaining the pre-trained student model.

[0017] In one embodiment, the step of completing supervised training based on user profile features, the inference chain output by the teacher model, and the set of reference interest circles includes:

[0018] The user profile features, user historical behavior sequences, and the set of interest circles of actual user interactions are input into the teacher model to obtain the inference chain and the set of reference interest circles.

[0019] The pre-trained student model is trained under supervision based on the user profile features, the inference chain, and the set of reference interest circles to obtain the trained student model.

[0020] In one embodiment, the supervised training of the pre-trained student model based on the user profile features, the inference chain, and the reference interest circle set to obtain the trained student model includes:

[0021] The user profile features are input into the pre-trained student model, which triggers the student model to output the corresponding inference chain and interest circle prediction results.

[0022] The inference chain and interest circle prediction results output by the student model are compared with the inference chain and reference interest circle set output by the teacher model, respectively, and the comprehensive supervision loss is calculated.

[0023] The parameters of the pre-trained student model are iteratively optimized based on the comprehensive supervised loss until the comprehensive supervised loss meets the preset supervised training conditions, thus obtaining the trained student model.

[0024] In one embodiment, the step of constructing and maximizing the objective function by combining a multi-dimensional reward function, interest-based credit weight allocation, adaptive correction of word-level advantage values, and parameter upper limit truncation constraint mechanism to complete the model reinforcement learning iterative optimization includes:

[0025] Construct a target reward function based on the integrity of the inference chain, the accuracy of interest circle matching, and the effectiveness of recommendation;

[0026] The student model is given a multi-dimensional reward score based on the target reward function, and the model parameters in the student model are iteratively optimized based on the feedback of the reward score.

[0027] Based on the basic advantage value and credit score of each interest circle output by the student model, the advantage value is corrected, and an objective function is constructed by combining the upper limit truncation mechanism. The student model is then iteratively optimized by maximizing the objective function through reinforcement learning.

[0028] In one embodiment, the step of performing multi-dimensional reward scoring on the student model according to the target reward function includes:

[0029] Based on the training progress of the model training and the set of interest circles of real user interactions, a basic reward function is constructed and a basic reward value is obtained.

[0030] Calculate the gated reward value based on the preset gated factor and the base reward value;

[0031] The gated reward value is corrected based on a nonlinear efficiency penalty coefficient to obtain a target reward value; the target reward value is used to provide reward feedback to the trained student model.

[0032] In one embodiment, the set of interest circles based on the model training progress and real user interactions is used to construct a basic reward function and obtain a basic reward value, including:

[0033] Based on the hybrid exponential moving average algorithm, the current training progress of the student model is calculated, and based on the current training progress combined with the adaptive course learning mechanism, the semantic accuracy weight and grammatical format weight in the basic reward function are determined.

[0034] Based on the current training progress, the set of interest circles of real user interaction, the semantic accuracy weight, and the grammatical format weight, the basic reward value of the basic reward function is obtained.

[0035] In one embodiment, calculating the gated reward value based on the preset gated factor and the base reward value includes:

[0036] Calculate the values ​​of the preset validity gate factor, deduplication gate factor, and accuracy gate factor respectively;

[0037] The values ​​of the legality gate factor, the deduplication gate factor, and the accuracy gate factor are multiplied together to obtain the comprehensive gate coefficient;

[0038] The gating reward value is obtained based on the base reward value and the comprehensive gating coefficient.

[0039] In one embodiment, the step of correcting the gated reward value based on a nonlinear efficiency penalty coefficient to obtain a target reward value includes:

[0040] Among the optimal inference chains and interest circle prediction results output by the student model, the inference chain and interest circle prediction result with the shortest inference length are determined.

[0041] The dynamic efficiency baseline is calculated based on the minimum inference length hyperparameter, the inference chain with the shortest inference length, and the prediction results of the interest sphere.

[0042] Calculate the nonlinear efficiency penalty coefficient corresponding to the inference chain generated by the student model based on the dynamic efficiency baseline;

[0043] The target reward value is obtained by multiplying the gated reward value by the nonlinear efficiency penalty coefficient.

[0044] In one embodiment, the student model's basic advantage values ​​and credit scores for each interest circle are adjusted to correct the advantage values, and an objective function is constructed using an upper limit truncation mechanism. The student model is then iteratively optimized through reinforcement learning by maximizing the objective function, including:

[0045] Determine the basic advantage value of each interest circle in the set of interest circle prediction results output by the student model, and assign a credit score to each interest circle.

[0046] Based on the credit scores of each interest circle, a baseline credit statistic is calculated, and based on the baseline credit statistic, the basic advantage value corresponding to the prediction result of each interest circle is redistributed to obtain the corrected advantage value of each interest circle.

[0047] An upper limit truncation mechanism is adopted, and a reinforcement learning optimization objective function is constructed based on the corrected advantage values ​​of each interest zone. The student model is iteratively optimized by maximizing the objective function to obtain the reinforcement learning optimized student model.

[0048] The reinforcement learning-optimized student model is used to output inference chain and interest circle prediction results.

[0049] In one embodiment, the process of generating the interest circle set includes:

[0050] Obtain a set of basic interest tags, which contains multiple basic interest tags in a predefined semantic space;

[0051] Based on user behavior data, the basic interest tags in the basic interest tag set are initially aggregated to obtain an initial subset of interest tags;

[0052] Semantic cleaning and structural optimization are performed on each of the initial interest tag subsets, and the initial interest tag subsets are merged twice to obtain the merged interest tag subsets.

[0053] Semantic induction is performed on each of the interest tag subsets to generate the interest circle name corresponding to each of the interest tag subsets, thus obtaining the interest circle set.

[0054] In one embodiment, the preliminary aggregation of basic interest tags in the basic interest tag set based on user behavior data to obtain an initial subset of interest tags includes:

[0055] Based on user behavior data, the degree of collaborative association among each basic interest tag in the basic interest tag set is calculated, and a collaborative relationship network is constructed based on the degree of collaborative association.

[0056] By using a pre-defined clustering method, the basic interest tags in the collaborative relationship network are aggregated into multiple initial interest tag subsets.

[0057] According to a second aspect of the present disclosure, a recommended apparatus is provided, the apparatus comprising:

[0058] The first acquisition module is configured to acquire user profile features and user historical behavior sequences. The user profile features are used to characterize the statistical distribution features of each user's interactive behavior in the video platform at each level of interest circle under a predefined semantic space. The user historical behavior sequence is a sequence formed by mapping each user's interactive behavior to the predefined semantic space within a preset time period and sorting it by time.

[0059] The output module is configured to execute the student model, which has been trained, to analyze and process the user profile features and the user's historical behavior sequence to obtain the inference chain and interest circle prediction results; the inference chain is the explicit semantic expression of the logical reasoning process of the student model; the interest circle in the interest circle prediction results is the plain text semantic expression unit of a user's single interest topic in the predefined semantic space, which is used to determine the recommendation object;

[0060] The student model is trained in a phased manner: it is pre-trained by relying on the teacher model and a pre-set interest circle corpus to learn the semantic interpretation of interest circles, and then supervised training is completed based on user profile features, the inference chain output by the teacher model and the reference interest circle set. The objective function is constructed and maximized by combining multi-dimensional reward function, interest circle credit weight allocation, word level advantage value adaptive correction and parameter upper limit truncation constraint mechanism to complete the model reinforcement learning iterative optimization.

[0061] According to a third aspect of the present disclosure, an electronic device is provided, comprising:

[0062] processor;

[0063] Memory used to store the processor's executable instructions;

[0064] The processor is configured to execute the instructions to implement the recommended method described in the first aspect above.

[0065] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided such that, when instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is able to perform the recommended method described in the first aspect.

[0066] According to a fifth aspect of the present disclosure, a computer program product is provided, the computer program product including instructions that, when executed by a processor of an electronic device, enable the electronic device to perform the recommended method described in the first aspect.

[0067] The technical solutions provided by the embodiments of this disclosure have at least the following beneficial effects:

[0068] The trained student model analyzes user profile features and historical user behavior sequences, outputting an explicit semantic inference chain and interest circle prediction results as plaintext semantic expression units. Based on these interest circle prediction results, the recommended objects are determined. The student model sequentially completes self-supervised pre-training and supervised training, and combines a multi-dimensional reward function, interest circle credit weight allocation, adaptive correction of word-level dominance values, and parameter upper limit truncation constraints to construct and maximize the optimization objective function to complete reinforcement learning iterative optimization. This method effectively solves the problems of semantic black box, poor interpretability, and unstable prediction results in traditional methods. On the one hand, user modeling using interest circles in plaintext semantic form is adaptable to various business scenarios, transforming the traditional mode of directly predicting specific items into predicting stable plaintext semantic user interest circles, thus decoupling user intent planning from underlying item matching. On the other hand, the explicit inference chain provides clear logical attribution for the recommendation process. Simultaneously, multi-stage training is completed based on the teacher-student model architecture, and optimization methods such as credit weight allocation, dominance value correction, and truncation constraints further ensure the accuracy of interest circle prediction. By combining the mapping relationship between interest groups and candidate items for content distribution, the accuracy and content diversity of downstream recommendation processes can be effectively improved. Attached Figure Description

[0069] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure, and are not intended to unduly limit this disclosure.

[0070] Figure 1 This is a flowchart illustrating a recommended method according to an exemplary embodiment.

[0071] Figure 2 This is a flowchart illustrating the pre-training steps of a student model according to an exemplary embodiment.

[0072] Figure 3 This is a flowchart illustrating a self-supervised pre-training method for a student model according to an exemplary embodiment.

[0073] Figure 4 This is a flowchart illustrating a supervised training method for a student model according to an exemplary embodiment.

[0074] Figure 5 This is a flowchart illustrating a student model supervised training step based on user profile features and inference chain, according to an exemplary embodiment.

[0075] Figure 6 This is a flowchart illustrating a reward feedback method according to an exemplary embodiment.

[0076] Figure 7 This is a flowchart illustrating a method for determining a target reward value according to an exemplary embodiment.

[0077] Figure 8 This is a flowchart illustrating a step for obtaining a basic reward value according to an exemplary embodiment.

[0078] Figure 9 This is a flowchart illustrating a method for calculating a gating reward value according to an exemplary embodiment.

[0079] Figure 10 This is a flowchart illustrating a step for determining a target reward value according to an exemplary embodiment.

[0080] Figure 11 This is a flowchart illustrating a step for determining a student model optimized through reinforcement learning, according to an exemplary embodiment.

[0081] Figure 12 This is a flowchart illustrating a method for generating a set of interest circles according to an exemplary embodiment.

[0082] Figure 13 This is a flowchart illustrating a step for generating an initial subset of interest tags according to an exemplary embodiment.

[0083] Figure 14 This is a block diagram illustrating a recommended apparatus according to an exemplary embodiment.

[0084] Figure 15 This is a block diagram illustrating an electronic device according to an exemplary embodiment. Detailed Implementation

[0085] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0086] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0087] It should also be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for display, data used for analysis, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties.

[0088] The recommendation method provided in this application can be applied to various online recommendation scenarios such as short videos, e-commerce, and social media. The terminal and server interact via the network, with the terminal collecting historical user behavior data and user profile features within the application and uploading them to the server. The server pre-trains the student model through a full-process, phased process: first, self-supervised pre-training is conducted using the teacher model and a pre-set interest circle corpus; then, supervised training is completed by combining user profile features, the inference chain output by the teacher model, and a set of reference interest circles; finally, a multi-dimensional reward function, interest circle credit weight allocation, adaptive correction of word-level advantage values, and parameter upper limit truncation constraint mechanism are used to construct and maximize the optimization of the objective function, completing iterative optimization of the model's reinforcement learning. The server calls the trained student model to analyze and process the user data uploaded by the terminal, outputting an inference chain as an explicit semantic expression of the model's logical reasoning, and plaintext semantic units representing user interest topics, i.e., interest circle prediction results; subsequently, corresponding recommendation objects are matched based on the interest circle prediction results and sent to the terminal. The terminal receives the recommendation objects and displays them, ultimately realizing content recommendation for users.

[0089] The terminal can be, but is not limited to, electronic devices with data interaction and content display functions such as smartphones, tablets, laptops, desktop computers, smart TVs, and smart wearable devices. The server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. This application embodiment does not limit this.

[0090] Before describing the embodiments of this application, it should be noted that the terms "in response to" and "based on" are used herein to indicate corresponding user behavior, system events, or model execution conditions, such as a user opening an application, a page completing loading, a user generating historical behavior records, model inference being triggered, or training conditions being met. Subsequent actions are executed under the premise that the corresponding behavior occurs, the event occurs, or the preset conditions are met: for example, after a user launches a recommendation application, based on historical behavior data generated by the user, when model training conditions are met, or based on the mapping relationship between interest circles and candidate items. The timing of the subsequent actions executed in response to the behavior, event, or condition is as follows: for example, acquiring user profile features, collecting user historical behavior sequences, starting a student model for inference, outputting the inference chain and interest circle prediction results, and determining and displaying recommended items. It is worth noting that the time of execution of the subsequent actions and the time when the behavior occurs, the event occurs, or the condition is met are not necessarily strongly correlated. For example, in some cases, subsequent actions can be executed immediately upon the occurrence of the behavior or the fulfillment of the condition: for instance, after a user enters the recommendation page, the terminal immediately obtains the user profile features and historical behavior sequence of the current user; while in other cases, subsequent actions may be executed only after a short processing period following the occurrence of the behavior or the fulfillment of the condition: for instance, after the terminal inputs user data into the student model, the model needs to complete calculations such as interest extraction, psychological modeling, and intent inference before outputting the inference chain and interest circle prediction results. This process involves steps such as feature parsing, semantic reasoning, and circle matching, and requires a short processing period. Furthermore, "model inference triggering" refers to a process where the terminal automatically starts the local student model to perform analysis and calculations when the recommendation conditions are met. This process directly triggers the corresponding functional responses in the technical solution of this application, such as initiating inference based on user profile features and historical behavior sequences, outputting explicit inference chains, generating interest circle prediction results, and determining recommended items based on mapping relationships—core processes such as these.

[0091] Based on the above terminology and execution logic, the following embodiments will describe in detail the complete process of the terminal acquiring user data, initiating student model inference, outputting explicit inference chains and interest circle prediction results, and displaying recommended items based on interest circle matching, so as to clearly illustrate the implementation process and technical effects of the recommendation method of this application.

[0092] Figure 1 This is a flowchart illustrating a recommended method according to an exemplary embodiment, such as... Figure 1 As shown, this recommended method is applied to the terminal and includes the following steps.

[0093] In step S110, user profile features and user historical behavior sequences are obtained.

[0094] Among them, user profile features are used to characterize the statistical distribution characteristics of user interaction behaviors in the video platform at each level of interest circles under the predefined semantic space; user historical behavior sequence is a sequence formed by mapping user interaction behaviors to the predefined semantic space within a preset time period and sorting them by time.

[0095] Specifically, user profile features are used to depict users' long-term stable interests and preferences on video platforms. They are static distribution features obtained by summarizing and statistically analyzing users' historical interaction behaviors within a preset time period across multiple levels of interest circles in a predefined semantic space. Only statistical values ​​such as the proportion and frequency of each type of interest circle are retained, without retaining the chronological order of the behavior. User historical behavior sequences, on the other hand, are used to capture users' short-term interest drift. Using a preset time period as the filtering range, each user interaction behavior within the period is uniformly mapped to the corresponding interest circle in the predefined semantic space, and then strictly arranged in order according to the chronological order of the interaction to form an ordered sequence with time-series information. This sequence can reflect users' recent dynamic changes in temporary browsing preferences. The two complement each other from two dimensions: long-term static statistics and short-term dynamic time series, to complete the comprehensive characterization of user interests.

[0096] In implementation, the server collects and acquires the user profile features and historical behavior sequences of the target users. The user profile features consist of the statistical distribution of the user's historical interaction behavior on a predefined hierarchical interest circle system, accurately representing the user's long-term interest distribution. The user historical behavior sequence is a sequence formed by the user's positive interaction behaviors such as clicks, views, and favorites within a preset time range, ordered chronologically, directly reflecting the user's short-term interest drift. After acquiring the above two types of data, the server performs format standardization and data cleaning, mapping the user historical behavior sequence to the predefined interest circle system, transforming it into standardized behavior sequence data corresponding to the interest circle, thus preparing data for subsequent model analysis and processing.

[0097] In step S120, the trained student model analyzes and processes the user profile features and user historical behavior sequences to obtain the inference chain and interest circle prediction results.

[0098] Among them, the reasoning chain is the explicit semantic expression of the logical reasoning process of the student model; the interest circle in the interest circle prediction result is the plain text semantic expression unit of a user's single interest topic in a predefined semantic space, which is used to determine the recommended object.

[0099] In implementation, the server calls a pre-deployed, trained student model, inputting pre-processed user profile features and standardized user historical behavior sequences into the student model. The student model on the server has a built-in predefined closed semantic space and interest circle system, which can deeply integrate the long-term interest preferences reflected by user profile features and the short-term interest trends reflected by user historical behavior sequences. First, it extracts core interest features from the user historical behavior sequences, and then combines them with user profile features to complete the modeling of the user's deep psychological needs. Subsequently, it executes a complete reasoning process of "interest extraction, psychological modeling, and intent inference" through the built-in logical reasoning module. Finally, the student model on the server outputs the corresponding reasoning chain and interest circle prediction results. The reasoning chain is an explicit semantic expression of the model's logical reasoning process, which can clearly present the derivation logic from user behavior to interest intent. The interest circle prediction results are a set of plaintext interest circles that match the user's short-term and long-term interests. Based on the mapping relationship between the interest circle prediction results and the locally preset interest circles and candidate items, the server can quickly match the corresponding candidate items, providing a core basis for the subsequent distribution of recommended objects.

[0100] For example, the recommendation method in this application provides a hierarchical structured interest tagging system to organize the semantics of video content in an application or platform, supporting user interest modeling and content understanding. This hierarchical structured interest tagging system includes multiple levels of interest tags. For instance, it may include four levels of interest tags in descending order of content scope (Level 1 > Level 2 > Level 3 > Level 4). Level 1 interest tags describe the macro-level content domain, providing a top-level division of the overall interest space. Level 2 interest tags further subdivide the semantics within the Level 1 domain, depicting more specific content directions. Level 3 interest tags express relatively clear interest themes. Level 4 interest tags are fine-grained interest points, used to describe specific video content semantics or interest expressions.

[0101] The hierarchical structured interest tagging system is constructed using a two-way approach combining top-down and bottom-up methods. The first-level and some second-level interest classification frameworks ensure the semantic rationality and business usability of the overall system structure. The bottom-up part involves semantic understanding and clustering analysis of massive amounts of video content within the application or platform to extract numerous fine-grained interest points from the data, which are then progressively merged to form third- and fourth-level interest tags. In this way, the hierarchical structured interest tagging system possesses a clear semantic structure and can reflect the true distribution characteristics of content.

[0102] In one exemplary embodiment, the third-level interest tags are coarse-grained and cannot fully characterize a user's specific interests, while the fourth-level interest tags are numerous and have fine semantic granularity, making their direct use in user interest modeling prone to resulting in overly discrete interest expressions. Therefore, this application constructs an intermediate layer between the third-level and fourth-level interest tags in the hierarchical structured interest tagging system, serving as an interest circle. This interest circle acts as an interest unit for expressing user interest modeling and is used in the subsequent model training process, thereby enabling the student model to make accurate recommendations based on user interests. The generation process of this interest circle and the set of interest circles will be described in detail in the following embodiments, and will not be repeated here.

[0103] In an exemplary embodiment, the sequence of video identifiers (IDs) that a user has clicked or watched in the past month is mapped into plaintext interest circles with a hierarchical structure (interest units containing hierarchical paths). Then, the recommendation system inputs the converted plaintext interest circles, connected with user profile features, into a trained student model. The student model internally performs explicit inference, analyzes the user's deep intent (e.g., infers "want to learn shooting skills" from "watching NBA"), and finally outputs a list of text labels containing multiple predicted interest circles (interest circle prediction results).

[0104] Subsequently, the interest circle prediction results output by the student model are used as high-dimensional control signals to input into the downstream recommendation engine. The downstream recommendation engine adds an interest recall channel in the recall stage, increases the weight of videos that hit the interest circle in the ranking stage, and inserts videos corresponding to novel interests in the diversity control stage to complete the distribution and recommendation of short video content and candidate recommended items (i.e. recommended objects).

[0105] Specifically, the final output of the student model (interest circle prediction result) can be a list of text labels or a high-dimensional control signal with key information, which is used by the downstream recommendation engine to perform specific filtering and ranking operations, including but not limited to the following applications:

[0106] During the recall phase, a recall channel based on user interests can be added. Using the interest circles output by the student model as search criteria, a search operation is performed in the inverted index or vector index of the video library to directly obtain candidate videos belonging to the corresponding interest circles, thereby improving the richness and coverage of the recall results (candidate content).

[0107] During the ranking phase, when scoring and ranking candidate videos, it is determined whether the tags carried by the video match the predicted interest circles output by the student model. If they match, a weight enhancement process is applied to the estimated score of the video, so that videos that match the user's deep interests rank higher in the final recommendation list, thereby improving the matching degree between the recommendation results and the user's interests.

[0108] During the diversity control phase, based on the novel interest circles that users have not interacted with in their history, as output by the student model, videos of the corresponding categories are inserted into the recommendation list at a preset ratio. This proactively expands the boundaries of users' interests, effectively breaks the information cocoon, and enhances the diversity and exploratory nature of the recommended content.

[0109] In the aforementioned recommendation method, a trained student model analyzes user profile features and historical user behavior sequences, outputting an explicit semantic inference chain and an interest circle prediction result as a plaintext semantic expression unit. Based on this interest circle prediction result, the recommended object is determined. The student model sequentially completes self-supervised pre-training and supervised training, and combines a multi-dimensional reward function, interest circle credit weight allocation, adaptive correction of word-level advantage values, and parameter upper limit truncation constraints to construct and maximize the optimization objective function to complete reinforcement learning iterative optimization. This method effectively solves the problems of semantic black box, poor interpretability, and unstable prediction results in traditional methods. On the one hand, user modeling using interest circles in plaintext semantic form is adaptable to various business scenarios, transforming the traditional direct prediction of specific items into prediction of stable plaintext semantic user interest circles, thus decoupling user intent planning from underlying item matching. On the other hand, the explicit inference chain provides clear logical attribution for the recommendation process. Simultaneously, multi-stage training is completed based on the teacher-student model architecture, and optimization methods such as credit weight allocation, advantage value correction, and truncation constraints further ensure the accuracy of interest circle prediction. By combining the mapping relationship between interest groups and candidate items for content distribution, the accuracy and content diversity of downstream recommendation processes can be effectively improved.

[0110] In one embodiment, the student model is obtained through a phased, progressive training process:

[0111] Step 1: Based on the teacher model and the pre-set interest circle corpus, complete the self-supervised pre-training and learn the semantic interpretation of interest circles;

[0112] Step 2: Supervised training is then completed based on user profile features, the inference chain output by the teacher model, and the set of reference interest circles.

[0113] Step 3: Combine multi-dimensional reward function, interest circle credit weight allocation, word-level advantage value adaptive correction and parameter upper limit truncation constraint mechanism to construct and maximize the optimization of objective function, and complete the model reinforcement learning iterative optimization.

[0114] In one exemplary embodiment, such as Figure 2 As shown, the student model undergoes self-supervised pre-training based on the teacher model and a pre-set interest circle corpus during training. The steps involved in learning semantic interpretations of interest circles include:

[0115] Step 201: Input the interest circle name, basic interest tags and target video text metadata into the teacher model, output the semantic explanation corresponding to each interest circle, and construct a corpus containing interest circles and semantic explanations.

[0116] In implementation, a semantic space is predefined in the server. This semantic space is represented as a set of interest circles, specifically in the following form: (N 27000), of which Representing interest groups, each This can be, but is not limited to, a hierarchical path with a depth of L=4, such as sports → ball games → basketball → NBA. During the construction of the interest-based social circle system, the name of each interest circle is extracted.

[0117] This interest-based hierarchical system bridges the gap between semantics and behavior. Furthermore, in this embodiment, posterior item-item correlations can be used as implicit collaborative signals to guide the aggregation process of displayed semantic content. This strategy compresses the massive item tag space into a compact, behavior-aligned interest-based hierarchical system, thereby ensuring that the inference unit is not only semantically interpretable but also strictly consistent with the user's underlying actual behavioral patterns.

[0118] Subsequently, the server inputs the names of interest circles, hierarchical basic interest tags, and textual metadata of the target video (e.g., OCR (Optical Character Recognition), ASR (Automatic Speech Recognition), etc.) into the teacher model, which serves as the large language model. The teacher model then performs semantic parsing and summarization on each interest circle, generating clear, standardized plaintext semantic explanations that can be learned by the student model. Then, each interest circle is paired and stored with its corresponding semantic explanation to form a closed semantic corpus for domain knowledge injection. Specifically, this corpus contains multiple pairs of corpora consisting of interest circles and their corresponding semantic explanations. The specific form of these corpora is as follows: = .

[0119] Step 202: Based on the interest circles and corresponding semantic interpretations contained in the corpus, the student model is pre-trained in a self-supervised manner to obtain the pre-trained student model.

[0120] In implementation, the server uses all or part of the corpus pairs as training data and employs continuous pre-training to perform self-supervised learning on the student model. Specifically, the student model is driven to predict and generate corresponding semantic interpretations by giving the names of interest circles. Then, the model parameters are continuously updated iteratively by calculating the semantic loss between the predicted semantic interpretations output by the student model and the true semantic interpretations (output by the teacher model). This allows the student model to fully learn the plaintext semantic knowledge of the interest circle system, complete domain adaptation and semantic injection, and obtain a pre-trained student model with basic semantic understanding capabilities.

[0121] In this embodiment, a closed semantic corpus is constructed based on the teacher model, combined with hierarchical interest circles, basic interest tags, and video text metadata. This compresses the massive tag space into an interest circle system aligned with user behavior, effectively bridging the gap between semantic expression and actual user behavior, and ensuring that the inference unit has both semantic interpretability and behavioral matching. Then, the corpus is used to conduct self-supervised pre-training of the student model. The model parameters are iteratively optimized through semantic loss, and domain knowledge and semantic information are injected. This allows the student model to fully grasp the semantic logic of each interest circle, solidify the model's semantic understanding ability, and lay a solid foundation for subsequent supervised training, reinforcement learning, and online inference prediction.

[0122] In one exemplary embodiment, such as Figure 3 As shown, in step S202, based on the interest circles and corresponding semantic interpretations contained in the corpus, the student model is self-supervised pre-trained to obtain the pre-trained student model. This can be achieved through the following steps:

[0123] In step S211, each interest circle in the corpus is input into the student model, triggering the student model to output the predicted semantic interpretation corresponding to each interest circle.

[0124] In practice, the server takes each interest circle in the constructed corpus as input data and feeds it into the student model to be trained in sequence. This drives the student model to perform semantic understanding and generation of the input interest circles and output the predicted semantic interpretation corresponding to each interest circle, thus completing the forward inference process of self-supervised pre-training.

[0125] Specifically, the server takes specific interest circles from the constructed domain definition corpus, such as hierarchical interest circles like "sports-ball sports-basketball-NBA games" and "food preparation-Chinese cooking-home-style dishes-scrambled eggs with tomatoes," as input data and feeds them sequentially into the student model to be trained. This drives the student model to perform semantic parsing, feature extraction, and text generation on the aforementioned interest circles, and outputs a predicted semantic interpretation that corresponds one-to-one with each interest circle. For example, for the interest circle "basketball-NBA games," it generates "This interest circle is used to describe users' preferences for professional shooting skills, training methods, and practical applications of NBA professional games." This completes the forward inference process for plaintext semantic understanding of interest circles during the self-supervised pre-training process.

[0126] In step S212, based on the predicted semantic interpretation output by the student model and the actual semantic interpretation corresponding to the interest circles in the corpus, the semantic loss is calculated to obtain the semantic loss result.

[0127] In practice, the server compares the predicted semantic interpretations output by the student model with the actual semantic interpretations corresponding to the interest circles in the corpus, and uses a preset semantic loss function to calculate the semantic difference between the two, thereby obtaining the semantic loss result that characterizes the degree of prediction bias of the model.

[0128] Specifically, the server uses LoRA (Low-Rank Adaptation) technology to perform self-supervised continuous pre-training on the student model. The [c_i] in the Prompt template "Define the user interest circle:[c_i].Explanation:" is replaced with the specific interest circle name and input into the student model to obtain the predicted semantic explanation output by the student model. The server then compares the predicted semantic explanation with the actual semantic explanation pre-generated by the teacher model in the corpus sentence by sentence. The differences between the two in terms of semantic expression, content accuracy, and structural regularity are calculated through a preset semantic loss function. For example, the preset semantic loss function is the cross-entropy loss function. Then, by minimizing the cross-entropy loss, the difference between the content generated by the model and the real semantic explanation is measured, and finally, the semantic loss result used to measure the degree of model prediction bias is obtained.

[0129] In step S213, the parameters of the student model are iteratively updated based on the semantic loss results until the semantic loss meets the preset pre-training conditions, thus obtaining the pre-trained student model.

[0130] In practice, the server updates and iteratively optimizes the parameters of the student model using the backpropagation algorithm based on the semantic loss results, continuously reducing the gap between the model's prediction results and the actual results. The above process is repeated until the semantic loss converges to a preset threshold or meets the preset training stopping condition, and finally a pre-trained student model with the ability to understand the semantics of interest circles is obtained.

[0131] In this embodiment, the server uses a self-supervised pre-training process to enable the student model to fully learn and master the semantic knowledge and expression norms of interest circles, providing a semantically aligned and performance-stable foundation model for subsequent supervised training based on the teacher model.

[0132] In one embodiment, such as Figure 4 As shown, the specific process of supervising training in step 2, based on user profile features, the inference chain output by the teacher model, and the set of reference interest circles, includes:

[0133] Step 401: Input the user profile features, user historical behavior sequence, and user real interaction interest circle set into the teacher model to obtain the inference chain and reference interest circle set.

[0134] In implementation, the server inputs user profile features representing long-term user preferences, user historical behavior sequences representing short-term behavior, and a set of real user interaction interest circles as the posterior standard answer into the teacher model. This drives the teacher model to perform reverse engineering reasoning, generating a structured explicit reasoning chain according to the fixed logic of interest extraction, mental modeling, and intent inference, and outputting a high-quality set of reference interest circles as the standard answer for supervised training of the student model.

[0135] For example, user profile features ( ): This represents the long-term distribution of a user's interests. In practice, this feature is composed of the statistical distribution of a user's historical interactions on a hierarchical interest circle system (for example, the first-level category "sports" accounts for 30%, and the second-level category "sports-basketball" accounts for 15%).

[0136] User historical behavior sequence ( ): This indicates a user's short-term interest drift. In practice, this sequence is a list of the user's most recent N positive interaction interest circles arranged in chronological order.

[0137] The set of interest circles for users' future real interactions ( ): This serves as a supervisory signal (GroundTruth) for "hindsight." This data is only used as input when the teacher model generates the inference chain, guiding the teacher model to find logical paths in reverse. It is a set of interest circles for users' future real positive interactions.

[0138] Step 402: Supervised training is performed on the pre-trained student model based on user profile features, inference chain, and reference interest circle set to obtain the trained student model.

[0139] In implementation, the server takes user profile features as input and the inference chain output by the teacher model and the set of reference interest circles as supervision labels to conduct supervised fine-tuning on the pre-trained student model. This allows the student model to learn the complete inference path from user behavior data to interest prediction. By minimizing the comprehensive supervision loss, the model parameters are continuously optimized so that it can autonomously generate inference chains and accurate interest circle prediction results that are consistent with the logic of the teacher model. Finally, a trained student model that can be directly used for online recommendation is obtained.

[0140] In this embodiment, the student model completes the entire construction from domain knowledge learning to explicit reasoning ability through a two-stage training process under the guidance of the teacher model. This effectively solves the problems of semantic black box, weak interpretability, and non-standard reasoning logic in traditional recommendation models, while ensuring that the output interest circles and reasoning chains strictly conform to the closed semantic space and business standards.

[0141] In one exemplary embodiment, such as Figure 5 As shown, the specific processing steps of step 402 include:

[0142] Step 501: Input the user profile features into the pre-trained student model, triggering the student model to output the corresponding inference chain and interest circle prediction results.

[0143] In implementation, the server inputs user profile features representing the long-term interest distribution of users into the student model that has completed self-supervised pre-training. The student model is driven to perform forward reasoning according to the structured reasoning logic of interest extraction → mental modeling → intent inference. It automatically generates reasoning chains containing explicit derivation logic and interest circle prediction results that match user preferences, realizing complete generative reasoning from user features to interest intent.

[0144] Specifically, the server inputs user profile features representing the long-term distribution of user interests and user historical behavior sequences representing short-term interest drift into a self-supervised pre-trained student model. This drives the student model to perform forward generation according to a fixed structured backward reasoning logic of interest extraction → mental modeling → intent inference. Specifically: First, it extracts historical behavior patterns that are highly correlated with future preferences from user profile features and user historical behavior sequences, completing the interest extraction step; second, it maps the extracted behavior patterns to the user's corresponding deep psychological needs such as skill acquisition, emotional resonance, and leisure and entertainment, completing the mental modeling step; finally, based on the mental modeling results, it locates the most matching set of interest circles in a predefined interest circle system, while taking into account both prediction accuracy and diversity, and finally automatically outputs an explicit reasoning chain containing complete three-step derivation logic, as well as interest circle prediction results that highly match the user's true preferences, realizing a complete generative explicit reasoning process from user feature data to user interest intent.

[0145] Step 502: Compare the inference chain and interest circle prediction results output by the student model with the inference chain and reference interest circle set output by the teacher model, respectively, and calculate the comprehensive supervision loss.

[0146] In practice, the server compares the inference chain and interest circle prediction results generated by the student model with the standard inference chain and reference interest circle set obtained by the teacher model through reverse engineering, level by level and sequence by sequence. Based on the sequence cross-entropy loss function, the inference chain generation loss and interest circle prediction loss are calculated respectively, and the two types of losses are weighted and fused to obtain the comprehensive supervision loss used to measure the overall bias of the model.

[0147] Step 503: Iteratively optimize the parameters of the pre-trained student model based on the comprehensive supervised loss until the comprehensive supervised loss meets the preset supervised training conditions, and obtain the trained student model.

[0148] In implementation, the server adopts LoRA low-rank adaptation technology. Based on the comprehensive supervised loss, the parameters of the pre-trained student model are updated and iteratively optimized through the backpropagation algorithm. The gap between the student model output and the teacher model supervision signal is continuously narrowed. The input, inference, comparison and update process is repeated until the comprehensive supervised loss converges to the preset threshold or reaches the maximum number of iterations. Finally, a trained student model that can be deployed to the terminal for online recommendation is obtained.

[0149] In this embodiment, the server uses a supervised fine-tuning process with the teacher model output as the supervision signal to enable the student model to fully learn the mapping relationship from user profile features to explicit inference chains and precise interest circles, ensuring that the inference logic is standardized and the prediction results are accurate, while maintaining lightweight efficiency and meeting the needs of terminal inference and industrial-grade deployment.

[0150] In one exemplary embodiment, such as Figure 6 As shown, the specific processing steps in step 3 include:

[0151] Step 601: Construct a target reward function based on the integrity of the inference chain, the accuracy of interest circle matching, and the effectiveness of recommendation.

[0152] In implementation, the server constructs a target reward function for reinforcement learning based on three core dimensions surrounding the student model's output: inference chain integrity, interest circle matching accuracy, and recommendation effectiveness. Inference chain integrity assesses the coherence and completeness of the explicit semantic inference chain generated by the model, ensuring the traceability of the model's inference process. Interest circle matching accuracy measures the degree of fit between the model's predicted interest circles and the user's actual interests and the pre-defined semantic space of closed interest circles, avoiding interest prediction bias and semantic boundary violations. Recommendation effectiveness comprehensively considers the diversity and novelty of interest circles, as well as their actual benefit to downstream content distribution. By weighting and fusing these three evaluation indicators, a multi-dimensional coupled target reward function is formed, serving as the core basis for subsequent model reward scoring and optimization guidance, thus constraining the quality of model generation from multiple perspectives.

[0153] Step 602: Perform multi-dimensional reward scoring on the student model according to the target reward function, and iteratively optimize the model parameters in the student model based on the reward score feedback.

[0154] In implementation, the server invokes the constructed target reward function to comprehensively quantify and score the inference chain and interest circle prediction results of the student model's single inference output, obtaining a corresponding multi-dimensional reward score. This score directly reflects the student model's overall performance in inference logic, interest prediction, and recommendation effectiveness. Subsequently, the server system uses the reward score as a feedback signal to introduce it into the reinforcement learning process, forming positive incentives or negative constraints based on the score, and iteratively optimizing the student model's network parameters using the backpropagation algorithm. Through continuous score feedback and parameter updates, the model's generation strategy is constantly corrected, guiding the student model to gradually improve the standardization of the inference chain and the accuracy of interest circle prediction.

[0155] Step 603: Based on the basic advantage value and credit score of each interest circle output by the student model, the advantage value is corrected, and an objective function is constructed by combining the upper limit truncation mechanism. The student model is then iteratively optimized by maximizing the objective function through reinforcement learning.

[0156] In implementation, after incentivizing the student model through the objective reward function, the server extracts the basic advantage value corresponding to each interest circle in the prediction results of each interest circle output by the student model. Simultaneously, considering dimensions such as whether the interest circle is legitimate, repetitive, matches the user's true interests, and belongs to exploratory content, a differentiated credit score is assigned to each interest circle. Then, based on the credit score, a baseline credit statistic is calculated, and this baseline credit statistic is used to redistribute and correct the basic advantage value at the word level, distinguishing the weights of different output types such as valid interests, repetitive content, and semantic illusions. On this basis, the server introduces an upper limit truncation mechanism to constrain the parameter range, avoiding parameter overflow and optimization runaway issues during training, thus constructing the final reinforcement learning optimization objective function. The server system continuously iterates and trains the student model with reinforcement learning, aiming to maximize this objective function, constantly strengthening the model's ability to generate high-quality inference chains and accurate interest circles, ultimately completing the deep optimization of the model after supervised training.

[0157] Specifically, the basic advantage value is calculated based on the group relative strategy optimization algorithm. That is, for a set of inference chains and interest circle prediction results generated by the student model in a single instance, the difference between the quality of a single output and the mean within the group is calculated by combining the comprehensive score distribution of the entire group output. This yields the basic advantage value of the response level, which is used to characterize the overall quality of a single output result. Since the initial basic advantage value is shared by the entire output and cannot distinguish the contribution differences of different interest circles and words within the group, fine-grained correction is subsequently completed by combining the credit scores of each interest circle.

[0158] The baseline credit statistic is a benchmark value obtained by calculating the arithmetic mean of the credit scores of all interest circles in the prediction results of a single interest circle in the student model. After calculating the baseline credit statistic, the credit score of a single interest circle can be compared with it to determine the output quality of each interest circle. Based on the comparison results, the basic advantage value is redistributed and corrected at the word level. The weight of effective interest circles is increased and the weight of invalid content such as duplicate content and semantic illusions is reduced, so as to achieve fine-grained optimization of model training.

[0159] Upper bound truncation is a parameter constraint strategy applied to the optimization process of reinforcement learning. This application employs an asymmetric upper bound truncation method to limit the numerical range of importance sampling ratios and advantage values ​​during model optimization. This prevents parameters from amplifying without limit or fluctuating abnormally, avoiding problems such as uncontrolled training gradients and deviations in optimization direction. This mechanism sets reasonable upper and lower bounds, preserving the positive optimization space corresponding to high-quality outputs and encouraging effective model exploration while constraining the training risks caused by extreme values. Combined with corrected word-level advantage values, it constructs the optimization objective function, ensuring a smooth and controllable convergence of the student model's iteration process. Ultimately, this improves the stability and compliance of the model's generated inference chain and interest layer prediction results.

[0160] In this embodiment, by constructing a target reward function around the integrity of the inference chain, the accuracy of interest circle matching, and the effectiveness of recommendation, the quality of model output can be comprehensively measured from multiple dimensions. Then, by relying on the reward score to iteratively optimize the model parameters, the model's generation logic and prediction ability can be continuously corrected. At the same time, the advantage value is corrected by combining the basic advantage value of interest circle and credit score, and the optimization objective function is constructed with an upper limit truncation mechanism to carry out reinforcement learning iteration. This achieves fine-grained strategy optimization at the word level, efficiently distinguishes effective interests, repetitive content, and semantic illusions, avoids abnormal fluctuations in training parameters, improves the logical integrity of the inference chain and the accuracy of interest circle prediction, and ensures that the training process is stable and controllable.

[0161] In one exemplary embodiment, such as Figure 7 As shown, the specific processing steps of step 602 include:

[0162] Step 701: Based on the training progress of the model training and the set of interest circles of real user interactions, construct a basic reward function and obtain the basic reward value.

[0163] In implementation, the server dynamically adjusts the semantic accuracy weight and grammatical format weight based on the current reinforcement learning training progress of the student model using a course learning mechanism. Taking the set of interest circles generated by real user interactions as a reference standard, a multi-dimensional basic reward function is constructed, which includes hierarchical semantic accuracy rewards, multi-level normalized diversity rewards, exploratory / novelty rewards, and structural compliance rewards. The interest circle set generated by the student model is comprehensively scored to obtain the corresponding basic reward value.

[0164] Step 702: Calculate the gate reward value based on the preset gate factor and basic reward value.

[0165] In implementation, the server applies weighted constraints to the basic reward value based on three types of multiplicative constraint gate factors: legality gate, deduplication gate, and accuracy gate threshold. This process removes illusory content generated by the student model that exceeds the closed interest circle system, duplicate and redundant content, and invalid results that do not meet the accuracy requirements. The constrained result is then multiplied by the basic reward value to obtain a gated reward value that conforms to business specifications.

[0166] Step 703: Correct the gating reward value based on the nonlinear efficiency penalty coefficient to obtain the target reward value; the target reward value is used to provide reward feedback to the trained student model.

[0167] In implementation, the server dynamically determines the optimal inference length baseline based on the group-level minimum sufficient inference constraint mechanism. It calculates a nonlinear efficiency penalty coefficient based on the relative length excess of the inference chain generated by the student model and the difference in prediction performance. This nonlinear efficiency penalty coefficient is used to correct the gating reward value, suppressing redundant and inefficient inference, and obtaining a target reward value that balances accuracy, diversity, compliance and inference efficiency. This target reward value is used for reinforcement learning reward feedback for the student model to perform token-level adaptive strategy optimization.

[0168] In this embodiment, the server uses a complete reinforcement learning reward process, including multi-dimensional reward construction, gating constraints, and efficiency penalty correction, to ensure that the interest circles and inference chains generated by the student model are strictly aligned with the business objectives. This effectively suppresses illusions, repetitive and redundant inference, and significantly improves the accuracy, diversity, and online inference efficiency of the model's generated results within a closed semantic space.

[0169] In one exemplary embodiment, such as Figure 8 As shown, the specific process of constructing the basic reward function and obtaining the basic reward value based on the training progress of the model training and the set of interest circles of real user interaction in step 701 above includes:

[0170] Step 801: Calculate the current training progress of the student model based on the hybrid exponential moving average algorithm, and determine the semantic accuracy weight and grammatical format weight in the basic reward function based on the current training progress and the adaptive course learning mechanism.

[0171] In implementation, the server uses an Exponential Moving Average (EMA) algorithm with mixed accuracy to statistically analyze the output of the student model, calculate the current reinforcement learning training progress, and dynamically allocate weights based on this progress through an adaptive curriculum learning mechanism. In the early stages of training, a higher proportion of grammatical format weights is assigned to force the model to learn the correct output format. As training progresses, the semantic accuracy weight is gradually increased to guide the model to focus on the accuracy of interest-based predictions. That is, in the early stages of training, the grammatical format weight is higher than the semantic accuracy weight. This forces student models to learn a compliant output format; as training progresses, the semantic accuracy weights... Gradually increase the weight of semantic accuracy over syntactic format, guiding student models to focus on the semantic accuracy of the output.

[0172] Step 802: Based on the current training progress, the set of interest circles of real user interactions, semantic accuracy weights, and grammatical format weights, obtain the basic reward value of the basic reward function.

[0173] In implementation, the server uses the set of interest circles of real user interaction as a reference standard to calculate the hierarchical semantic accuracy reward, multi-level normalized diversity reward, exploratory novelty reward and structural compliance reward respectively. The semantically related reward is weighted with the semantic accuracy weight, and the structurally related reward is weighted with the syntax format weight and then summed to obtain the basic reward value used to measure the quality of the model output.

[0174] Specifically, define the basic reward function. The basic reward function is a weighted sum of semantic and structural rewards:

[0175] + (1)

[0176] in, For the training progress, as well as These are semantic accuracy weight and syntactic format weight, respectively.

[0177] The basic reward function includes: hierarchical semantic accuracy reward ( Multi-level normalized diversity rewards ) and exploratory / novelty rewards ( ),in:

[0178] Hierarchical semantic accuracy reward ( It not only evaluates whether the prediction result is accurate, but also calculates the prediction path. With the real path The degree of overlap at the level of the interest circle system is used to achieve semantic alignment from coarse to fine granular, preventing overly strict accuracy rewards from causing sparse accuracy reward signals.

[0179] (2)

[0180] in, Predicted interest circles representing the output of the student model Belongs to the set of predicted interest circles (A set of interest circle prediction results) This is a collection of interest groups for users' future real-world interactions. Representing actual interest circles A collection of interest groups that represent users' future real interactions. , For hierarchical weights, This characterizes the degree of overlap between predicted and actual paths within interest circles. The number of interest circle prediction results (output results) output for the student model.

[0181] Multi-level normalized diversified rewards ): Calculate the normalized entropy values ​​of the generated interest circle prediction results at the first and second levels of the interest circle system (e.g., interest circle depth L=4). Encourage students to generate more diverse interest circles in their models, and promote the exploration of interest diversity:

[0182] (3)

[0183] (4)

[0184] in, The normalized entropy value, The original information entropy measures the set of interest circles. The greater the entropy, the more uniform the distribution of the generated interest circles and the stronger their diversity. A collection of interest groups Non-repeating elements (interest circles after deduplication). The number of elements after deduplication, i.e., the number of different interest categories. This is the maximum possible entropy (the entropy value when all categories are equally probable), used for normalization.

[0185] : Interest circle prediction results (interest circle set) generated by the student model. :from The first-level interest tag set extracted from; :from The second-level interest tag set extracted from it.

[0186] Exploratory / novelty rewards ( ): Rewards are given for accurately matching real interactions, but not appearing in the user's recent history of interest circles. The interest circles within the model are designed to foster students' ability to explore and break through existing interest circles.

[0187] (5)

[0188] in, This represents the predicted interest circle set output by the student model. A collection of interest circles for users' future real interactions (a pre-built interest circle system). The output of the student model should include valid interest circles belonging to the legal system. This is a collection of interest groups that the user has recently interacted with. This refers to the new interest circles remaining after excluding users' historical interests from the existing effective interest circles.

[0189] Structural rewards include: format compliance rewards ( ) and length efficiency bonus ( ).

[0190] By substituting specific values ​​into the above basic reward function formula, we can calculate the basic reward value used to measure the quality of the model output at the current training progress.

[0191] In this embodiment, the server guides the student model from format compliance to semantic accuracy in stages through a course learning method that uses training progress awareness and dynamic weight allocation, providing a stable and reliable basic reward signal for subsequent reinforcement learning reward calculation.

[0192] In one exemplary embodiment, such as Figure 9 As shown, the specific processing steps of step 702 include:

[0193] Step 901: Calculate the values ​​of the preset validity gate factor, deduplication gate factor, and accuracy gate factor respectively.

[0194] In implementation, the server calculates the values ​​of three types of gating factors for the set of predicted interest circles (interest circle prediction results) output by the student model: First, it calculates the proportion of valid content belonging to the predefined closed interest circle system in the output results to obtain the legitimacy gating factor; second, it detects and penalizes duplicate interest circle items to obtain the deduplication gating factor; and third, it matches the predicted interest circles with the user's actual interactive interest circles to determine whether the preset accuracy threshold is reached to obtain the accuracy gating factor.

[0195] Specifically, legitimacy gating factor ( ): Calculate the proportion of the predicted interest circles (i.e., the predicted interest circle set) output by the student model that belongs to the pre-built interest circle system (the set of interest circles for the user's future actual interactions). If the student model generates a large number of illusions (i.e., most of the interest circles in the output interest circle prediction results are not within the interest circle system), the legality gating factor coefficient will be reduced to a minimum (preset minimum value) to significantly reduce the total reward.

[0196] Deduplication gating factor ( ): Displays penalties for duplicate items, forcing the student model to generate a unique set of interest tags, thus eliminating duplicates.

[0197] Accuracy Gating Factor (max( ): Set accuracy threshold =0.2. This accuracy gating factor acts as a hard constraint, ensuring that only the base hit rate... Only when certain standards are met can student models receive higher rewards. This means that high prediction accuracy is a strict prerequisite for obtaining high returns, preventing reward hacking.

[0198] Step 902: Multiply the values ​​of the validity gating factor, the deduplication gating factor, and the accuracy gating factor to obtain the comprehensive gating coefficient.

[0199] In implementation, the server multiplies the calculated validity gating factor, deduplication gating factor, and accuracy gating factor element-wise. Through a multiplication constraint gating mechanism, these three constraints are merged into a comprehensive gating coefficient, which is used to constrain and correct the base reward value. Specifically, the comprehensive gating coefficient is expressed as follows: .

[0200] Step 903: Based on the base reward value and the comprehensive gating coefficient, obtain the gating reward value.

[0201] In implementation, the server multiplies the calculated base reward value with the comprehensive gating coefficient, and uses the comprehensive gating coefficient to apply weighted constraints to the base reward value, eliminating abnormal rewards caused by unqualified generated results such as illusions, repetitions, and low precision, and finally obtaining a stable, compliant gating reward value that is aligned with business objectives.

[0202] Specifically, the formula for calculating the gating reward value is as follows:

[0203] (6)

[0204] In this embodiment, the server strictly ensures that the model-generated content meets industrial-grade requirements in terms of the legality of the interest system, the uniqueness of the output, and the accuracy of the prediction through multi-level multiplication gating constraints. This effectively suppresses reward speculation and invalid generation, providing reliable reward signals for subsequent reinforcement learning optimization.

[0205] In one exemplary embodiment, such as Figure 10 As shown, in order to suppress verbose inference by student models due to insufficient marginal information gain and to optimize online inference efficiency, this embodiment does not employ a static length penalty. Instead, it dynamically determines the efficiency baseline based on the overall performance of each interest layer prediction result group (the output results of each group) and designs a nonlinear penalty mechanism based on relative performance differences. The specific processing steps of step 703 include:

[0206] Step 1001: Among the optimal inference chains and interest circle prediction results output by the student model, determine the inference chain and interest circle prediction result with the shortest inference length.

[0207] In implementation, the server iterates through all inference chains and interest circle prediction results generated by the student model in the current batch that meet the business specifications, counts the number of tokens contained in each inference chain, and selects the inference chain and interest circle prediction results with the fewest tokens and the most concise expression as the benchmark sample for efficient inference.

[0208] Step 1002: Calculate the dynamic efficiency baseline based on the minimum inference length hyperparameter, the inference chain with the shortest inference length, and the prediction results of the interest sphere.

[0209] In implementation, the server uses the number of tokens in the sample with the shortest inference chain as the core reference, combined with a preset minimum necessary inference length hyperparameter. Dynamic adjustments are made, and the weighted sum of the two is used as the dynamic efficiency baseline for the current batch. This dynamic efficiency baseline ensures that the inference chain has complete expressive power and can also serve as an objective standard for measuring the degree of redundancy.

[0210] Specifically, dynamic efficiency baseline The calculation formula is as follows:

[0211] (7)

[0212] in, The prediction results for each interest circle in the student model output represent the group of prediction results. The first output for the student model Group interest circle prediction results. For the first The token length of the inference chain for each response sample. This indicates that the interest circle prediction results are based on real tags. Number of interest groups hit.

[0213] Step 1003: Calculate the nonlinear efficiency penalty coefficient corresponding to the inference chain generated by the student model based on the dynamic efficiency baseline.

[0214] In implementation, the server calculates the difference between the actual length of each generated inference chain and the dynamic efficiency baseline. This difference is then input into a preset nonlinear penalty function (such as an exponential penalty or a piecewise penalty function) to obtain the corresponding nonlinear efficiency penalty coefficient. A stronger penalty is imposed on redundant inference that exceeds the baseline, while a lower penalty is maintained for efficient inference that is close to the baseline.

[0215] Step 1004: Multiply the gated reward value by the nonlinear efficiency penalty coefficient to obtain the target reward value.

[0216] In practice, the server multiplies the gating reward value that has completed multi-level gating constraints with the non-linear efficiency penalty coefficient of the corresponding inference chain sample by sample. While preserving compliance and quality constraints, it further suppresses lengthy and inefficient inference behavior, and finally obtains the target reward value that takes into account accuracy, diversity, compliance and inference efficiency, which is used for gradient updates in reinforcement learning.

[0217] Specifically, the nonlinear efficiency penalty coefficient The construction process includes: calculating the relative length excess. Let the tolerance be... ,but:

[0218] (8)

[0219] To penalize inefficient (excessively long inference) inference chains with poor output accuracy in the model, embodiments of this application base the penalty on the performance gap between any output result and the optimal result: Dynamically modulate the penalty decay amplitude:

[0220] (9)

[0221] In the formula for calculating the nonlinear efficiency penalty coefficient, To provide a safe lower bound for the reward and prevent the reward value from collapsing; squared terms. A nonlinear penalty mechanism is introduced, meaning the system tolerates small length fluctuations but imposes exponential penalties on significant redundancy; scaling factor It plays a crucial moderating role: the penalty is lighter when the model's predicted interest sphere set is accurate; and it is lighter when the output is incorrect and the inference is lengthy. The value increases, thus amplifying the severity of the punishment.

[0222] Thus, after obtaining the nonlinear efficiency penalty coefficient, this efficiency penalty coefficient is used as a multiplicative regularization term to apply to the gated reward, resulting in the final target reward value: Through this mechanism, the student model is guided to automatically converge to the "minimum sufficient" inference path while ensuring prediction accuracy.

[0223] In this embodiment, the server guides the student model to compress the inference length as much as possible while ensuring the integrity of the inference and the accuracy of the semantics, through a dynamic efficiency baseline and a non-linear penalty mechanism. This significantly improves the efficiency of online inference and the user experience, while avoiding information loss due to excessive simplification.

[0224] In one exemplary embodiment, such as Figure 11 As shown, the specific processing steps of step 603 include:

[0225] Step 1101: Determine the basic advantage value of each interest circle in the set of interest circle prediction results output by the student model, and assign a credit score to each interest circle.

[0226] In implementation, the server uses Group Relative Strategy Optimization (GRPO) to calculate the basic advantage value of the response level corresponding to the prediction results of each interest circle in the output results (inference chain and interest prediction results) of each group. At the same time, based on whether each interest circle hits the real label, whether it belongs to the preset interest circle system, whether it is content that the user has interacted with in the past, and whether it is a duplicate generated item, a differentiated fine-grained credit score is assigned to each interest circle to distinguish recursive errors, exploratory errors and valid interests.

[0227] Specifically, a set of outputs from the student model The prediction results of each inference chain and interest circle are represented as follows: = …, Each is calculated by normalization Basic Advantage Value However, under this method, all tokens within the same round of response share the same advantage value, making it impossible to distinguish between valid interest circles and illusions within each interest circle prediction result (predicted interest circle set). To solve this credit allocation problem, this embodiment introduces TAA (Token Advantage Allocation), which specifically includes the following steps:

[0228] Step (1): Credit score allocation for interest circles: For the generated list (predicted set of interest circles)... Interest circles Assign a fine-grained credit score .set up This is a collection of interest groups that the user has recently interacted with. For the cumulative generation history (cumulative output history) of the student model, it is the previous response in the current response. The generated set of interest groups. The formula for calculating the credit score is as follows:

[0229] (10)

[0230] Optionally, two different error penalty mechanisms are designed in the above formula (10):

[0231] Recursive error: refers to the error that repeatedly occurs in the generation of history in the student model. ( ), impose a heavier punishment.

[0232] Exploratory errors: These are new errors that the student model misses on its first attempt, and are subject to a lighter penalty. The key lies in setting... Prioritizing exploration encourages students to develop model strategies that cover the global classification system (a pre-built interest-based hierarchical system). At the same time, repeated punishment Strictly enforce the uniqueness of sets to eliminate redundancy in the interest circles generated by student models.

[0233] Step 1102: Calculate the baseline credit statistic based on the credit score of each interest circle, and redistribute the basic advantage value corresponding to the prediction result of each interest circle according to the baseline credit statistic to obtain the corrected advantage value of each interest circle.

[0234] In practice, the server calculates the average credit score of all interest circles within the currently generated sequence to obtain a baseline credit statistic. This baseline credit statistic is then combined with the response-level basic advantage value to redistribute the word-level advantage value corresponding to each interest circle. This results in effective interest circles gaining a higher advantage, while illusionary and erroneous interest circles have their advantage suppressed, achieving precise credit allocation at the word level.

[0235] Specifically, referring to the specific example in step 1101 above (the content of step (1)), after obtaining the initial credit score... Then, the advantage values ​​of each interest group were further adjusted. The specific process is as follows:

[0236] Step (2): Advantage redistribution: Let To correspond to the generated interest circles The set of token lexical indices. First, calculate all the tokens in an interest circle prediction result. Baseline credit statistics for each interest group (i.e., calculating the baseline credit statistic of each interest group) The average credit score of each interest circle), the specific formula is as follows (11):

[0237] (11)

[0238] Subsequently, for those belonging to (Prediction results of a certain inference chain and interest circle) The first in the interest circle prediction results Interest circles Any token word, Its final corrected advantage value Defined as:

[0239] = (12)

[0240] in, for ( The Middle The base advantage value (based on the prediction results of each inference chain and interest circle). For the first Interest circles Corresponding credit score The difference from the baseline credit statistic.

[0241] Formula (12) combines the response-level advantage that distinguishes the prediction results of different interest circles with the token-level advantage that distinguishes the prediction results of different interest circles, so that effective interests are explicitly reinforced while illusions are suppressed, thereby enabling the student model to perform precise fine-grained optimization at the token level.

[0242] Step 1103: Using an upper limit truncation mechanism, a reinforcement learning optimization objective function is constructed based on the corrected advantage values ​​of each interest layer. The student model is iteratively optimized by maximizing the objective function to obtain the reinforcement learning optimized student model.

[0243] Among them, the student model optimized by reinforcement learning is used to output the prediction results of inference chain and interest circle.

[0244] In implementation, the server adopts an asymmetric Clip-higher truncation mechanism, with the corrected lexical-level advantage value as the core, to construct a reinforcement learning optimization objective function for interest circle generation. By maximizing this objective function, the parameters of the student model are iteratively updated, enabling the model to accurately strengthen effective interests, suppress illusions and redundant generation, and finally obtain a reinforcement learning optimized student model that can stably output explicit inference chains and accurate interest circle prediction results.

[0245] Specifically, reinforcement learning optimizes the objective function as follows:

[0246] (13)

[0247] in, [⋅] represents the expectation operator. The variable represents the importance sampling ratio. This represents the dominant value of the corrected redistribution derived above. Following an asymmetric truncation design, a truncation threshold is set. =0.2 and =0.28, to allow for a wider exploration of positive dominance values. Clipping function. Regularity coefficient. The Kronbach-Leibler divergence (KL divergence) measures the difference between two distributions.

[0248] In this embodiment, the server solves the problems of coarse credit allocation and inability to distinguish between valid interests and illusions in traditional reinforcement learning by using a complete process of word-level credit allocation, advantage redistribution and upper limit truncation optimization, which significantly improves the accuracy, compliance and interpretability of model generation.

[0249] In one exemplary embodiment, such as Figure 12 As shown, the process of generating interest circle sets includes:

[0250] Step 1201: Obtain the basic interest tag set.

[0251] The basic interest tag set contains multiple basic interest tags in a predefined semantic space.

[0252] In implementation, the server obtains a set of basic interest tags from a pre-built closed interest semantic space. This set of basic interest tags consists of multiple fine-grained basic interest tags obtained through manual sorting, business definition, and data statistics, covering various interest directions that users may generate, and providing a source of basic tags for the subsequent construction of interest circles.

[0253] Specifically, taking a four-tiered interest tagging system as an example, the basic interest tags are the fourth-tier tags, which are the lowest level (the most granular) of the interest tagging system. Each tag corresponds to a specific content interest tag.

[0254] Step 1202: Based on user behavior data, perform preliminary aggregation of basic interest tags in the basic interest tag set to obtain an initial subset of interest tags.

[0255] In implementation, the server uses users' historical clicks, browsing, dwell time, and interaction data to count the co-occurrence frequency and correlation between different basic interest tags. It then clusters and initially aggregates basic interest tags that are highly related in behavior and have similar orientations to form multiple initial interest tag subsets with intrinsic connections.

[0256] Specifically, the server calculates the degree of collaborative association between different fourth-level interest tags based on user behavior data in the recommendation system, such as video co-click relationships, co-view relationships, or co-occurrence relationships in user behavior sequences, and constructs a collaborative relationship network between interest tags. Based on this, clustering methods are used to aggregate interest tags with strong collaborative relationships, thus forming several initial subsets of interest tags. Each subset represents a type of interest topic that frequently appears in user consumption behavior.

[0257] Step 1203: Perform semantic cleaning and structural optimization on each initial interest tag subset, and then merge each initial interest tag subset a second time to obtain the merged interest tag subsets.

[0258] In implementation, the server performs semantic cleaning operations such as semantic deduplication, ambiguity removal, and noise filtering on each initial interest tag subset. At the same time, it optimizes the hierarchical relationship and association logic between tags. Furthermore, it merges the initial interest tag subsets with similar semantics and blurred boundaries a second time to obtain multiple interest tag subsets with clear structure and pure semantics.

[0259] Step 1204: Semantically summarize each subset of interest tags to generate the name of the interest circle corresponding to each subset of interest tags, thus obtaining the set of interest circles.

[0260] In implementation, the server performs high-level semantic abstraction and summarization on each merged subset of interest tags, extracting circle names that can uniformly represent the overall interest direction of the subset, thereby forming a set of interest circles with clear semantics, standardized structure, and clear hierarchy, which are used for subsequent user interest prediction and model training.

[0261] In this embodiment, a comprehensive and well-defined set of basic interest tags is first obtained. Then, user behavior data is used to initially aggregate the tags, which can effectively uncover the inherent relationships between tags. Subsequently, through semantic cleaning, structural optimization, and secondary merging, noise can be removed, ambiguity eliminated, and the tag system structure regularized. Finally, a standardized set of interest circles is generated through semantic induction, thereby constructing a closed interest circle system that is comprehensive, semantically clear, structurally standardized, and closely matches the distribution of real user interests. This provides a stable and reliable semantic foundation and standard reference for subsequent user interest inference, model supervised training, and reinforcement learning optimization, improving the accuracy, standardization, and interpretability of interest prediction.

[0262] In one exemplary embodiment, such as Figure 13 As shown, the specific processing steps of step 1202 include:

[0263] Step 1301: Based on user behavior data, calculate the degree of collaborative association among each basic interest tag in the basic interest tag set, and construct a collaborative relationship network based on the degree of collaborative association.

[0264] In implementation, the server uses historical behavioral data generated by users in business scenarios to count the co-occurrence frequency, association probability, and behavioral similarity between different basic interest tags. This quantifies the degree of collaborative association between basic interest tags and organizes the related basic interest tags into nodes and edges to build a collaborative relationship network that reflects the inherent association between tags.

[0265] Step 1302: Using a preset clustering method, the basic interest tags in the collaborative relationship network are aggregated into multiple initial interest tag subsets.

[0266] In implementation, the server uses a preset clustering algorithm to cluster the basic interest tags in the collaborative relationship network, classifying basic interest tags with high degree of collaborative association, similar semantics, and close behavioral binding into the same category, thereby aggregating the overall basic interest tags into multiple subsets of initial interest tags with intrinsic relationships.

[0267] In this embodiment, by constructing a collaborative relationship network based on user behavior and performing clustering and aggregation, we can objectively mine the inherent relationships between basic interest tags from the perspective of real behavior distribution. This makes the generated initial interest tag subset more closely match the actual distribution of user interests, providing an accurate and reliable data foundation for the subsequent construction of a standardized and unified set of interest circles.

[0268] It should be understood that, although Figures 1 to 13The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figures 1 to 13 At least some of the steps in the process may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but may be executed at different times. The execution order of these steps or stages is not necessarily sequential, but may be executed in turn or alternately with other steps or at least some of the steps or stages in other steps.

[0269] It is understood that the same / similar parts between the various embodiments of the methods described above in this specification can be referred to each other. Each embodiment focuses on the differences from other embodiments, and relevant parts can be referred to the description of other method embodiments.

[0270] Figure 14 This is a block diagram illustrating a recommended apparatus according to an exemplary embodiment. (Refer to...) Figure 14 The device 1400 includes a first acquisition module 1401 and an output module 1402.

[0271] The first acquisition module 1401 is configured to acquire user profile features and user historical behavior sequence. The user profile features are used to characterize the statistical distribution features of each user's interactive behavior in the video platform under each level of interest circle in the predefined semantic space. The user historical behavior sequence is a sequence formed by mapping each user's interactive behavior to the predefined semantic space within a preset time period and sorting it by time.

[0272] Output module 1402 is configured to execute the student model trained to analyze and process user profile features and user historical behavior sequences to obtain inference chain and interest circle prediction results; the inference chain is the explicit semantic expression of the logical reasoning process of the student model; the interest circle in the interest circle prediction result is the plain text semantic expression unit of a user's single interest topic in a predefined speech space, which is used to determine the recommendation object;

[0273] The student model is trained in a phased manner: it completes self-supervised pre-training based on the teacher model and a pre-set interest circle corpus, learns the semantic interpretation of interest circles, and then completes supervised training based on user profile features, the inference chain output by the teacher model and the reference interest circle set. It constructs and maximizes the optimization of the objective function by combining multi-dimensional reward function, interest circle credit weight allocation, word level advantage value adaptive correction and parameter upper limit truncation constraint mechanism, and completes the model reinforcement learning iterative optimization.

[0274] In one exemplary embodiment, the device 1400 further includes:

[0275] The first construction module is used to input the name of the interest circle, the basic interest tag and the target video text metadata into the teacher model, output the semantic explanation corresponding to each interest circle, and build a corpus containing the interest circle and the semantic explanation.

[0276] The pre-training module is used to perform self-supervised pre-training on the student model based on the interest circles and corresponding semantic interpretations contained in the corpus, so as to obtain the pre-trained student model.

[0277] In an exemplary embodiment, the pre-training module is specifically used to input each interest circle in the corpus into the student model, triggering the student model to output the predicted semantic interpretation corresponding to each interest circle;

[0278] Based on the predicted semantic interpretation output by the student model and the actual semantic interpretation corresponding to the interest circles in the corpus, the semantic loss is calculated to obtain the semantic loss result.

[0279] The parameters of the student model are iteratively updated based on the semantic loss results until the semantic loss meets the preset pre-training conditions, thus obtaining the pre-trained student model.

[0280] In one exemplary embodiment, the device 1400 further includes:

[0281] The processing module is used to input user profile features, user historical behavior sequences, and sets of interest circles of actual user interactions into the teacher model to obtain the inference chain and the set of reference interest circles.

[0282] The supervised training module is used to supervise the training of the pre-trained student model based on user profile features, inference chains, and reference interest circle sets, so as to obtain the trained student model.

[0283] In an exemplary embodiment, the supervised training module is specifically used to input user profile features into the pre-trained student model, triggering the student model to output the corresponding inference chain and interest circle prediction results;

[0284] The inference chain and interest circle prediction results output by the student model are compared with the inference chain and reference interest circle set output by the teacher model, respectively, and the comprehensive supervision loss is calculated.

[0285] The parameters of the pre-trained student model are iteratively optimized based on the comprehensive supervised loss until the comprehensive supervised loss meets the preset supervised training conditions, thus obtaining the trained student model.

[0286] In one exemplary embodiment, the device 1400 further includes:

[0287] The module is used to construct a target reward function based on the integrity of the inference chain, the accuracy of interest circle matching, and the effectiveness of the recommendation.

[0288] The incentive module is used to give multi-dimensional reward scores to the student model according to the target reward function, and to iteratively optimize the model parameters in the student model based on the feedback of the reward scores.

[0289] The reinforcement learning module is used to correct the advantage value based on the basic advantage value and credit score of each interest circle output by the student model, and to construct the objective function by combining the upper limit truncation mechanism. The student model is then iteratively optimized by maximizing the objective function.

[0290] In one exemplary embodiment, the incentive module is specifically used to construct a basic reward function and obtain a basic reward value based on the training progress of the model training and the set of interest circles of real user interactions;

[0291] Calculate the gate reward value based on the preset gate factor and the base reward value;

[0292] The gating reward value is corrected based on the nonlinear efficiency penalty coefficient to obtain the target reward value; the target reward value is used to provide reward feedback to the trained student model.

[0293] In an exemplary embodiment, the incentive module is specifically used to calculate the current training progress of the student model based on a hybrid exponential moving average algorithm, and to determine the semantic accuracy weight and grammatical format weight in the basic reward function based on the current training progress and in conjunction with an adaptive course learning mechanism.

[0294] Based on the current training progress, the set of interest circles of real user interactions, semantic accuracy weights, and grammatical format weights, the basic reward value of the basic reward function is obtained.

[0295] In an exemplary embodiment, the excitation module is specifically used to calculate the values ​​of a preset validity gating factor, a deduplication gating factor, and an accuracy gating factor, respectively.

[0296] The values ​​of the validity gating factor, deduplication gating factor, and accuracy gating factor are multiplied together to obtain the comprehensive gating coefficient;

[0297] The gating reward value is obtained based on the base reward value and the comprehensive gating coefficient.

[0298] In an exemplary embodiment, the activation module is specifically used to determine the inference chain and interest circle prediction result with the shortest inference length among the optimal inference chains and interest circle prediction results output by the student model.

[0299] The dynamic efficiency baseline is calculated based on the minimum inference length hyperparameter, the inference chain with the shortest inference length, and the prediction results of the interest sphere.

[0300] Calculate the nonlinear efficiency penalty coefficient corresponding to the inference chain generated by the student model based on the dynamic efficiency baseline;

[0301] The target reward value is obtained by multiplying the gating reward value by the nonlinear efficiency penalty coefficient.

[0302] In an exemplary embodiment, the reinforcement learning module is specifically used to determine the basic advantage value of each interest circle in a set of interest circle prediction results output by the student model, and to assign a credit score to each interest circle.

[0303] Based on the credit scores of each interest circle, a baseline credit statistic is calculated, and based on the baseline credit statistic, the basic advantage value corresponding to the prediction result of each interest circle is redistributed to obtain the corrected advantage value of each interest circle.

[0304] An upper limit truncation mechanism is adopted, and a reinforcement learning optimization objective function is constructed based on the corrected advantage values ​​of each interest circle. The student model is iteratively optimized by maximizing the objective function to obtain the reinforcement learning optimized student model.

[0305] The student model optimized by reinforcement learning is used to output the prediction results of inference chains and interest circles.

[0306] In one exemplary embodiment, the device 1400 further includes:

[0307] The second acquisition module is used to acquire a set of basic interest tags, which contains multiple basic interest tags in a predefined semantic space.

[0308] The aggregation module is used to perform preliminary aggregation of basic interest tags in the basic interest tag set based on user behavior data, to obtain an initial subset of interest tags.

[0309] The merging module is used to perform semantic cleaning and structural optimization on each initial interest tag subset, and to perform a second merging on each initial interest tag subset to obtain the merged interest tag subsets.

[0310] The generation module is used to perform semantic summarization on each subset of interest tags, generate the name of the interest circle corresponding to each subset of interest tags, and obtain the set of interest circles.

[0311] In one exemplary embodiment, the aggregation module is specifically used to calculate the degree of collaborative association among each basic interest tag in the basic interest tag set based on user behavior data, and to construct a collaborative relationship network based on the degree of collaborative association.

[0312] By using a pre-defined clustering method, the basic interest tags in the collaborative relationship network are aggregated into multiple initial interest tag subsets.

[0313] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0314] Figure 15 This is a block diagram illustrating an electronic device 1500 for performing a recommended method according to an exemplary embodiment. For example, the electronic device 1500 may be a server. (Refer to...) Figure 15 The electronic device 1500 includes a processing component 1520, which further includes one or more processors, and memory resources represented by memory 1522 for storing instructions, such as application programs, that can be executed by the processing component 1520. The application programs stored in memory 1522 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 1520 is configured to execute instructions to perform the methods described above.

[0315] Electronic device 1500 may further include: a power supply component 1524 configured to perform power management of electronic device 1500, a wired or wireless network interface 1526 configured to connect electronic device 1500 to a network, and an input / output (I / O) interface 1528. Electronic device 1500 may operate on an operating system stored in memory 1522, such as Windows Server, MacOS X, Unix, Linux, FreeBSD, or similar.

[0316] In an exemplary embodiment, a computer-readable storage medium including instructions is also provided, such as a memory 1522 including instructions, which can be executed by a processor of an electronic device 1500 to perform the above-described method. The storage medium may be a computer-readable storage medium, such as a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device.

[0317] In an exemplary embodiment, a computer program product is also provided, which includes instructions that can be executed by a processor of an electronic device 1500 to perform the above-described method.

[0318] It should be noted that the above-mentioned apparatus, electronic equipment, computer-readable storage medium, computer program product, etc., may also include other implementation methods according to the description of the method embodiments. For specific implementation methods, please refer to the description of the relevant method embodiments, which will not be elaborated here.

[0319] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the claims.

[0320] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A recommendation method, characterized in that, The method includes: The system acquires user profile features and user historical behavior sequences. The user profile features are used to characterize the statistical distribution features of each user's interactive behavior within the video platform in each level of interest circle under a predefined semantic space. The user historical behavior sequence is a sequence formed by mapping each user's interactive behavior within a preset time period to the predefined semantic space and sorting it by time. The trained student model analyzes and processes the user profile features and the user's historical behavior sequence to obtain inference chain and interest circle prediction results. The inference chain is an explicit semantic expression of the logical reasoning process of the student model. The interest circle prediction results are plain text semantic expression units of a user's single interest topic in the predefined semantic space, used to determine the recommendation object. The student model is trained in a phased manner: it is pre-trained by relying on the teacher model and a pre-set interest circle corpus to learn the semantic interpretation of interest circles, and then supervised training is completed based on user profile features, the inference chain output by the teacher model and the reference interest circle set. The objective function is constructed and maximized by combining multi-dimensional reward function, interest circle credit weight allocation, word level advantage value adaptive correction and parameter upper limit truncation constraint mechanism to complete the model reinforcement learning iterative optimization.

2. The recommended method according to claim 1, characterized in that, The process involves self-supervised pre-training based on a teacher model and a pre-defined corpus of interest-based semantics, learning semantic interpretations of interest-based circles, including: Input the interest circle name, basic interest tags and target video text metadata into the teacher model, output the semantic explanation corresponding to each interest circle, and construct a corpus containing the interest circle and the semantic explanation; Based on the interest circles and corresponding semantic interpretations contained in the corpus, the student model is pre-trained under self-supervised supervision to obtain the pre-trained student model.

3. The recommended method according to claim 2, characterized in that, The process of performing self-supervised pre-training on the student model based on the interest circles and corresponding semantic interpretations contained in the corpus to obtain the pre-trained student model includes: The student model is then input into each interest circle in the corpus, triggering the student model to output the predicted semantic explanation corresponding to each interest circle. Based on the predicted semantic interpretation output by the student model and the actual semantic interpretation corresponding to the interest circle in the corpus, the semantic loss is calculated to obtain the semantic loss result. The parameters of the student model are iteratively updated based on the semantic loss result until the semantic loss meets the preset pre-training conditions, thus obtaining the pre-trained student model.

4. The recommended method according to claim 1, characterized in that, The supervised training is then completed based on user profile features, the inference chain output by the teacher model, and the set of reference interest circles, including: The user profile features, user historical behavior sequences, and the set of interest circles of actual user interactions are input into the teacher model to obtain the inference chain and the set of reference interest circles. The pre-trained student model is trained under supervision based on the user profile features, the inference chain, and the set of reference interest circles to obtain the trained student model.

5. The recommended method according to claim 4, characterized in that, The process of supervising the pre-trained student model based on the user profile features, the inference chain, and the reference interest circle set to obtain a trained student model includes: The user profile features are input into the pre-trained student model, which triggers the student model to output the corresponding inference chain and interest circle prediction results. The inference chain and interest circle prediction results output by the student model are compared with the inference chain and reference interest circle set output by the teacher model, respectively, and the comprehensive supervision loss is calculated. The parameters of the pre-trained student model are iteratively optimized based on the comprehensive supervised loss until the comprehensive supervised loss meets the preset supervised training conditions, thus obtaining the trained student model.

6. The recommendation method according to claim 1, characterized in that, The process involves constructing and maximizing the objective function by combining a multi-dimensional reward function, interest-based credit weight allocation, adaptive correction of word-level advantage values, and a parameter upper limit truncation constraint mechanism to complete iterative optimization of the model reinforcement learning, including: Construct a target reward function based on the integrity of the inference chain, the accuracy of interest circle matching, and the effectiveness of recommendation; The student model is given a multi-dimensional reward score based on the target reward function, and the model parameters in the student model are iteratively optimized based on the feedback of the reward score. Based on the basic advantage value and credit score of each interest circle output by the student model, the advantage value is corrected, and an objective function is constructed by combining the upper limit truncation mechanism. The student model is then iteratively optimized by maximizing the objective function through reinforcement learning.

7. The recommended method according to claim 6, characterized in that, The step of performing multi-dimensional reward scoring on the student model based on the target reward function includes: Based on the training progress of the model training and the set of interest circles of real user interactions, a basic reward function is constructed and a basic reward value is obtained. Calculate the gated reward value based on the preset gated factor and the base reward value; The gated reward value is corrected based on a nonlinear efficiency penalty coefficient to obtain a target reward value; the target reward value is used to provide reward feedback to the trained student model.

8. The recommended method according to claim 7, characterized in that, The set of interest circles based on model training progress and real user interactions is used to construct a basic reward function and obtain a basic reward value, including: Based on the hybrid exponential moving average algorithm, the current training progress of the student model is calculated, and based on the current training progress combined with the adaptive course learning mechanism, the semantic accuracy weight and grammatical format weight in the basic reward function are determined. Based on the current training progress, the set of interest circles of real user interaction, the semantic accuracy weight, and the grammatical format weight, the basic reward value of the basic reward function is obtained.

9. The recommended method according to claim 7, characterized in that, The step of calculating the gated reward value based on the preset gated factor and the base reward value includes: Calculate the values ​​of the preset validity gate factor, deduplication gate factor, and accuracy gate factor respectively; The values ​​of the legality gate factor, the deduplication gate factor, and the accuracy gate factor are multiplied together to obtain the comprehensive gate coefficient; The gating reward value is obtained based on the base reward value and the comprehensive gating coefficient.

10. The recommended method according to claim 7, characterized in that, The step of correcting the gated reward value based on a nonlinear efficiency penalty coefficient to obtain the target reward value includes: Among the optimal inference chains and interest circle prediction results output by the student model, the inference chain and interest circle prediction result with the shortest inference length are determined. The dynamic efficiency baseline is calculated based on the minimum inference length hyperparameter, the inference chain with the shortest inference length, and the prediction results of the interest sphere. Calculate the nonlinear efficiency penalty coefficient corresponding to the inference chain generated by the student model based on the dynamic efficiency baseline; The target reward value is obtained by multiplying the gated reward value by the nonlinear efficiency penalty coefficient.

11. The recommendation method according to claim 6, characterized in that, The student model's basic advantage value and credit score for each interest circle are used to correct the advantage value, and an objective function is constructed using an upper limit truncation mechanism. The student model is then iteratively optimized through reinforcement learning by maximizing the objective function, including: Determine the basic advantage value of each interest circle in the set of interest circle prediction results output by the student model, and assign a credit score to each interest circle. Based on the credit scores of each interest circle, a baseline credit statistic is calculated, and based on the baseline credit statistic, the basic advantage value corresponding to the prediction result of each interest circle is redistributed to obtain the corrected advantage value of each interest circle. An upper limit truncation mechanism is adopted, and a reinforcement learning optimization objective function is constructed based on the corrected advantage values ​​of each interest zone. The student model is iteratively optimized by maximizing the objective function to obtain the reinforcement learning optimized student model. The reinforcement learning-optimized student model is used to output inference chain and interest circle prediction results.

12. The recommendation method according to claim 1, characterized in that, The process of generating the interest circle set includes: Obtain a set of basic interest tags, which contains multiple basic interest tags in a predefined semantic space; Based on user behavior data, the basic interest tags in the basic interest tag set are initially aggregated to obtain an initial subset of interest tags; Semantic cleaning and structural optimization are performed on each of the initial interest tag subsets, and the initial interest tag subsets are merged twice to obtain the merged interest tag subsets. Semantic induction is performed on each of the interest tag subsets to generate the interest circle name corresponding to each of the interest tag subsets, thus obtaining the interest circle set.

13. The recommended method according to claim 12, characterized in that, The initial aggregation of basic interest tags in the basic interest tag set based on user behavior data yields an initial subset of interest tags, including: Based on user behavior data, the degree of collaborative association among each basic interest tag in the basic interest tag set is calculated, and a collaborative relationship network is constructed based on the degree of collaborative association. By using a pre-defined clustering method, the basic interest tags in the collaborative relationship network are aggregated into multiple initial interest tag subsets.

14. A recommendation device, characterized in that, The device includes: The first acquisition module is configured to acquire user profile features and user historical behavior sequences. The user profile features are used to characterize the statistical distribution features of each user's interactive behavior in the video platform at each level of interest circle under a predefined semantic space. The user historical behavior sequence is a sequence formed by mapping each user's interactive behavior to the predefined semantic space within a preset time period and sorting it by time. The output module is configured to execute the student model, which has been trained, to analyze and process the user profile features and the user's historical behavior sequence to obtain the inference chain and interest circle prediction results; the inference chain is the explicit semantic expression of the logical reasoning process of the student model; the interest circle in the interest circle prediction results is the plain text semantic expression unit of a user's single interest topic in the predefined semantic space, which is used to determine the recommendation object; The student model is trained in a phased manner: it is pre-trained by relying on the teacher model and a pre-set interest circle corpus to learn the semantic interpretation of interest circles, and then supervised training is completed based on user profile features, the inference chain output by the teacher model and the reference interest circle set. The objective function is constructed and maximized by combining multi-dimensional reward function, interest circle credit weight allocation, word level advantage value adaptive correction and parameter upper limit truncation constraint mechanism to complete the model reinforcement learning iterative optimization.

15. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the recommended method as described in any one of claims 1 to 13.

16. A computer-readable storage medium, characterized in that, When the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device is enabled to perform the recommended method as described in any one of claims 1 to 13.

17. A computer program product, the computer program product comprising instructions, characterized in that, When the instructions are executed by the processor of the electronic device, the electronic device is able to perform the recommended method as described in any one of claims 1 to 13.