Recommendation method and system based on language modeling and elastic reasoning collaborative architecture
By constructing a recommendation system based on a collaborative architecture of language modeling and elastic inference, the problem of inconsistent recommendation performance caused by differences in model size was solved. This system accurately models user interests and dynamically adapts to model size in recommendation tasks, thereby improving the performance and efficiency of the recommendation system.
Patent Information
- Application Number
- CN202511735548.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-25
- Publication Date
- 2026-01-02
AI Technical Summary
Existing recommendation methods based on large language models suffer from inconsistent recommendation performance due to differences in model size. Furthermore, the flexible architecture is not optimized for the semantic characteristics of recommendation tasks, making it difficult to adaptively select the optimal computational scale to achieve the best recommendation performance.
A recommendation system based on a collaborative architecture of language modeling and elastic reasoning is adopted. Through a data module, a language representation construction module, an elastic sub-model generation module, a dynamic routing training module, and an adaptive recommendation module, multi-scale nested sub-models are constructed. The semantic understanding and generation capabilities of the large language model are utilized, and the optimal sub-model is selected for recommendation by a dynamic routing mechanism.
It enables accurate modeling of user interests in recommendation tasks, dynamically adapts to different model sizes, improves the balance between recommendation performance and efficiency, and enhances the generalization ability and personalized recommendation effect of the recommendation system.
Smart Images
Figure CN121256145A_ABST
Abstract
Description
[0001] This invention belongs to the field of recommendation technology, specifically a recommendation method and system based on a collaborative architecture of language modeling and flexible reasoning. Background Technology
[0002] Recommender systems, as a core technology for information filtering, have been widely applied in e-commerce, short video platforms, news feeds, and other scenarios. In recent years, with the rapid development of large language models, researchers have begun to explore the linguisticization of recommendation tasks, that is, transforming structured data such as user behavior and item attributes into natural language form, and utilizing the powerful semantic understanding and generation capabilities of large language models for unified modeling. Recommender systems unify various recommendation tasks (such as rating prediction, sequence recommendation, and explanation generation) into text-to-text generation tasks, significantly improving the model's generalization ability.
[0003] However, recommendation methods based on large language models generally suffer from inconsistent recommendation performance due to differences in model size. To alleviate this problem, recent research has proposed a nested Transformer architecture, which constructs multi-scale substructures within a single model to achieve flexible inference capabilities with one-time training and multi-level inference. Nevertheless, existing work still has significant limitations: firstly, recommendation methods using large language models for text generation do not consider the performance inconsistencies caused by differences in model size, making it difficult to effectively utilize large language models for optimal recommendations; secondly, the flexible architecture is not optimized for the semantic characteristics of recommendation tasks, and its sub-models may lack sufficient semantic modeling capabilities. More importantly, there is currently no method that can adaptively select the sub-model with the optimal computational scale based on the dynamic needs of the recommendation task to achieve optimal recommendation performance.
[0004] Therefore, how to deeply integrate the semantic advantages of language-based recommendation with the efficiency advantages of flexible reasoning to build a model that can accurately model user interests and flexibly adapt to different scales according to recommendation tasks has become an urgent technical challenge. Summary of the Invention
[0005] To address the above problems, this invention proposes a recommendation method and system based on a collaborative architecture of language modeling and flexible reasoning, the key technologies of which include:
[0006] The data module is used to collect user interaction data and item metadata from smart terminals and store them in a database;
[0007] The language representation building module is used to format user behavior and item information into natural language prompts;
[0008] The elastic sub-model generation module is used to structurally segment the feedforward layer and multi-head attention to construct multi-scale nested sub-models;
[0009] The dynamic routing training module is responsible for training an elastic recommendation model with task-adaptive capabilities.
[0010] The adaptive recommendation module is used during the inference phase to load the trained elastic recommendation model, dynamically select the optimal sub-model based on the target user's context information and the current recommendation task, and generate personalized recommendation results.
[0011] In the recommendation system described in this invention, the data module synchronizes user interaction data and item metadata recorded by the smart terminal to the database via network protocols such as HTTPS.
[0012] In the recommendation system of this invention, the language representation construction module, based on the language modeling paradigm, uses rating prediction task template, sequence recommendation task template, explanation generation task template, comment task template and direct recommendation template to format user behavior and item information into natural language prompts, and uses an encoder in a large language model to encode them, generating user language representations and item language representations with rich semantics.
[0013] The recommendation system of the present invention, wherein the elastic sub-model generation module specifically includes:
[0014] (1) The elastic sub-model generation module constructs a unified Transformer backbone based on language representation, and divides the feedforward MLP into nested MLP sets in proportions of 1 / 4, 1 / 2, and 1. The specific division algorithm is as follows:
[0015]
[0016] Among them, MLP i Let W1 represent the feedforward layer of the i-th decoder, σ(x) represent the activation function, x represent the hidden state of the data, and W1 and W2 are two relevant weight matrices of the MLP. W[0:m] i ] indicates the first m of the weight matrix W i A submatrix composed of rows.
[0017] (2) Divide the Multi-Head Attention (MHA) head into nested self-attention head sets according to the same proportions of 1 / 4, 1 / 2, and 1. The specific splitting algorithm is as follows:
[0018] MHA i (x) = Concat(head1,...head) i )·W O [0:m i ],
[0019] Among them, MHA i This represents multi-head attention in the i-th layer decoder. Concat combines multiple self-attention heads. iTo splice, W O To convert the result into the final output dimension for the output weight matrix, m i Indicates the first m i A submatrix composed of rows.
[0020] The recommendation system of the present invention, wherein the dynamic route training module specifically includes:
[0021] (1) When the dynamic routing training module executes the elastic training sub-module training, it randomly samples the sub-model for optimization in each round. Throughout the training process, it randomly samples the nested sub-model. For each step, it performs random sampling and splitting of granularity, and uses a standard stochastic gradient-based optimizer for training:
[0022]
[0023] in Let M be the random sampling loss function. i It is the i-th nested sub-model. Input data to the model, where y corresponds to M. i The output of the model.
[0024] (2) When the dynamic routing training module executes the task-aware routing submodule training, it uses the task type and the task output of different sub-models as new training data, and iteratively optimizes the objective loss function through gradient descent until the model converges. The original output logits of the model are converted into a probability distribution through the softmax function, and the subnet with the highest probability is selected. The expression of the softmax function is as follows:
[0025]
[0026] Where z is the original output logits vector of the model, z i is the original output of the i-th category, and K is the total number of categories.
[0027] In the recommendation system of the present invention, the adaptive recommendation module receives user requests, extracts task features, selects the optimal sub-model by the routing network, performs efficient inference, and returns recommendation results during the service phase.
[0028] The beneficial effects of this invention are that it transforms the recommendation task into a language understanding and generation problem, making full use of the semantic generalization ability of large language models to accurately capture the evolution of user interests; and in the recommendation modeling, a nested structure is used to support multi-level computational complexity within a single model, and through a task-aware dynamic routing mechanism, the optimal sub-model is automatically selected according to actual needs to achieve a dynamic balance between performance and efficiency. Attached Figure Description
[0029] Figure 1This is a system framework diagram of the present invention;
[0030] Figure 2 This invention presents the RMSE and MAE values in three datasets for the rating prediction task.
[0031] Figure 3 This invention demonstrates the variation of NDCG values across three datasets with different recommendation list lengths in a sequence recommendation task.
[0032] Figure 4 This invention explains the BLEU4, ROUGE-1, ROUGE-2, and ROUGE-L values in three datasets under a generation task.
[0033] Figure 5 This invention presents the RMSE and MAE values in three datasets under the comment task.
[0034] Figure 6 This invention demonstrates the variation of NDCG values under different recommendation list lengths in three datasets for a direct recommendation task. Detailed Implementation
[0035] The specific embodiments and working principles of the present invention will be further described in detail below with reference to the accompanying drawings.
[0036] To make the objectives, technical solutions, and advantages of this invention clearer, the following detailed description, in conjunction with the accompanying drawings, provides a recommended method and system based on a language modeling and flexible reasoning collaborative architecture. It should be understood that the specific implementation methods described herein are merely illustrative of the invention and are not intended to limit the invention. Any changes, modifications, additions, alterations, or substitutions made by those skilled in the art within the scope of this invention should be covered by the claims of this invention.
[0037] Figure 1 This is a system framework diagram of the present invention. From Figure 1As can be seen, a personalized recommendation method and system based on a collaborative architecture of language modeling and flexible reasoning comprises five parts: a data module, a language representation construction module, a flexible sub-model generation module, a dynamic routing training module, and an adaptive recommendation module. The data module includes several smart terminals and a database; the language representation construction module includes templates for rating prediction tasks, sequence recommendation tasks, recommendation reason tasks, comment tasks, direct recommendation tasks, and the encoder part of a large language model; the flexible sub-model generation module includes a multi-scale segmentation sub-module, nested sub-modules, sub-model identifiers, and a decoder in the large language model; the dynamic routing training module includes a flexible training sub-module and a task-aware routing sub-module; finally, the adaptive recommendation module loads the trained model and selects different sub-models according to different tasks to achieve optimal recommendation performance.
[0038] Furthermore, the following example illustrates this further:
[0039] Let the user set be U, the item set be V, and the interaction between the user set and the item set be combined in the recommendation task set X.
[0040] according to Figure 1 The processing flow shown is implemented in the following steps:
[0041] S1: In the data module, user interaction data U and item metadata V recorded by the smart terminal are synchronized to the database via network protocols such as HTTPS.
[0042] S2: The language representation construction module reads data from the database, uses rating prediction task templates, sequence recommendation task templates, recommendation reason task templates, comment task templates and direct recommendation templates to combine the interaction between the user set and the item set in the recommendation task set X, and encodes it into the language representation of users and items through the encoder of the large language model.
[0043] S3: The elastic sub-model generation module constructs a unified Transformer backbone based on language representations, and splits the feedforward MLP into nested MLP sets in proportions of 1 / 4, 1 / 2, and 1. The specific splitting algorithm is as follows:
[0044]
[0045] Among them, MLP i Let σ represent the feedforward layer of the i-th decoder, σ represent the activation function, x represent the hidden state of the data, W1 and W2 are two relevant weight matrices of the MLP, and m i Indicates the first m i A submatrix composed of rows.
[0046] S4: Divide the Multi-Head Attention (MHA) head into nested self-attention head sets using the same proportions of 1 / 4, 1 / 2, and 1. The specific partitioning algorithm is as follows:
[0047] MHA i (x) = Concat(head1,...head) i )·W O [0:m i ],
[0048] Among them, MHA i This represents multi-head attention in the i-th layer decoder. Concat combines multiple self-attention heads. i To splice, W O To convert the result into the final output dimension for the output weight matrix, m i Indicates the first m i A submatrix composed of rows.
[0049] S5: When the training module executes the elastic training submodule, it randomly samples sub-models for optimization in each round, and randomly samples nested sub-models throughout the entire training process. For each step, it performs random sampling at a specific granularity and uses a standard stochastic gradient-based optimizer for training.
[0050]
[0051] in Let M be the random sampling loss function. i It is the i-th nested sub-model. Input data to the model, where y corresponds to M. i The output of the model.
[0052] S6: When the training module executes the task-aware routing submodule training, it uses the task type and the task output of different sub-models as new training data, and iteratively optimizes the objective loss function using gradient descent until the model converges. The original model output logits are then converted into a probability distribution using the softmax function, and the sub-network with the highest probability is selected. The softmax function expression is shown below:
[0053]
[0054] Where z is the original output logits vector of the model, z i is the original output of the i-th category, and K is the total number of categories.
[0055] S7: During the service phase, the adaptive recommendation module receives user requests, extracts task features, selects the optimal sub-model by the routing network, performs efficient inference, and returns the recommendation results.
[0056] Figure 2-6 The recommendation model's performance was measured across six metrics. Five of these metrics are: Root Mean Squared Error (RMSE), Mean Absolute Error (MAE), Normalized Discounted Cumulative Gain (NDCG), Bilingual Evaluation Understudy (BLEU), and Recall-Oriented Understudy for Gisting Evaluation (ROUGE). The metrics and their calculation formulas are shown below:
[0057] RMSE Metric: In this invention, the RMSE metric focuses on the user's rating of an item in rating prediction and review tasks.
[0058] For the predicted result y i The RMSE indicator is calculated as follows:
[0059]
[0060] MAE Metric: In this invention, the MAE metric focuses on the user's rating of a project in rating prediction and review tasks.
[0061] For the predicted result y i The formula for calculating the MAE index is:
[0062]
[0063] NDCG Metric: In this invention, the NDCG metric focuses on the ranking quality of items in the recommendation list during a sequence recommendation task. Items ranking higher in the recommendation list contribute more to the user's experience.
[0064] The optimal order for items.
[0065]
[0066] Among them, rel idx NDCG@k is the relevance score of the item at location idx (1 for clicks / purchases, 0 otherwise), and NDCG@k is the cumulative gain from the loss.
[0067] BLEU score: The BLEU score is used in recommendation tasks to analyze how many phrases in the generated text appear in the reference text. The specific calculation formula is as follows:
[0068]
[0069] in, To generate text, For reference text, w n It represents the weight of the n-th gram, and BP is the length penalty factor used to penalize cases where the generated text is too short. n It is the accuracy of n-grams.
[0070] ROUGE metric: The ROUGE metric is used in the recommendation task to analyze and calculate the overlap between the generated text and the reference text (ReferenceSummaries) at the phrase and longest common subsequence levels. The specific calculation formula is as follows:
[0071]
[0072] Among them, gram N R represents an n-gram, which is a sequence of N words. LCS It is the recall rate based on the longest common subsequence, P. LCS Precision is based on the longest common subsequence, and β is used to balance recall and precision.
Claims
1. A recommendation method and system based on a collaborative architecture of language modeling and flexible reasoning, which integrates the language modeling paradigm and the concept of nested structures to achieve efficient, accurate, and adaptive personalized recommendations under a single unified architecture, characterized in that... The key technologies of this system mainly include: The data collection module is used to collect user interaction data and item metadata from smart terminals and store them in a database; The language representation building module is used to format user behavior and item information into natural language prompts; The elastic sub-model generation module is used to structurally segment the feedforward layer and multi-head attention to construct multi-scale nested sub-models; The dynamic routing training module is responsible for training an elastic recommendation model with task-adaptive capabilities. The adaptive recommendation module is used during the inference phase to load the trained elastic recommendation model, dynamically select the optimal sub-model based on the target user's context information and the current recommendation task, and generate personalized recommendation results.
2. In the recommendation method and system based on a language modeling and flexible reasoning collaborative architecture as described in claim 1, the data module synchronizes user interaction data and item metadata recorded by the smart terminal to the database via network protocols such as HTTPS.
3. In the recommendation method and system based on the collaborative architecture of language modeling and flexible reasoning as described in claim 1, the language representation construction module, based on the language modeling paradigm, uses rating prediction task template, sequence recommendation task template, explanation generation task template, comment task template and direct recommendation template to format user behavior and item information into natural language prompts, and uses an encoder in a large language model to encode them, generating user language representations and item language representations with rich semantics.
4. The recommendation method and system based on a collaborative architecture of language modeling and flexible reasoning according to claim 1, wherein the flexible sub-model generation module specifically includes: (1) The elastic sub-model generation module constructs a unified Transformer backbone based on language representation, and divides the feedforward MLP into nested MLP sets in proportions of 1 / 4, 1 / 2, and 1. The specific division algorithm is as follows: Among them, MLP i Let W[i] represent the feedforward layer of the i-th decoder, x represent the hidden state of the data, σ(·) represent the activation function, and W1 and W2 are two relevant weight matrices of the MLP. i ] indicates the first m of the weight matrix W i Submatrix composed of rows; (2) Divide the Multi-Head Attention (MHA) head into nested self-attention head sets according to the same proportions of 1 / 4, 1 / 2, and 1. The specific splitting algorithm is as follows: MHA i (x)=Concat(head1,...head i )·W O [0:m i ], Among them, MHA i This represents multi-head attention in the i-th layer decoder. Concat combines multiple self-attention heads. i To splice, W O The result is converted into the final output dimension to output the weight matrix.
5. The recommendation method and system based on a collaborative architecture of language modeling and flexible reasoning according to claim 1, wherein the dynamic routing training module specifically includes: (1) When the dynamic routing training module executes the elastic training sub-module training, it randomly samples the sub-model for optimization in each round. Throughout the training process, it randomly samples the nested sub-model. For each step, it performs random sampling and splitting of granularity, and uses a standard stochastic gradient-based optimizer for training: in, Let M be the random sampling loss function. i It is the i-th nested sub-model. Input data to the model, where y corresponds to M. i The output of the model; (2) When the dynamic routing training module executes the task-aware routing submodule training, it uses the task type and the task output of different sub-models as new training data, and iteratively optimizes the objective loss function through gradient descent until the model converges. The original output logits of the model are converted into a probability distribution through the softmax function, and the subnet with the highest probability is selected. The expression of the softmax function is as follows: Where z is the original output logits vector of the model, z i is the original output of the i-th category, and K is the total number of categories.
6. The recommendation method and system based on a language modeling and elastic reasoning collaborative architecture as described in claim 1, wherein in the service phase, the adaptive recommendation module receives user requests, extracts task features, selects the optimal sub-model by the routing network, performs efficient reasoning, and returns the recommendation result.