A collaborative sequence recommendation method based on a large language model
By employing collaborative embedding, mapping alignment, sequence encoding, and prompt mapping methods, combined with a large language model, the problem of insufficient utilization of collaborative preferences and temporal dependencies in recommendation methods is addressed, thereby improving the accuracy and robustness of the recommendation system.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- CHONGQING UNIV OF POSTS & TELECOMM
- Filing Date
- 2026-05-15
- Publication Date
- 2026-06-12
Smart Images
Figure CN122196276A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the fields of deep learning, recommendation systems and large language models, and specifically relates to a collaborative sequence recommendation method based on a large language model. Background Technology
[0002] Recommender systems have been widely applied in e-commerce, short videos, news, online education, and social media platforms. Their core task is to recommend items that users may be interested in based on their historical behavior and item attributes. As the scale of user behavior and the number of items continue to increase, how to more accurately model user preferences and improve the personalization of recommendation results has become an important research direction in the field of recommender systems.
[0003] Among existing recommendation methods, collaborative filtering methods learn latent representations of user preferences and item features by mining historical interaction relationships between users and items, which can effectively characterize global collaborative preference information. However, traditional collaborative filtering methods mainly model based on static interaction relationships, which fails to fully utilize the dynamic process of user interests changing over time and makes it difficult to effectively capture temporal dependencies in user behavior sequences.
[0004] On the other hand, sequence modeling-based recommendation methods can extract patterns in the evolution of user interests and the transition relationships between items by modeling users' historical interaction items in chronological order. In particular, Transformer-based sequence modeling methods, which utilize self-attention mechanisms to model long-distance dependencies, demonstrate strong feature extraction capabilities in recommendation tasks. However, relying solely on sequence modeling methods fails to adequately utilize the collaborative information inherent in the global user-item interaction relationships.
[0005] In recent years, large language models have demonstrated superior capabilities in natural language understanding, reasoning, and generation. Some recommendation methods based on large language models, such as CoLLM, typically express user preferences by constructing prompts and directly listing historical user interactions, and then the large language model performs recommendation inference. However, this approach often suffers from problems such as lengthy prompts, loosely organized historical interaction information, and difficulty in concisely expressing collaborative preferences and temporal features.
[0006] Therefore, how to simultaneously utilize collaborative preference information in user-item interactions, temporal evolution information in user historical behavior sequences, and semantic reasoning capabilities of large language models, and inject user preferences into large language models using user embedding representations rather than historical item enumeration, has become a pressing technical problem to be solved in this field. Summary of the Invention
[0007] To address the problems of insufficient utilization of collaborative information, weak integration of sequence modeling and large language model, and lengthy prompt input in existing technologies, this invention proposes a collaborative sequence recommendation method based on a large language model, comprising: acquiring user-item interaction data to be recommended; inputting the user-item interaction data into a trained recommendation model to obtain the user's ranking score for the target item; and recommending items based on the ranking score; wherein the recommendation model includes a collaborative embedding module, a mapping alignment module, a sequence encoding module, a fusion module, a prompt mapping module, and a large language model inference module;
[0008] Training the recommendation model includes: acquiring raw user-item interaction data; inputting the raw user-item interaction data into a collaborative embedding module to obtain user collaborative embeddings and item collaborative embeddings; mapping the user collaborative embeddings and item collaborative embeddings to a unified feature space through a mapping alignment module; arranging the user's historical interaction items in chronological order to construct a user interaction item sequence; inputting the user interaction item sequence into a sequence encoding module for modeling to obtain user sequence embeddings and item sequence embeddings; fusing the user sequence embeddings with the mapped user collaborative embeddings to obtain a user fusion representation; fusing the item sequence embeddings with the mapped item collaborative embeddings to obtain an item fusion representation; converting the user fusion representation into a user prompt embedding and the item fusion representation into an item prompt embedding through a prompt mapping module; inputting the user prompt embeddings, item prompt embeddings, and recommendation task instructions into a large language model to obtain the user's ranking score for the target item; constructing a loss function based on the target item ranking score and updating the model parameters according to the loss function; when the loss function converges, the trained recommendation model is obtained.
[0009] The beneficial effects of this invention are:
[0010] This invention learns user collaborative embeddings and item collaborative embeddings using traditional collaborative models, effectively extracting global collaborative preference information from original user-item interaction data. It models user interaction item sequences using a Transformer, effectively capturing the temporal evolution of user interests and item transfer dependencies. A fusion module unifies collaborative embeddings and sequence embeddings, enabling complementary utilization of long-term preference information and dynamic interest information. Instead of directly enumerating historical user interaction items in prompts, this invention converts user fusion representations into user prompt embeddings and item fusion representations into item prompt embeddings before inputting them into a large language model. This allows for more compact injection of user preference information into the large language model, reducing the input burden caused by verbose prompts. Leveraging the semantic understanding and reasoning capabilities of the large language model, this invention performs recommendation inference on the fused user preference representation, thereby improving the accuracy, robustness, and generalization ability in recommendation tasks. This invention is applicable to various tasks such as click prediction, purchase prediction, rating prediction, next item recommendation, and ranking recommendation, possessing strong versatility and engineering application value. Attached Figure Description
[0011] Figure 1 This is a flowchart of a collaborative sequence recommendation method based on a large language model provided in an embodiment of the present invention;
[0012] Figure 2 This is a diagram illustrating the overall framework of a user interaction sequence modeling and collaborative embedding fusion model based on a large language model, as provided in this embodiment of the invention.
[0013] Figure 3 This is a schematic diagram of the user embedding prompt mapping and large language model recommendation reasoning process provided in the embodiments of the present invention. Detailed Implementation
[0014] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0015] A collaborative sequence recommendation method based on a large language model, such as Figures 1-3 As shown, the method includes: acquiring user-item interaction data to be recommended; inputting the user-item interaction data into a trained recommendation model to obtain the user's ranking score for the target item; and recommending items based on the ranking score. The recommendation model includes a collaborative embedding module, a mapping alignment module, a sequence encoding module, a fusion module, a prompt mapping module, and a large language model inference module.
[0016] Training the recommendation model includes: acquiring raw user-item interaction data; inputting the raw user-item interaction data into a collaborative embedding module to obtain user collaborative embeddings and item collaborative embeddings; mapping the user collaborative embeddings and item collaborative embeddings to a unified feature space through a mapping alignment module; arranging the user's historical interaction items in chronological order to construct a user interaction item sequence; inputting the user interaction item sequence into a sequence encoding module for modeling to obtain user sequence embeddings and item sequence embeddings; fusing the user sequence embeddings with the mapped user collaborative embeddings to obtain a user fusion representation; fusing the item sequence embeddings with the mapped item collaborative embeddings to obtain an item fusion representation; converting the user fusion representation into a user prompt embedding and the item fusion representation into an item prompt embedding through a prompt mapping module; inputting the user prompt embeddings, item prompt embeddings, and recommendation task instructions into a large language model to obtain the user's ranking score for the target item; constructing a loss function based on the target item ranking score and updating the model parameters according to the loss function; when the loss function converges, the trained recommendation model is obtained.
[0017] A collaborative sequence recommendation method based on a large language model includes the following steps:
[0018] S1: Collaborative Embedding Learning. Obtain raw user-project interaction data, which includes at least user identifiers, project identifiers, and historical user interactions with projects. In one embodiment, the interaction records include one or more of clicks, views, favorites, add-to-cart, purchases, and ratings. Based on the raw user-project interaction data, train a traditional collaborative model to learn user collaborative embeddings and project collaborative embeddings.
[0019] In one specific embodiment, the traditional collaborative model uses matrix factorization to map users and items into the same latent space and is trained through interaction reconstruction error; in another specific embodiment, the traditional collaborative model uses a BPR model and is trained through the positive and negative sample ranking relationship; in yet another specific embodiment, the traditional collaborative model uses a neural collaborative filtering model to extract more complex nonlinear interaction features.
[0020] S2: Co-embedding Mapping Alignment. Since the co-embedding and the sequence embedding obtained from subsequent sequence encoding may be located in different feature spaces or have different dimensions, the user co-embedding and the item co-embedding are transformed through the mapping alignment module.
[0021] The original user-item interaction data is input into the collaborative embedding module for processing, including: constructing a user-item interaction matrix or interaction sample set based on user and item identifiers; determining sample labels or interaction weights based on interaction behavior type, rating information, or interaction frequency, and negatively sampling non-interacted items to form positive and negative sample pairs; the collaborative embedding module uses any one of matrix factorization, Bayesian personalized ranking, or neural collaborative filtering models to establish user embedding tables and item embedding tables respectively; when using a matrix factorization model, the user-item interaction relationship is reconstructed through the inner product or similarity of user embeddings and item embeddings; when using a Bayesian personalized ranking model, user embeddings and item embeddings are optimized based on the ranking differences between positive and negative sample items corresponding to the same user; when using a neural collaborative filtering model, user embeddings and item embeddings are input into a multilayer perceptron to learn nonlinear interaction features; after training, user collaborative embeddings and item collaborative embeddings are read from the user embedding table and item embedding table respectively.
[0022] In one embodiment, the mapped user collaboration embedding and project collaboration embedding are respectively represented as follows:
[0023] e_u' = W_u e_u^{cf} + b_u
[0024] e_i' = W_i e_i^{cf} + b_i
[0025] Where W_u and W_i are mapping matrices, and b_u and b_i are bias terms. In another embodiment, the mapping alignment module employs a multilayer perceptron to enhance the nonlinear representation capability of the cooperative embedding.
[0026] S3: Construct a sequence of user interaction items. Arrange each user's historical interaction items in chronological order to construct a sequence of user interaction items. Let the sequence of user u's historical interaction items be represented as:
[0027] S_u = {i_1, i_2, …, i_n}
[0028] Where i_k represents the items in user u's k-th interaction, and n represents the number of historical interaction items for that user. In practical applications, the sequence can be truncated, padded, or sampled.
[0029] In this embodiment, mapping user collaborative embeddings and item collaborative embeddings to a unified feature space via a mapping alignment module includes: determining the target dimensions of user sequence embeddings and item sequence embeddings, and setting user-side mapping sub-networks and item-side mapping sub-networks respectively; inputting user collaborative embeddings into user-side mapping sub-networks and item collaborative embeddings into item-side mapping sub-networks, wherein the user-side mapping sub-networks and item-side mapping sub-networks employ at least one of linear transformation layers, multilayer perceptron mapping layers, or fully connected layers with nonlinear activation functions; performing at least one of activation function processing, normalization processing, or discarding processing on the mapping results, so that the mapped user collaborative embeddings and mapped item collaborative embeddings are the same in dimension as user sequence embeddings and item sequence embeddings respectively or compatible in semantic space; wherein the parameters of user-side mapping sub-networks and item-side mapping sub-networks are independent of each other or partially shared, and the mapped user collaborative embeddings and mapped item collaborative embeddings are used for feature fusion by a subsequent fusion module.
[0030] S4: Sequence Encoding Modeling. The sequence of user interaction items is input into the sequence encoding module for modeling. This module employs a Transformer encoder, including an item embedding layer, a position embedding layer, and a multi-layer self-attention network. For the k-th item in the sequence, the item identifier is first mapped to an item vector, then added to the position embedding to obtain the input vector:
[0031] x_k = e_{i_k} + p_k
[0032] Subsequently, the input representation sequence is fed into a multi-layer self-attention network to obtain the hidden state sequence:
[0033] H^l = Encoder(H^{l-1}), l = 1, 2, …, L
[0034] In one embodiment, the user sequence embedding is obtained by average pooling of the last hidden state; in another embodiment, the user sequence embedding is obtained by the hidden state at the last position; in yet another embodiment, the user sequence embedding is obtained by the hidden state corresponding to a specially marked position. The item sequence embedding can be directly extracted from the item embedding matrix, or obtained by inputting the target item into the encoding module.
[0035] S5: Construction of Fusion Representation. The mapped user collaboration embedding and user sequence embedding are fused to obtain the user fusion representation; the mapped item collaboration embedding and item sequence embedding are fused to obtain the item fusion representation.
[0036] The fusion of user sequence embeddings and mapped user collaborative embeddings includes: performing dimensional consistency processing on the user sequence embeddings and the mapped user collaborative embeddings; when using concatenation fusion, concatenating the user sequence embeddings and the mapped user collaborative embeddings and inputting them into a fully connected layer or a nonlinear mapping layer to obtain a user fusion representation; when using weighted summation fusion, linearly combining the two according to preset weights or learnable weights to obtain a user fusion representation; when using gated fusion, generating user gate coefficients based on the user sequence embeddings and the mapped user collaborative embeddings, and using the user gate coefficients to adaptively adjust the proportion of short-term interest features and long-term collaborative preference features of users to obtain a user fusion representation.
[0037] The fusion of the project sequence embedding and the mapped project collaborative embedding includes: performing dimensional consistency processing on the project sequence embedding and the mapped project collaborative embedding; when using concatenation fusion, the project sequence embedding and the mapped project collaborative embedding are concatenated and input into a fully connected layer or a nonlinear mapping layer to obtain a fused project representation; when using weighted summation fusion, the two are linearly combined according to preset weights or learnable weights to obtain a fused project representation; when using gated fusion, project gate coefficients are generated based on the project sequence embedding and the mapped project collaborative embedding, and the proportion of sequence transition features and global collaborative features of the target project is adaptively adjusted using the project gate coefficients to obtain a fused project representation.
[0038] In a preferred embodiment, a splicing method is used for fusion, and the user fusion representation and the project fusion representation are respectively represented as follows:
[0039] e_u = [e_u^{seq} || e_u']
[0040] e_i = [e_i^{seq} || e_i']
[0041] In other embodiments, weighted summation, gating fusion, or attention fusion can be used instead of splicing fusion.
[0042] S6: Prompt Mapping Construction. To avoid the problem of verbose input caused by methods such as CoLLM directly enumerating the user's historical interaction items in the prompt words, this invention uses a prompt mapping module to convert the user fusion representation into a user prompt embedding, and the item fusion representation into an item prompt embedding, and injects user preference information into the large language model in an embedded representation manner.
[0043] The prompt mapping module converts the user fusion representation into a user prompt embedding and the item fusion representation into an item prompt embedding. The prompt mapping module uses any one of vector projection, discretization encoding, soft prompt construction, or prefix prompt construction to convert the user fusion representation into a user prompt embedding and the item fusion representation into an item prompt embedding. The large language model performs recommendation inference based on the user prompt embedding and the item prompt embedding, instead of explicitly enumerating the user's historical interaction items in the prompt text.
[0044] In one embodiment, the prompt mapping module uses a linear projection method to map the user fusion representation and the item fusion representation into consecutive soft prompt vector sequences, respectively:
[0045] P_u = Proj_u(e_u)
[0046] P_i = Proj_i(e_i)
[0047] In another embodiment, the cue mapping module employs a prefix cue construction method, mapping the user fusion representation and the item fusion representation to prefix key-value representations inserted into the multi-layer attention module of the large language model. In yet another embodiment, the cue mapping module uses a discretization encoding method, converting the fusion representation into several discrete cue tags, which are then input into the large language model along with the recommendation task instructions. Regardless of the implementation method, the large language model receives user cue embeddings and item cue embeddings, rather than an explicitly listed historical item list.
[0048] S7: Large Language Model Inference. The user prompt embedding, item prompt embedding, and recommendation task instructions generated by the prompt mapping module are input into the large language model. The recommendation task instructions describe the current recommendation goal, such as "determine if the user is interested in the target item," "predict whether the user will click on the target item," or "rank the candidate items."
[0049] The input of user prompts, item prompts, and recommendation task instructions into the large language model includes: the large language model being either a decoder-based large language model fine-tuned by instructions or an encoder-decoder-based large language model; the recommendation task instructions are used to describe click prediction, purchase prediction, rating prediction, next item recommendation, or ranking recommendation tasks; the large language model outputs the user's ranking score for the target item.
[0050] In one embodiment, the large language model outputs the probability of a user's interaction with a target item; in another embodiment, the large language model outputs the relevance judgment result between the user and the target item; in yet another embodiment, the large language model outputs the ranking score of multiple candidate items. Let the output of the large language model be:
[0051] y_ui = LLM(P_u, P_i, T).
[0052] Where T represents the recommendation task instruction, and ŷ_{ui} represents the prediction result of user u for item i.
[0053] S8: Model Training. A loss function is constructed based on the output of the large language model and the true labels to train the entire model. For click-through rate prediction tasks, the cross-entropy loss function can be used; for ranking and recommendation tasks, the BPR loss function can be used; furthermore, the instruction supervision loss from the large language model output can be introduced to form a joint loss function.
[0054] L_total = λ_1 L_ce + λ_2 L_bpr + λ_3 L_ins + λ_4 ||Θ||_2^2
[0055] Where λ1, λ2, λ3, and λ4 are the weighting coefficients of the corresponding loss terms, L ce For cross-entropy loss, L bpr For Bayesian personalized ranking loss, L ins Let Θ be the instruction supervision loss, and Θ be the set of model parameters, ‖Θ‖2 2 This is the L2 regularization term for the model parameters.
[0056] The parameters of the collaborative embedding module, mapping alignment module, sequence encoding module, fusion module, prompt mapping module, and large language model are updated by backpropagation algorithm. When the preset training rounds or model convergence conditions are met, the trained recommendation model is obtained.
[0057] During the inference phase, a sequence of user interaction items is first constructed based on the target user's historical interaction records. The user fusion representation and item fusion representation are then calculated by the trained co-embedding module and sequence encoding module. Next, the prompt mapping module converts them into user prompt embeddings and item prompt embeddings respectively, and inputs them into the large language model in conjunction with the recommendation task instructions. Finally, the large language model outputs the recommendation score or ranking result of the target item.
[0058] The method proposed in this invention is applicable to various recommendation tasks such as click prediction, purchase prediction, rating prediction, next item recommendation, and ranking recommendation, and has good versatility and engineering application prospects.
[0059] The above-described embodiments further illustrate the purpose, technical solution, and advantages of the present invention. It should be understood that the above-described embodiments are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made to the present invention within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A collaborative sequence recommendation method based on a large language model, characterized in that, include: Obtain user-item interaction data to be recommended; input the user-item interaction data into the trained recommendation model to obtain the user's ranking score for the target item; Items are recommended based on ranking scores; the recommendation model includes a collaborative embedding module, a mapping alignment module, a sequence encoding module, a fusion module, a hint mapping module, and a large language model inference module. Training the recommendation model includes: acquiring raw user-item interaction data; inputting the raw user-item interaction data into a collaborative embedding module to obtain user collaborative embeddings and item collaborative embeddings; mapping the user collaborative embeddings and item collaborative embeddings to a unified feature space through a mapping alignment module; arranging the user's historical interaction items in chronological order to construct a user interaction item sequence; inputting the user interaction item sequence into a sequence encoding module for modeling to obtain user sequence embeddings and item sequence embeddings; fusing the user sequence embeddings with the mapped user collaborative embeddings to obtain a user fusion representation; fusing the item sequence embeddings with the mapped item collaborative embeddings to obtain an item fusion representation; converting the user fusion representation into a user prompt embedding and the item fusion representation into an item prompt embedding through a prompt mapping module; inputting the user prompt embeddings, item prompt embeddings, and recommendation task instructions into a large language model to obtain the user's ranking score for the target item; constructing a loss function based on the target item ranking score and updating the model parameters according to the loss function; when the loss function converges, the trained recommendation model is obtained.
2. The collaborative sequence recommendation method based on a large language model according to claim 1, characterized in that, User-project interaction data includes at least one of the following: user identifier, project identifier, interaction behavior type, interaction time, rating information, and project text attributes; the user interaction project sequence is constructed in ascending or descending order based on the interaction occurrence time.
3. The collaborative sequence recommendation method based on a large language model according to claim 1, characterized in that, The original user-item interaction data is input into the collaborative embedding module for processing, including: constructing a user-item interaction matrix or interaction sample set based on user and item identifiers; determining sample labels or interaction weights based on interaction behavior type, rating information, or number of interactions, and negatively sampling non-interactive items to form positive and negative sample pairs; and inputting the positive and negative sample pairs into the collaborative embedding module to establish user embedding tables and item embedding tables.
4. The collaborative sequence recommendation method based on a large language model according to claim 1, characterized in that, Mapping user collaborative embeddings and item collaborative embeddings to a unified feature space via a mapping alignment module includes: determining the target dimensions of user sequence embeddings and item sequence embeddings, and setting user-side mapping sub-networks and item-side mapping sub-networks respectively. The user-side mapping sub-networks and item-side mapping sub-networks employ at least one of linear transformation layers, multilayer perceptron mapping layers, or fully connected layers with nonlinear activation functions; inputting user collaborative embeddings into user-side mapping sub-networks and item collaborative embeddings into item-side mapping sub-networks, and performing activation function processing and normalization processing on the mapping results to ensure that the mapped user collaborative embeddings and mapped item collaborative embeddings are dimensionally identical to the user sequence embeddings and item sequence embeddings, respectively.
5. The collaborative sequence recommendation method based on a large language model according to claim 1, characterized in that, The process of inputting the user interaction item sequence into the sequence encoding module for modeling includes: the sequence encoding module is a Transformer encoder, and each item in the user interaction item sequence is first mapped into an item vector through the item embedding layer of the Transformer encoder; the item vectors are then pooled and weighted and aggregated; the aggregated sequence and position are then embedded into a multi-layer self-attention network for modeling to extract the temporal evolution features of user interests.
6. The collaborative sequence recommendation method based on a large language model according to claim 1, characterized in that, The fusion of user sequence embeddings and mapped user collaborative embeddings includes: performing dimensionality consistency processing on the user sequence embeddings and mapped user collaborative embeddings; when using concatenation fusion, the user sequence embeddings and mapped user collaborative embeddings are concatenated and input into a fully connected layer or a nonlinear mapping layer to obtain a user fusion representation; when using weighted summation fusion, the two are linearly combined according to preset weights or learnable weights to obtain a user fusion representation; when using gated fusion, user gate coefficients are generated based on the user sequence embeddings and mapped user collaborative embeddings, and the proportion of short-term interest features and long-term collaborative preference features of users is adaptively adjusted using the user gate coefficients to obtain a user fusion representation.
7. The collaborative sequence recommendation method based on a large language model according to claim 1, characterized in that, The fusion of project sequence embeddings and mapped project collaborative embeddings includes: performing dimensional consistency processing on the project sequence embeddings and mapped project collaborative embeddings; when using concatenation fusion, the project sequence embeddings and mapped project collaborative embeddings are concatenated and input into a fully connected layer or a nonlinear mapping layer to obtain a fused project representation; when using weighted summation fusion, the two are linearly combined according to preset weights or learnable weights to obtain a fused project representation; when using gated fusion, project gate coefficients are generated based on the project sequence embeddings and mapped project collaborative embeddings, and the proportion of sequence transition features and global collaborative features of the target project is adaptively adjusted using the project gate coefficients to obtain a fused project representation.
8. The collaborative sequence recommendation method based on a large language model according to claim 1, characterized in that, The prompt mapping module converts the user fusion representation into a user prompt embedding and the item fusion representation into an item prompt embedding. This includes: the prompt mapping module using any one of vector projection, discretization coding, soft prompt construction, or prefix prompt construction; and using the prompt mapping module to convert the user fusion representation into a user prompt embedding and the item fusion representation into an item prompt embedding.
9. The collaborative sequence recommendation method based on a large language model according to claim 1, characterized in that, The user prompts, item prompts, and recommendation task instructions are all input into the large language model. The large language model can be either a decoder-based large language model or an encoder-decoder-based large language model that has been fine-tuned by instructions. The recommendation task instructions are used to describe click prediction, purchase prediction, rating prediction, and next item recommendation. The large language model outputs the user's ranking score for the target item.
10. The collaborative sequence recommendation method based on a large language model according to claim 1, characterized in that, The model's loss function is: L total = λ1L ce + λ2 L bpr + λ3 L ins + λ4 ||Θ||2 2 ; Where λ1, λ2, λ3, and λ4 are the weighting coefficients of the corresponding loss terms, L ce For cross-entropy loss, L bpr For Bayesian personalized ranking loss, L ins Let Θ be the instruction supervision loss, and Θ be the set of model parameters, ‖Θ‖2 2 This is the L2 regularization term for the model parameters.