Multi-behavior generation type recommendation method based on multi-granularity sequence modeling
By employing a multi-granularity sequence modeling method, combining product-level tokens with a behavior-category related mapping layer, the redundancy and inefficiency issues of multiple behavior sequences in generative recommendation systems are resolved, thereby improving recommendation performance and accuracy.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-08
- Publication Date
- 2026-04-03
AI Technical Summary
Existing generative recommendation systems suffer from problems such as sequence redundancy and low computational efficiency, single and coarse modeling granularity, and shallow utilization of behavioral information when processing multi-behavioral sequences, making it difficult to effectively capture the dependencies and temporal patterns of multiple user behaviors.
A multi-granularity sequence modeling approach is adopted, which performs attention modeling by aggregating tokens at the item level and combining them with a linear mapping layer related to behavior categories, deeply fusing item and behavior sequences to generate user interest representations.
It improves the model's expressive power and recommendation performance, enabling it to more accurately predict users' target interaction objects under various behavioral conditions, and increases the normalized loss cumulative gain and click-through rate.
Smart Images
Figure CN121786779A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of recommender system technology, specifically a multi-behavior generative recommender method based on multi-granularity sequence modeling. Background Technology
[0002] Thanks to breakthroughs in sequence modeling using large language models, generative recommender systems have become a cutting-edge research area in recommender systems. Unlike traditional cascaded recommender systems, generative recommender systems, as a unified end-to-end architecture, break the constraints between upper and lower recommendation stages in cascaded recommender systems. They can leverage their powerful sequence modeling capabilities and high computational scalability to improve recommender performance by processing longer sequences of user behavior.
[0003] However, existing research on generative recommender systems, in its evolution from traditional models, generally inherits a key limitation: the neglect of the diversity of user behavior. In real-world applications (such as e-commerce and content platforms), user behavior is rich and varied, including clicks, favorites, adding to cart, and purchases. These different types of behavior, with their varying degrees of intent and informational value, are crucial for accurately characterizing users' dynamic interests. Therefore, effectively integrating multi-behavioral information into the generative recommender architecture is a vital and urgent problem to be solved.
[0004] Currently, there are two main approaches to existing technical solutions for modeling multi-behavioral sequences under a generative architecture: Explicit Insertion Method: This method treats behavior categories as independent tokens and explicitly inserts them into the user's item sequence to form a mixed sequence, which is then modeled using a standard token-by-token prediction method.
[0005] Implicit fusion method: This method does not change the sequence structure, but implicitly integrates the representation of behavior types into the representation of items through embedding addition or vector shifting, thereby indirectly guiding the model to make predictions.
[0006] However, the above-mentioned existing technical solutions have the following obvious drawbacks: Sequence redundancy and low computational efficiency: Explicit insertion significantly increases the length of the input sequence (usually doubling it), which brings huge and unnecessary computational overhead to Transformer-based generative models, seriously affecting the training and inference efficiency of the model.
[0007] The modeling granularity is singular and coarse: Existing methods generally only perform flattened attention modeling at the token level, failing to differentiate granularity at the item level, which is more in line with business logic. This makes it difficult for the model to capture the overall semantics of multiple tokens within the same item, as well as the evolutionary relationship of the item as a whole unit in the sequence.
[0008] Superficial utilization of behavioral information: Whether it is explicit insertion or implicit fusion, the utilization of behavioral type information is relatively superficial. They do not model the behavioral sequence itself as an independent sequence rich in user intent information, resulting in the incomplete exploration and utilization of the dependencies and temporal patterns between behaviors.
[0009] Based on the above analysis, there is an urgent need in this field for a new method for multi-behavioral sequence modeling that can achieve multi-granularity, high efficiency, and deep integration within a generative recommendation framework. Summary of the Invention
[0010] The problem addressed by this invention is how to achieve a novel method for multi-behavioral sequence modeling that is multi-granular, highly efficient, and deeply integrated.
[0011] To address the aforementioned issues, this invention provides a multi-behavior generative recommendation method, recommendation system, electronic device, and storage medium based on multi-granularity sequence modeling.
[0012] In a first aspect, the present invention provides a multi-behavior generative recommendation method based on multi-granularity sequence modeling, comprising the following steps: Obtain the user's historical behavior sequence, which includes multiple interactive items and their corresponding behavior categories, wherein each interactive item belongs to an item set and each behavior category belongs to a behavior category set. Each interactive item is vector-quantized to generate an item token sequence; The behavior type sequence is tokenized to generate a behavior type token sequence; A hierarchical modeling strategy is adopted. First, multiple tokens of the same item are aggregated at the item level to obtain an item-level representation sequence. Based on the item-level representation sequence, combined with behavior category information, attention modeling between item sequences is performed through a behavior category-related linear mapping layer to obtain item sequence representations. The behavior type token sequence is modeled separately to obtain the behavior type sequence representation; The item sequence representation is fused with the behavior category sequence representation to generate a user interest representation; Based on the user interest representation, a target item token sequence is generated autoregressively through a decoder to predict the item the user will interact with next.
[0013] Optionally, the step of vector quantization for each interactive item to generate an item token sequence includes: Each item is quantized into a discrete sequence of tokens, where the length of the token sequence is a fixed value.
[0014] Optionally, the aggregation of multiple tokens for the same item at the item level includes: Model multiple tokens for the same item using a self-attention mechanism; The last token representation from the attention mechanism output is taken as the aggregated result for that item.
[0015] Optionally, the attention modeling between item sequences through a linear mapping layer related to behavior categories includes: For each type of behavior, define a query linear mapping layer, a key linear mapping layer, and a value linear mapping layer respectively; The linear mapping layer is used to process the item-level representation sequence to generate a query matrix, a key matrix, and a value matrix; When calculating the attention matrix, a linear mapping layer associated with behavior type pairs is introduced to characterize the dependencies between different behavior types.
[0016] Optionally, the formula for calculating the attention matrix is: ; In the formula, Represents the elements of the attention matrix. Indicates querying matrix elements, Represents the key matrix elements, Representation of the relationship between the type of behavior Related linear mapping layer parameters.
[0017] Optionally, when aggregating the value matrix to represent the generated item sequence, a linear mapping layer related to the behavior category is further introduced, wherein the aggregation formula is: ; In the formula, Represents the sequence of items. Represents the elements of a value matrix. This represents the parameters of the aggregated linear mapping layer associated with the behavior type.
[0018] Optionally, the separate modeling of the behavior type token sequence includes: The behavior type token sequence is encoded using a converter model to obtain a behavior type sequence representation.
[0019] Optionally, the fusion of item sequence representation and behavior category sequence representation includes: Add the item sequence representation to the behavior category sequence representation; The summation result is input into a multilayer perceptron to generate a user interest representation.
[0020] Optionally, the step of generating the target item token sequence autoregressively via the decoder includes: User interest representations are used as input to the decoder; The model is trained using the cross-entropy loss function, where the loss function is calculated as follows: ; In the formula, For sequence length, For the model in the first The probability of correctly predicting a location.
[0021] Secondly, embodiments of the present invention provide a multi-behavior generative recommendation system based on multi-granularity sequence modeling, comprising: The sequence acquisition module 101 is used to acquire the user's historical behavior sequence, which consists of a series of interactive items and their corresponding behavior types. The tokenization module 102 is used to perform vector quantization on the interactive items to generate an item token sequence, and to tokenize the behavior types to generate a behavior type token sequence. Layered modeling module 103 includes: The item-level aggregation unit is used to aggregate multiple tokens of the same item to generate an item-level representation sequence. The item sequence modeling unit is used to perform attention calculation based on the item-level representation sequence and combined with a linear mapping layer related to behavior type to obtain the item sequence representation; A behavior sequence modeling unit is used to model the behavior type token sequence separately to obtain a behavior type sequence representation; The fusion module 104 is used to fuse the item sequence representation with the behavior category sequence representation to generate a user interest representation; The prediction module 105 is used to generate a target item token sequence autoregressively through a decoder, using the user interest representation as input, to predict the user's next interactive item.
[0022] Thirdly, embodiments of the present invention provide an electronic device, including a processor, a communication interface, a memory, and a bus, wherein the processor, the communication interface, and the memory communicate with each other through the bus, and the processor can call logical instructions in the memory to execute the steps of the method provided in the first aspect.
[0023] Fourthly, embodiments of the present invention provide a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the multi-behavior generative recommendation method based on multi-granularity sequence modeling as described in the first aspect.
[0024] The beneficial effects of the multi-behavior generative recommendation method based on multi-granularity sequence modeling of the present invention are as follows: This method aggregates multiple tokens for a single item at the item level and then models attention for both item sequences and behavior sequences separately. Furthermore, when modeling attention scores between items, it deeply incorporates multi-behavioral information by using parameters such as a linear mapping layer related to behavior types, enhancing the model's expressive power and recommendation performance. Compared to existing technologies, this model offers a certain improvement in recommendation performance. Specifically, because this method deeply integrates user behavior information at multiple granularities, it can more profoundly characterize user interest representations, thereby more accurately predicting the target interaction objects of users under various behavioral conditions. From a metrics perspective, this method shows improvements over existing technologies in terms of normalized depreciation cumulative gain and click-through rate on public datasets. Attached Figure Description
[0025] Figure 1 This is a flowchart of a multi-behavior generative recommendation method based on multi-granularity sequence modeling in an embodiment of the present invention; Figure 2 This is a structural block diagram of a multi-behavior generative recommendation system based on multi-granularity sequence modeling in an embodiment of the present invention; Figure 3 This is a structural block diagram of the electronic device in an embodiment of the present invention. Detailed Implementation
[0026] To better understand the purpose, technical solution, and advantages of this application, the application is described and explained below in conjunction with the accompanying drawings and embodiments.
[0027] Unless otherwise defined, the technical or scientific terms used in this application shall have the general meaning understood by one of ordinary skill in the art to which this application pertains. Words such as “a,” “an,” “an,” “the,” “the,” and “these” used in this application do not indicate quantitative limitation and may be singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that comprises a series of steps or modules (units) is not limited to the listed steps or modules (units) but may include steps or modules (units) not listed, or may include other steps or modules (units) inherent to these processes, methods, products, or devices. Words such as “connected,” “linked,” and “coupled” used in this application are not limited to physical or mechanical connections but may include electrical connections, whether direct or indirect. “Multiple” used in this application refers to two or more. “And / or” describes the relationship between related objects, indicating that three relationships may exist; for example, “A and / or B” can represent: A alone, A and B simultaneously, and B alone. Normally, the character " / " indicates that the objects before and after it are in an "or" relationship. The terms "first," "second," "third," etc., used in this application are merely to distinguish similar objects and do not represent a specific order of objects.
[0028] like Figure 1 As shown in the embodiment of the present invention, a multi-behavior generative recommendation method based on multi-granularity sequence modeling includes the following steps: S1. Obtain the user's historical behavior sequence. The user's historical behavior sequence includes multiple interactive items and their corresponding behavior types. Each interactive item belongs to an item set, and each behavior type belongs to a behavior type set. Specifically, based on a given user's historical behavior sequence (including interacted items and behavior types), predict the item the user is most likely to interact with next. This embodiment uses... Represents an independent set of users. Represents the set of all items. Represents a set of behavior types.
[0029] S2. Perform vector quantization on each interactive item to generate an item token sequence; For each interactive item, perform vector quantization to generate an item token sequence, including: Each item is quantized into a discrete sequence of tokens, where the length of the token sequence is a fixed value.
[0030] S3. Tokenize the behavior type sequence to generate a behavior type token sequence; Specifically, a user behavior sequence is defined as This method predicts the user's last interaction. To conform to the paradigm of generative recommendation architecture, each item needs to be vectorized. That is, each item... Quantized into discrete token sequences ,in To quantize the token sequence length. The quantized interactive item sequence is (in...). (For example) ; Similarly, for a sequence of behavior types, this method will treat each behavior type as a token and construct a sequence of behavior type tokens: ; S4. Using a hierarchical modeling strategy, firstly, multiple tokens of the same item are aggregated at the item level to obtain an item-level representation sequence. Aggregating multiple tokens for the same item at the item level, including: Model multiple tokens for the same item using a self-attention mechanism; The last token representation from the attention mechanism output is taken as the aggregated result for that item.
[0031] Specifically, in modeling item sequences, this embodiment adopts a hierarchical modeling scheme. First, at the item level, multiple tokens of the same item are aggregated to obtain: ; Among them, for , ,in For the aggregation function, a self-attention mechanism is used here, and the last token representation after attention modeling is taken as the aggregation result: ; ; S5. Based on the item-level representation sequence and combined with behavior category information, attention modeling between item sequences is performed through a linear mapping layer related to behavior categories to obtain the item sequence representation. Attention modeling between item sequences is performed using a linear mapping layer related to behavior types, including: For each type of behavior, define a query linear mapping layer, a key linear mapping layer, and a value linear mapping layer respectively; The item-level representation sequence is processed using a linear mapping layer to generate a query matrix, a key matrix, and a value matrix; When calculating the attention matrix, a linear mapping layer associated with behavior type pairs is introduced to characterize the dependencies between different behavior types.
[0032] Specifically, in this embodiment, in the second layer, modeling is performed between items in the item sequence, and behavior type information is incorporated into the calculation process of the self-attention mechanism. For each behavior type... Construct three linear mapping layers To get matrix: ; ; ; Then, calculate the attention matrix: ; In the formula, Represents the elements of the attention matrix. Indicates querying matrix elements, Represents the key matrix elements, Representation of the relationship between the type of behavior Related linear mapping layer parameters.
[0033] This includes introducing a linear mapping layer related to behavior type pairs. To characterize the dependency relationship between the two behaviors.
[0034] Finally, the aggregation matrix Obtain item sequence representation : ； In the formula, Represents the sequence of items. Represents the elements of a value matrix. This represents the parameters of the aggregated linear mapping layer associated with the behavior type.
[0035] A linear mapping layer related to behavior type pairs is also introduced in the aggregation phase. This is used to characterize the dependency relationship between the two behaviors during aggregation.
[0036] S6. Model the behavior type token sequence separately to obtain the behavior type sequence representation; Separate modeling of behavior-type token sequences includes: The behavior type token sequence is encoded using a converter model to obtain a behavior type sequence representation.
[0037] Specifically, for the behavior category token sequence, this embodiment directly uses a simple converter to model it, obtaining the behavior category representation. : ; S7. Integrate the item sequence representation with the behavior category sequence representation to generate a user interest representation; The fusion of item sequence representation and behavior category sequence representation includes: Add the item sequence representation to the behavior category sequence representation; The summation result is input into a multilayer perceptron to generate a user interest representation.
[0038] Specifically, after obtaining the item sequence representation and behavior type sequence representation Then, these sums are passed through a multilayer perceptron to obtain a user interest representation. : ; S8. Based on user interest representation, a target item token sequence is generated autoregressively through a decoder to predict the item the user will interact with next.
[0039] In this embodiment, the target item token sequence is generated autoregressively using a decoder, including: User interest representations are used as input to the decoder; The model is trained using the cross-entropy loss function.
[0040] Specifically, after obtaining the user representation, the target item token sequence is generated autoregressively in the decoder section using the user representation as input: ; The loss function L is calculated using the following formula: ; in, For sequence length, For the first The model predicts the probability score of the correct location, and finally uses gradient descent to train the model on the loss function L.
[0041] In summary, this invention, based on a generative recommendation architecture, first aggregates multiple tokens for the same item at the item level, and then models attention for both item sequences and behavior sequences separately. Furthermore, when modeling attention scores between items, it deeply incorporates multi-behavioral information by using parameters such as a linear mapping layer related to behavior types, enhancing the model's expressive power and recommendation performance. Compared to existing technologies, this model offers a certain improvement in recommendation performance. Specifically, because this method deeply integrates user behavior information at multiple granularities, it can more profoundly characterize user interest representations, thereby more accurately predicting the target interaction objects of users under various behavioral conditions. From a metrics perspective, this method shows certain improvements over existing technologies in terms of normalized depreciation cumulative gain and click-through rate on public datasets.
[0042] This invention also provides a multi-behavior generative recommendation system based on multi-granularity sequence modeling. This system is used to implement the above-described method embodiments, and details already described will not be repeated. The terms "module," "unit," and "subunit," etc., used below refer to combinations of software and / or hardware that achieve a predetermined function. Although the system described in the following embodiments is preferably implemented in software, hardware implementation or a combination of software and hardware is also possible and contemplated.
[0043] like Figure 2 As shown, Figure 2 This is a structural block diagram of the multi-behavior generative recommender system based on multi-granularity sequence modeling in this invention. The system includes: The sequence acquisition module 101 is used to acquire the user's historical behavior sequence, which consists of a series of interactive items and their corresponding behavior types. The tokenization module 102 is used to perform vector quantization on the interactive items to generate an item token sequence, and to tokenize the behavior types to generate a behavior type token sequence. Layered modeling module 103 includes: The item-level aggregation unit is used to aggregate multiple tokens of the same item to generate an item-level representation sequence. The item sequence modeling unit is used to perform attention calculation based on the item-level representation sequence and combined with a linear mapping layer related to behavior type to obtain the item sequence representation; A behavior sequence modeling unit is used to model the behavior type token sequence separately to obtain a behavior type sequence representation; The fusion module 104 is used to fuse the item sequence representation with the behavior category sequence representation to generate a user interest representation; The prediction module 105 is used to generate a target item token sequence autoregressively through a decoder, using the user interest representation as input, to predict the user's next interactive item.
[0044] like Figure 3 As shown in the figure, an electronic device provided by an embodiment of the present invention includes: a processor 610, a communication interface 620, a memory 630, and a communication bus 640, wherein the processor 610, the communication interface 620, and the memory 630 communicate with each other through the communication bus 640. The processor 610 can call logical instructions in the memory 630 to execute the following method: Obtain the user's historical behavior sequence, which includes multiple interactive items and their corresponding behavior categories, wherein each interactive item belongs to an item set and each behavior category belongs to a behavior category set. Each interactive item is vector-quantized to generate an item token sequence; The behavior type sequence is tokenized to generate a behavior type token sequence; A hierarchical modeling strategy is adopted. First, multiple tokens of the same item are aggregated at the item level to obtain an item-level representation sequence. Based on the item-level representation sequence, combined with behavior category information, attention modeling between item sequences is performed through a behavior category-related linear mapping layer to obtain item sequence representations. The behavior type token sequence is modeled separately to obtain the behavior type sequence representation; The item sequence representation is fused with the behavior category sequence representation to generate a user interest representation; Based on the user interest representation, a target item token sequence is generated autoregressively through a decoder to predict the item the user will interact with next.
[0045] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0046] This invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, is implemented to perform the methods provided in the above embodiments.
[0047] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of various embodiments or some parts of embodiments.
[0048] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A multi-behavior generative recommendation method based on multi-granularity sequence modeling, characterized in that, include: Obtain the user's historical behavior sequence, which includes multiple interactive items and their corresponding behavior categories, wherein each interactive item belongs to an item set and each behavior category belongs to a behavior category set. Each interactive item is vector-quantized to generate an item token sequence; The behavior type sequence is tokenized to generate a behavior type token sequence; A hierarchical modeling strategy is adopted. First, multiple tokens of the same item are aggregated at the item level to obtain an item-level representation sequence. Based on the item-level representation sequence, combined with behavior category information, attention modeling between item sequences is performed through a behavior category-related linear mapping layer to obtain item sequence representations. The behavior type token sequence is modeled separately to obtain the behavior type sequence representation; The item sequence representation is fused with the behavior category sequence representation to generate a user interest representation; Based on the user interest representation, a target item token sequence is generated autoregressively through a decoder to predict the item the user will interact with next.
2. The method according to claim 1, characterized in that, The step of vector quantization for each interactive item to generate an item token sequence includes: Each item is quantized into a discrete sequence of tokens, where the length of the token sequence is a fixed value.
3. The method according to claim 1, characterized in that, The aggregation of multiple tokens for the same item at the item level includes: Model multiple tokens for the same item using a self-attention mechanism; The last token representation from the attention mechanism output is taken as the aggregated result for that item.
4. The method according to claim 1, characterized in that, The attention modeling between item sequences through a linear mapping layer related to behavior types includes: For each type of behavior, define a query linear mapping layer, a key linear mapping layer, and a value linear mapping layer respectively; The linear mapping layer is used to process the item-level representation sequence to generate a query matrix, a key matrix, and a value matrix; When calculating the attention matrix, a linear mapping layer associated with behavior type pairs is introduced to characterize the dependencies between different behavior types.
5. The method according to claim 4, characterized in that, The formula for calculating the attention matrix is: ； In the formula, Represents the elements of the attention matrix. Indicates querying matrix elements, Represents the key matrix elements, Representation of the relationship between the type of behavior Related linear mapping layer parameters.
6. The method according to claim 4, characterized in that, When the aggregated value matrix is used to represent the generated item sequence, a further linear mapping layer related to the behavior category is introduced, and the aggregation formula is: ； In the formula, Represents the sequence of items. Represents the elements of a value matrix. This represents the parameters of the aggregated linear mapping layer associated with the behavior type.
7. The method according to claim 1, characterized in that, The separate modeling of the token sequences for each behavior type includes: The behavior type token sequence is encoded using a converter model to obtain a behavior type sequence representation.
8. The method according to claim 1, characterized in that, The fusion of item sequence representation and behavior category sequence representation includes: Add the item sequence representation to the behavior category sequence representation; The summation result is input into a multilayer perceptron to generate a user interest representation.
9. The method according to claim 1, characterized in that, The process of generating the target item token sequence autoregressively via a decoder includes: User interest representations are used as input to the decoder; The model is trained using the cross-entropy loss function, where the loss function is calculated as follows: ； In the formula, For sequence length, For the model in the first The probability of correctly predicting a location.
10. An electronic device, characterized in that, The system includes a memory and a processor, the memory storing a computer program, characterized in that the processor executes the computer program to implement the method as described in any one of claims 1 to 9.