Single-stage generative sequence recommendation method

Abstract

The invention discloses a single-stage generative sequence recommendation method, which belongs to the field of personalized recommendation systems, and comprises the following steps: S1, item code representation: performing formalized representation on items according to attribute granularity, and generating code representation of the items through a large language model and a product quantization technology; s2, constructing a user interaction sequence: splitting the user interaction sequence into a plurality of attributes to form an interaction sequence consisting of project attributes; s3, generative sequence recommendation: improving an encoder architecture of Transform, including a sequence projection layer, a spatial aggregation layer and a conversion layer, so as to adapt to a sequence recommendation task; s4, recommendation item prediction: based on the user interaction sequence, learning probability distribution, and generating prediction of a next interaction item; according to the method, prediction of the next item possibly interacted by the user can be directly generated, similarity query does not need to be carried out in the candidate set, and the sequence recommendation effect is effectively improved.

Description

Technical Field

[0001] The present invention belongs to the field of personalized recommendation systems, and particularly relates to a single-stage generative sequential recommendation method. Background Art

[0002] With the development of the Internet and information technology, the amount of information in online services such as social media, news media, and e-commerce platforms has increased exponentially. Faced with a vast amount of information, individual users can hardly process all the information they may encounter, thus facing the problem of information overload. The existence of recommendation systems is to help users filter out low-quality or personally-unrelated information in a short time.

[0003] The sequential recommendation task focuses on modeling the user profile through the user's interaction sequence, and improving the recommendation effect by capturing the user's time-varying interests. Its problem definition can be simply described as: predicting the items that the user may interact with at the next moment based on the user's historical behavior. Therefore, how to efficiently mine the user's preferences has become a difficult point in the research of sequential recommendation algorithms.

[0004] Most traditional recommendation model architectures are a two-stage process: first, the model learns the user's personalized preferences to obtain an estimated vector in the feature space, and then performs similarity matching in the candidate set to select the top N items with the highest scores as the recommendation results. In the industrial field, this process is usually completed in the model ranking stage, that is, scoring the candidates and outputting the items with higher scores. If the number of items is too large, the model ranking stage will be further divided into a rough ranking and a fine ranking stage. The generative sequential recommendation model is expected to optimize the above two-stage process into a single-stage process. After learning the personalized preferences on the user's interaction sequence, the recommendation results are gradually generated without the need to perform operations such as similarity calculation or item scoring.

[0005] Although there are already a few works that have designed generative recommendation methods, the existing technologies still have limitations. For example, the GPTRec sequential recommendation model is proposed in the existing technology. This model can obtain the user and item embedding vectors through SVD decomposition based on the interaction matrix between the user and the item, and then quantize the item embedding vectors to split the item IDs, so that each item is represented by multiple tokens. The item sequence is input into GPTRec, and after training the model, the model generates the recommendation results. In addition, a new Next-K recommendation strategy is also proposed in the existing technology. This strategy takes into account the items that have been recommended and generates recommendations item by item. The Next-K strategy can be used to generate complex interdependent recommendation lists. However, the item representation in the existing technology still comes from the decomposition of the interaction matrix, only solving the generative recommendation method, and not understanding the sequential recommendation task based on the semantic information of the items. Summary of the Invention

[0006] In view of the problems mentioned in the background art, the present invention proposes a single-stage generative sequence recommendation method to improve the existing recommendation algorithm structure and enhance the recommendation effect of the sequence recommendation algorithm.

[0007] Technical solution: To solve the above technical problems, the technical solution adopted by the present invention is as follows:

[0008] A single-stage generative sequence recommendation method, comprising the following steps:

[0009] S1: Item encoding representation: Represent items in the form of attribute granularity and generate the encoding representation of items through a large language model and product quantization technology;

[0010] S2: User interaction sequence construction: Split the user interaction sequence into multiple attributes to form an interaction sequence composed of item attributes;

[0011] S3: Generative sequence recommendation: Improve the encoder architecture of Transformer, including a sequence projection layer, a spatial aggregation layer, and a conversion layer, to adapt to the sequence recommendation task;

[0012] S4: Recommended item prediction: Based on the user interaction sequence, learn the probability distribution to generate the prediction of the next interaction item.

[0013] Preferably, the specific process of S1 is:

[0014] S11: Split the item into multiple groups of text tokens according to attribute granularity and input them into the large language model to generate the original embedding vectors;

[0015] The item is formalized in the form of attribute granularity as:

[0016] ,

[0017] where i represents the item; , , all represent attributes;

[0018] Split the text information in the attribute into the smallest processable unit token, and the attribute is specifically represented as:

[0019] ,

[0020] where represents the length of the attribute ; represents the first token in the attribute , represents the th token in the attribute;

[0021] Then, input the text information in all attributes into the large language model to obtain the original embedding vectors; the text information in the attributes The original embedding vectors obtained after the large language model processes the text information are expressed as:

[0022] ,

[0023] wherein, represents inputting the text information in the attribute into the large language model; d represents the dimension of the original vector .

[0024] Preferably, S12: Use product quantization to encode the original embedding vectors to generate integer codewords;

[0025] The encoding operation on the original embedding vectors is expressed as:

[0026] ,

[0027] wherein, , both represent using product quantization to encode the original embedding vectors ; m represents the number of codewords obtained by encoding in the product quantization process.

[0028] Preferably, S13: Introduce the codeword offset method to ensure the mutual exclusivity between codewords;

[0029] Suppose the model introduces a total of attributes, and each attribute uses codewords for encoding. Starting from the first codeword of the first attribute, each codeword is offset according to the following formula, specifically:

[0030] ,

[0031] wherein, represents the th codeword in the th attribute. The on the right side of the equation is the original codeword after product quantization, and the left side is the new codeword after offset;

[0032] After adapting the project text information, the large language model, and the product quantization technology to the generative model, the encoded representation of each project can be obtained, specifically:

[0033] ,

[0034] wherein, represents the encoded representation of project I Indicates the th codeword in the th attribute, The th codeword in the th attribute, Indicates the th codeword in the th attribute.

[0035] Preferably, in S2, each item is split into multiple attributes in place, and the original interaction sequence composed of items is replaced by an interaction sequence composed of item attributes. The encoding representation of the interaction sequence composed of item attributes is:

[0036] ,

[0037] where represents the encoding of the user interaction sequence; A represents the attribute granularity; u represents the user; represents the encoding of the attribute in the first item; represents the encoding of the attribute in the first item; represents the encoding of the attribute in the first item; represents the number of items in the interaction sequence; I represents the item.

[0038] Preferably, in S3, the sequence projection layer is based on the three classic matrices of the self-attention mechanism , and introduces the matrix U to compress the long-term historical behavior sequence information and extract the features of the user on the attribute granularity sequence. The formal representation of the sequence projection layer is:

[0039] ,

[0040] where X represents the input sequence, represents the matrix, represents the value matrix; represents the query matrix; represents the key matrix; represents the input representation of each learning module, is a multi-layer perceptron; is the activation function selected in the model, represents splitting the output result into four sub-vectors.

[0041] Preferably, in S3, a new aggregation attention mechanism is constructed in the spatial aggregation layer. The formalization of the spatial aggregation layer is:

[0042] ,

[0043] Among them, X represents the input sequence, represents the value matrix; represents the transpose operation; is the activation function selected in the model; and respectively represent the attention biases of time and position information, represents the scaling operation.

[0044] Preferably, in S3, the conversion layer can play the roles of feature extraction, feature crossing, and representation conversion, and the specific formal representation is:

[0045] ,

[0046] Among them, X represents the input sequence, represents a single-layer linear layer, represents the layer normalization operation; represents the formalization of the conversion layer; represents the formalization of the spatial aggregation layer; represents the exclusive NOR operation; represents the matrix;

[0047] The feature extraction function is to convert the original attribute information into an attention-aggregated embedding vector after being processed by each layer of the network;

[0048] Feature crossing is to directly interact the output of the attention layer with other features;

[0049] Representation conversion is to virtually perform a gating operation through element-wise dot product.

[0050] Preferably, in S4, the user sequence encoding is embedded, specifically:

[0051] ,

[0052] Among them, represents the user interaction sequence encoding after embedding processing; represents the embedding operation; represents the encoding of the user interaction sequence;

[0053] The objective of the generative sequence recommendation model is expressed as:

[0054] ,

[0055] Among them, represents at the user interaction sequence at time The codeword predicted by the representation model at time t+1; K represents the vocabulary; p represents the probability;

[0056] Probability value is calculated by performing a softmax operation on the output of the stacked learning module and each element in the vocabulary, specifically as follows:

[0057] ,

[0058] where, represents the user interaction sequence at time, represents the codeword predicted by the model for the next time; represents the prediction result output after stacking layers of learning modules, represents the vocabulary the embedding vectors of all codewords in.

[0059] Beneficial effects: Compared with the prior art, the present invention has the following advantages:

[0060] (1) The present invention constructs a single-stage generative sequence recommendation algorithm, which can directly generate predictions for the next item that the user may interact with, without performing similarity queries in the candidate set, effectively improving the effect of sequence recommendation.

[0061] (2) The present invention is a generative sequence recommendation algorithm based on text information. The algorithm first designs a method for codeword offset for the item coding result obtained by product quantization to meet the requirements of generative model training. Then, the user's item interaction sequence is unfolded according to the attribute dimension, because the generative model can notice the weights of different attributes on the interaction sequence at the single attribute granularity. A stackable generative learning module is also constructed in the model, as the core of the model, to capture the user's preferences from the user's interaction sequence, and finally generate the recommendation result. Finally, through experiments, it is proved that the single-stage generative sequence recommendation model has the potential to transform the recommendation paradigm.

[0062] (3) Most traditional recommendation model architectures are a two-stage process: first, the model learns the personalized preferences of users to obtain an estimated vector in the feature space, and then performs similarity matching in the candidate set to select the top N items with the highest scores as the recommendation results. In the industrial field, candidates are also scored during the model ranking stage to output items with high scores. If the number of items is too large, the model ranking stage will be further divided into a rough ranking and a fine ranking stage. The generative sequential recommendation model is expected to optimize the above two-stage process into a single-stage process. After learning personalized preferences on the user's interaction sequence, the recommendation results are gradually generated without the need for operations such as similarity calculation or item scoring. Description of the Drawings

[0063] Figure 1 is a flowchart of the single-stage generative sequential recommendation method of the present invention;

[0064] Figure 2 is an architecture diagram of the single-stage generative sequential recommendation method of the present invention. Specific Embodiments

[0065] The following further clarifies the present invention in conjunction with specific embodiments. The embodiments are implemented on the premise of the technical solution of the present invention. It should be understood that these embodiments are only used to illustrate the present invention and not to limit the scope of the present invention.

[0066] The single-stage generative sequential recommendation method provided in this embodiment includes the following steps:

[0067] S1: Item Encoding Representation: Represent the items in the form of attribute granularity and generate the encoding representation of the items through a large language model and product quantization technology;

[0068] The first problem to be solved in constructing the generative sequential recommendation framework is the representation of items. The item encoding representation in this embodiment integrates product quantization technology to meet the requirements of generative sequential recommendation.

[0069] S11: Split the items into multiple groups of text tokens according to attribute granularity and input them into the large language model to generate the original embedding vectors;

[0070] Since each item carries different attribute information (including title, brand, color, etc. in this embodiment), the items are first formalized in the form of attribute granularity, specifically:

[0071] ,

[0072] where i represents the item; , , all represent attributes.

[0073] Split the text information in the attribute into the smallest processable unit, i.e., token. Taking as an example, each attribute can be represented as:

[0074] ,

[0075] where, represents the length of the attribute, that is, the number of tokens included in the attribute ; represents the first token in the attribute , represents the -th token in the attribute.

[0076] Then, input the text information in all attributes into the large language model to obtain the original embedding vectors; for the original vectors obtained after embedding, taking the attribute as an example, it can be represented as:

[0077] ,

[0078] where, represents inputting the text information in the attribute into the large language model; d represents the dimension of the original vector .

[0079] S12: Encode the original embedding vectors using Product Quantization (PQ) to generate integer codewords;

[0080] After the embedding operation, it is also necessary to use product quantization to encode the original embedding vectors. In the present invention, the product quantization technology can not only reduce the vector dimension, but also the generated integer codewords are convenient for use in the subsequent generative module. In this embodiment, the formal representation is replaced by the abbreviation PQ (Product Quantization). The encoding operation of the original embedding vectors can be expressed as:

[0081] ,

[0082] where, , both represent encoding the original embedding vectors using product quantization; The elements of can be called "codewords" in this embodiment, represents the number of codebooks in the product quantization process, which is also the number of codewords obtained by encoding.

[0083] If the encoding results of product quantization are directly used, there will inevitably be duplicate codewords in different dimensions, which will interfere with the training of the generative model and cannot be decoded into the corresponding results when outputting the results, resulting in the phenomenon of information collapse. To prevent the above problems, a codeword offset method is designed in the present invention to ensure the mutual exclusivity of codewords.

[0084] S13: Introduce a codeword offset method to ensure the mutual exclusivity of codewords;

[0085] Suppose the model introduces a total of attributes, and each attribute uses codebooks for encoding. Starting from the first codeword of the first attribute, each codeword is offset according to the following formula:

[0086] ,

[0087] where refers to the th codeword in the th attribute. The on the right side of the equation is the original codeword after product quantization, and the left side is the new codeword after offset. The new codewords obtained through the offset formula can ensure that there are no identical codewords in different dimensions of different attributes. Here, all the codeword sets are defined as the vocabulary, denoted as .

[0088] After adapting the project text information, the large language model, and the product quantization technology to the generative model, the encoded representation of each project can be obtained:

[0089] ,

[0090] where represents the result obtained for each project through the steps in this section, represents the encoded representation of project I, represents the th codeword in the th attribute, the th codeword in the th attribute, represents the th codeword in the th attribute.

[0091] Next, based on the existing encoding methods and results, how to construct the user sequence required for generative sequence recommendation will be introduced.

[0092] S2: User interaction sequence construction: Split the user interaction sequence into multiple attributes to form an interaction sequence composed of project attributes;

[0093] The form of the user sequence used in this embodiment is different from the existing methods. In this application, each item on the sequence is split into multiple attributes in place, and the interaction sequence originally composed of items will become an interaction sequence composed of item attributes. The formal description can be expressed as:

[0094] ,

[0095] where, represents the user interaction sequence at the item granularity; represents the user interaction sequence at the item attribute granularity; represents the attribute of the item in the sequence of the first number; represents the attribute of the item in the sequence of the first number; represents the attribute of the item in the sequence of the first number; A represents the attribute granularity; u represents the user; I represents the item; represents the number of items in the interaction sequence.

[0096] According to the encoding step S1 introduced before, the encoding of the interaction sequence is obtained as:

[0097] ,

[0098] where, represents the encoding of the user interaction sequence; A represents the attribute granularity; u represents the user; represents the encoding of the attribute in the first item; represents the encoding of the attribute in the first item; represents the encoding of the attribute in the first item; represents the number of items in the interaction sequence; I represents the item.

[0099] Finally, after setting a sequence length, padding and truncation operations are performed for different situations.

[0100] S3: Generative sequence recommendation: Improve the encoder architecture of the Transformer, including a sequence projection layer, a spatial aggregation layer, and a transformation layer, to adapt to the sequence recommendation task;

[0101] Since the input sequence length will increase exponentially after splitting the interaction sequence according to the attribute granularity, the model needs to process a large vocabulary and data, and be able to capture the user's preferences on the long sequence.

[0102] In this embodiment, the encoder architecture of the Transformer will be improved to better meet the requirements of the sequential recommendation task. The improved architecture is divided into a sequential projection layer, a spatial aggregation layer, and a transformation layer.

[0103] Based on the three classic matrices of the self-attention mechanism, a matrix is added. The purpose of this matrix is to compress the long-term historical behavior sequence information of users and extract the features of users on the attribute-level sequence. The formal representation of the sequential projection layer is:

[0104] ,

[0105] ,

[0106] where X represents the input sequence, represents the matrix, represents the value matrix; represents the query matrix; represents the key matrix; represents the input representation of each learning module, is a multi-layer perceptron, designed as a single-layer linear layer in the model; is the activation function selected in the model, which can be simply understood as the input of the Sigmoid function multiplied by the output. This activation function is smoother than the ReLU activation function and is suitable for optimizing more complex models. represents the input of the SiLU function; The operation is to split the output result into four sub-vectors for the corresponding subsequent processes. In this embodiment, U, V, Q, and K represent matrices, which are parameter matrices that can be learned during the model training process. U(X) represents the new vector obtained by multiplying the input vector with the corresponding matrix and participating in the subsequent processing of the model.

[0107] In the spatial aggregation layer, a new aggregated attention mechanism is constructed to replace the scaled dot-product attention mechanism based on SoftMax in the Transformer. The purpose of this processing method is to remove the aggregation weight constraint in the sequence ( ), so that the model can better retain the interest intensity of users. The formal representation is:

[0108] ,

[0109] where X represents the input sequence, represents the part with V(X) removed on the right side of the formula; represents the value matrix; represents the transpose operation; is the activation function selected in the model; and are the attention biases that introduce temporal and positional information respectively, divided by This scaling operation also prevents numerical explosion caused by too high dimensions.

[0110] The transformation layer can play roles such as feature extraction, feature crossing, and representation transformation. The specific formal description is as follows:

[0111] ,

[0112] where X represents the input sequence, is also a single-layer linear layer, is the layer normalization operation; represents the formalization of the transformation layer; represents the formalization of the spatial aggregation layer; represents the exclusive NOR operation.

[0113] The above-mentioned feature extraction function can convert the original attribute information into an attention-aggregated embedding vector after being processed by each layer of the network. Feature crossing is that the output of the attention layer in the above formula directly interacts with other features. Representation transformation is to virtually perform a gating operation similar to that in a mixture-of-experts network through element-wise dot product.

[0114] Stacking method: Stack multiple learning modules, combine residual connection and layer normalization to increase the depth of the model.

[0115] The formal representation of each learning module has been shown from the formula in the sequence projection layer to the formula in the transformation layer. When performing the stacking operation, it is also necessary to normalize the input of each module and add a residual connection. Specifically, it can be described as:

[0116] ,

[0117] where, represents the input of the first layer when stacking modules; represents the output of stacking to the l-th layer; represents each learning module (Learning Block); represents the output of stacking to the (l - 1)-th layer, which is also used as the input of the l-th layer in the formula; represents the layer normalization operation; represents the number of layers of the stacking operation.

[0118] So far, the learning modules and their stacking methods in the generative architecture have been introduced.

[0119] S4: Recommendation item prediction: Based on the user interaction sequence, learn the probability distribution to generate the prediction of the next possible interaction item.

[0120] In this embodiment, it is a single-stage generative sequence recommendation method and a single-stage recommendation process.

[0121] First, after obtaining the encoded representation of the user interaction sequence in S2, a simple embedding operation needs to be performed on it before it can be input to the model for processing. The dimension of this embedding operation is not high and can generally be set to about 64 dimensions. The embedding operation is expressed as:

[0122] ,

[0123] where, represents the encoded user interaction sequence after embedding processing; represents the embedding operation; represents the encoding of the user interaction sequence; after embedding, can be input into the learning module for further processing.

[0124] Next, after the processing of the previous steps, the representation of the item and the user interaction sequence are already composed of individual codewords. This process is also similar to encoding the text information after word segmentation in the field of natural language processing as the input of the generative model. Therefore, the goal of the generative sequence recommendation model can be described as: based on the known user interaction sequence, learn a probability distribution, select the codeword with the highest probability as the output, and obtain the prediction of the next possible interaction item after multiple generations. The formal description of the above goal is as follows:

[0125] ,

[0126] where, represents the user interaction sequence at time, represents the codeword predicted by the model for the next time; K represents the vocabulary; p represents the probability.

[0127] The probability value is calculated by performing a softmax operation on the output of the stacked learning module and each element in the vocabulary, specifically as:

[0128] ,

[0129] where, represents the user interaction sequence at time, represents the codeword predicted by the model for the next time; represents after passing through The prediction result output after stacking the layer learning modules represents the vocabulary and the embedding vectors of all codewords in it.

[0130] As mentioned above, the length after encoding each attribute information is , and the number of attributes considered in the model is , so the number of codewords representing each item is . Therefore, to enable the model to generate predictions for the next interaction item, the number of codewords that need to be generated must be greater than . This goal is similar to the sequence-to-sequence task (Seq2Seq) in the field of natural language processing. The algorithm model in the present invention can also output a sequence of a specified length (which needs to be a multiple of ) to meet the recommendation requirements of Top@N.

[0131] In this embodiment, two datasets in different fields on the online e-commerce platform Amazon are selected for experimental evaluation, namely Office Products (Office) and Cell Phones and Accessories (Cell). The evaluation metrics are Recall@K and NDCG@K, where K = 10, 20.

[0132] The experimental results show that compared with the existing models, there are improvements in all four evaluation metrics, but the improvement effects on different datasets are not the same. The bold data in Table 1 are the model data with the best performance in the corresponding metrics. For the model data with sub-optimal performance, they are also underlined at the corresponding positions in the table.

[0133] Table 1 Comparison table of experimental results

[0134]

[0135] It can be found from the experimental data in Table 1 above that the performance of GenSR is higher than that of all baseline models, but the improvement amplitudes on different datasets and different experimental metrics are different. From the numerical values of the experimental metrics, the overall performance of GenSR on the Office dataset is higher than that on the Cell dataset.

[0136] The method of the present invention utilizes the attribute information contained in the project to split the user's interaction sequence into sequences at the single-attribute granularity. Then, a large language model is introduced for text information embedding operation, and the product quantization technology is used to encode the original vectors. In order to make the encoding result meet the requirements of the generative model, the present invention constructs an encoding offset method to ensure the mutual exclusivity of the codewords in different attributes. At the same time, combining the ideas in some classic works in the field of recommendation systems, a stackable generative learning module is constructed, enabling the model to deeply learn the personalized preferences on the user interaction sequence and providing more suitable recommendation results.

[0137] The above are only the preferred embodiments of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present invention.

Claims

1. A single-stage generative sequence recommendation method, characterized in that: The following steps are involved: S1: Item coding representation: The items are formally represented according to the attribute granularity, and the coding representation of the items is generated through the large language model and product quantization technology; S2: User interaction sequence construction: split the user interaction sequence into multiple attributes to form an interaction sequence composed of item attributes; S3: Generative Sequence Recommendation: Improve the Transformer encoder architecture, including sequence projection layer, spatial aggregation layer and conversion layer, to adapt to the sequence recommendation task; S4: Recommended item prediction: Based on the user interaction sequence, learn the probability distribution and generate the prediction of the next interaction item.

2. The single-stage generative sequence recommendation method according to claim 1, characterized in that: The specific process of S1 is: S11: Split the project into multiple groups of text tokens according to attribute granularity, and input them into the large language model to generate the original embedding vector; Projects are formally represented by attribute granularity as follows: ， Among them, i represents the project; , , Both represent attributes; Split the text information in the attribute into the smallest processable unit token, attribute Specifically expressed as: ， in, Representation attributes Length; Representation attributes The first token in Indicates the first Tokens; Then input the text information in all attributes into the large language model to get the original embedding vector; attribute The original embedding vector of Chinese text information after being processed by the large language model It is expressed as: ， in, Indicates that the attribute Chinese text information is input into the large language model; d represents the original vector Dimension.

3. The single-stage generative sequence recommendation method according to claim 2, characterized in that: S12: Encode the original embedded vector using product quantization to generate an integer codeword; For the original embedding vector The encoding operation is expressed as: ， in, , Both represent the use of product quantization to quantize the original embedding vector Coding; m represents the number of code words encoded during the product quantization process.

4. The single-stage generative sequence recommendation method according to claim 3, characterized in that: S13: Introduce a codeword offset method to ensure the mutual difference between codewords; Assume that the model introduces properties, each using Code words are encoded, starting from the first code word of the first attribute, and each code word is offset according to the following formula, specifically: ， in, Indicates The first of the attributes codewords, the right side of the equation is the original codeword after product quantization, and the left is the new codeword after offset; After adapting the project text information, large language model and product quantization technology to the generative model, the encoded representation of each project can be obtained, specifically: ， in, represents the coded representation of project I, Indicates The first of the attributes Code words, No. The first of the attributes Code words, Indicates The first of the attributes A code word.

5. The single-stage generative sequence recommendation method according to claim 1, characterized in that: In S2, each item is split into multiple attributes in situ, and the original interaction sequence composed of items is replaced by an interaction sequence composed of item attributes. The encoding of the interaction sequence composed of item attributes is expressed as: ， in, represents the encoding of the user interaction sequence; A represents the attribute granularity; u represents the user; Represents the first item in the property The encoding of Represents the first item in the property The encoding of Represents the first item in the property The encoding of represents the number of items in the interaction sequence; I represents the item.

6. The single-stage generative sequence recommendation method according to claim 1, characterized in that: In S3, the sequence projection layer is the three classic matrices in the self-attention mechanism On the basis of , the matrix U is introduced to compress the long-term historical behavior sequence information and extract the user's characteristics on the attribute granularity sequence; the formal representation of the sequence projection layer is: ， Where X represents the input sequence, represents the matrix, represents the value matrix; represents the query matrix; represents the bond matrix; represents the input representation of each learning module, is a multi-layer perceptron; is the activation function selected in the model, Indicates splitting the output result into four sub-vectors.

7. The single-stage generative sequence recommendation method according to claim 1, characterized in that: In S3, a new aggregation attention mechanism is constructed in the spatial aggregation layer, and the formalization of the spatial aggregation layer for: ， Where X represents the input sequence, represents the value matrix; Represents a transpose operation; is the activation function selected in the model; and Represents the attention bias of time and position information respectively, Represents a zoom operation.

8. The single-stage generative sequence recommendation method according to claim 1, characterized in that: In S3, the conversion layer can play the role of feature extraction, feature crossover and representation conversion. The specific formalization is as follows: ， Where X represents the input sequence, represents a single linear layer, Representation layer normalization operation; Formalization of the representation conversion layer; Formalization of the representation space aggregation layer; Indicates the same or same operation; represents a matrix; The feature extraction function converts the original attribute information into an attention-focused embedding vector after being processed by each layer of the network; Feature crossing is to directly interact the output of the attention layer with other features; The representation transformation is performed via element-wise dot products, virtually performing a gating operation.

9. The single-stage generative sequence recommendation method according to claim 1, characterized in that: In S4, the user sequence code is embedded, specifically: ， in, Represents the user interaction sequence encoding after embedding processing; Indicates an embedding operation; An encoding representing a sequence of user interactions; The objective of the generative sequence recommendation model is expressed as: ， in, Indicated in The sequence of user interactions at each moment, represents the codeword predicted by the model at time t+1; K represents the vocabulary; p represents the probability; Probability value The calculation of is obtained by performing a softmax operation on the output of the stacked learning module and each element in the vocabulary, specifically: ， in, Indicated in The sequence of user interactions at each moment, The codeword representing the model's prediction for the next moment; express In passing The prediction results output by the stacked layer learning modules, Representation vocabulary The embedding vectors of all codewords in .

Images